New Google Scraper About To Come!

BlazingSEO

For the past couple months my team has been working on our very own custom Google scraper. Now, unlike the others in the market (you know which two I'm talking about), this one is much... MUCH, different.

Unique Features:

- Web-hosted -- Login and scrape from anywhere in the world! This means no more server costs and no more worrying about Windows giving problems!
- No proxies needed! - That's right, we handle ALL proxy usuage! Imagine all the money you'll save
- Faster scraping - Since this is web-based, the speed of our scraper is determined by the depth of our pockets... meaining, the more servers we buy, the faster we will scrape! Imagine getting 1 million URLs returned in a matter of minutes!!
- Automatic 24/7 scraping - This by far is our most proud feature. With certain algorithms in place, we will intelligently take your list of footprints+keywords and scrape for you INFINITELY. This means that once you upload a handful of footprints+keywords, you can leave our website and come back 2 months later and it'll still be scraping new links with new footprints+keywords it finds for that platform you desire.
- Auto-FTP - Now you couldn't have infinite, 24/7, scraping without an auto-FTP feature could you?! We allow users to type in an FTP account for the results to be saved to automatically... the possibilities are endless once you have our automatic scraper and auto-FTP setup!


This post is not an official release, however. I'm making this discussion to see the amount of interest that we can expect to hit our servers upon release. Also, with that, we would like to know what YOU as a customer would be willing to pay per month to use a system like this. More than likely we will limit users by the # of links returned (don't be scared by that... it'll be something like $1.00 for 100xxxxxxx+ links). We know you can argue that you can use our competitors for "free", but don't forget the monthly Windows server cost and proxies you have to worry about on a daily basis .

Thanks for your time, looking forward to your suggestions, feedback, and comments!

gooner

Sounds awesome, can't wait to see it in action.
Personally, i'd pay $50 - $100/month for something like this.

Ferryman

Signing up for a testdrive :P

cefege

Sounds awesome! Need beta testers?

Kaine

Very good idea

You must have a very (very) huge list of good proxy for this work.

meph

This would be great, wanna test it

I am using Gscraper with their proxy service, so i am willing to pay $50-$70 - depends of speed

Nikodim

Would be great if service could be scalable to different needs. $1.00 for 100xxxxxxx sound neat. :@)

killerm

sounds good..can invest around 100 bucks a month for scraping as per above specs

Vanity

That sounds really good.

BlazingSEO

Thanks everyone for all the support and feedback!

@cefege - Everyone who posts in here (up to a certain point) will be given free beta tester access in the next couple weeks when we're ready to roll out. So you're already one step ahead

donaldbeck

I'll test it.

Tim89

@BanditIM I always knew you were going to bring something that was innovative out after that private conversation we had a while back.

sounds good, well done! hope it works out for you.

Justin

This will be very useful. I don't think I'll ever stop scraping on my own, but there are always times when all of my servers are occupied and I run across an awesome footprint I want to scrape and have to wait until one of my servers frees up and then end up forgetting about it all together. This will be great for those one off scrapes that I have sitting on a checklist somewhere. Are you going to be collecting a central database of all scraped links? 

Brumnick

sounds fantastic!

satyr85

There are few problems with such service:

1. User have links, and you have links

2. Most people who harvest targets to GSA use same footprints so you dont need to scrape same targets 1000's times. Scrape once, than give same results to XX-XXX members. Sounds like different way to buy a list

$1.00 for 1 00x xxx xxx+ links

So 1$ for 1000kk /1bilion links

I have own public proxy service - 2 serves scanning whole internet and looking for google proxies 24/7. It gives me 3-5k google passed proxies every 15 minutes, 24/7 and i can scrape with this service (using one gscraper license) 1 bilion links per 7 days (results could be better but footprints are shitty). Maybe if i improve my system i could scrape 1 bilion links in 1-3 days and you want for this (1 bilion links) only $1 ? 

Not possible

Why? Because number of public proxies is limited. You would need acces to huge bootnet and even with that costs of servers would be huge.

Just my 2 cents.

Anyway best of luck

davbel

I'll be happy to test too

Destroyerror

leave post to be beta tester

Brandon

I don't want to pay per link, I want to pay for a monthly service like I'm doing with GScraper. GScraper works VERY well for me, but if you can harvest more that would be worth a little more than I'm paying right now with GScraper. My installs run on servers I already own so my cost is just the monthly proxies from Gscraper.

Hjunc

Really interested how you want to scrape that much...

Anyways, same situation for me as Brandon. Servers are running anyways and i'm only paying for the proxy service.
Would also rather pay for a fix monthly amount instead of per link.

fakenickahl

$1 for every billion urls scraped??? Please take my money

BlazingSEO

@Tim89 - Thanks for the encouraging words, much appreciated!!

@Justin - That's up in the air yet. We realize people are very 'etchy' about sharing their scraped lists, and we respect that. We will NEVER sell the links scraped, what we will do to make our system more efficient is cache the links scraped by another user and provide them to a newest user scraping them. After a certain period (a few days most likely, to be determined), we will re-scrape if a new user requests the same footprint+keyword as a previous users a few days ago. This is how we will save costs because after a while we should have a pretty large database of links 'cached' that we don't have to go out and scrape for - we can simply give it to the user right away. Why waste resources when we can just reuse the same links?

A note with this -- No other use will know what footprint+keyword you scraped. The only way the cached results will be returned to another user is if they request the EXACT same search phrase. It'll all be completely anonymous and private, you won't know if your links are from a cache result or from an actual scrape.

@satyr84 - Read above comment first . 

Regarding pricing, the figure I threw out (1 billion as everyone seemed to interpret it as), was just a random number I typed to signify that we will be able to provide a LOT of URLs for a very little price. We aim to provide as many URLs as you would be able to scrape on Gscraper/Scrapebox in one month, for less than the price of what you'd pay for proxies + some server costs for gscraper. Also, keep in mind the strategic planning we have about links with my comment above.

Thanks for your comments though, I appreciate it.

@Brandon and @Hjunc - I understand your worry here, and it's something we are STILL going back and forth with. If we were to offer a simple monthly payment with no restrictions on links, then we'd have to limit on # of threads you'd use... we are leaning away from that because then that limits you on SPEED - which is the biggest issue with scraping massive amounts of links with the other software. Regardless, we aim to be able to charge per link at a price that would be similar to the amount of links you could scrape with GScraper in a month's time. So, for example purposes, you pay $60/mo for GScraper proxies (we'll neglect server costs for right now), and can scrape 500 million URLs a month. That would equate to:

$0.00000012 per link

If we charged $55/mo for 500 million URLs, you're getting a better deal with us - AND the best part is... it won't take our system a month to scrape that many URLs since our infrastructure is so scalable .

NOTE TO EVERYONE - These numbers are all for example purposes (especially the 1 billion for $1 haha). We will determine a more accurate pricing structure when the time comes after beta testing.

Keep up with the comments and suggestions everyone, everything will be considered and taken into account for upon release. Thanks again

Kaine

Personnaly i scrape really more : P

For exemple i have in one hour (no deduped) 426mo list for tier1.

I know by experience https://forum.gsa-online.de/discussion/9696/big-scrape-domain-by-extension-reward result are really different between those who scrape (X100).

We need to say more on the number of proxy that you will use, and the number of dedicated machine. 

Information of your potential scrape (but your project is interesting).

Say you have 100 clients in the beginning. If scrape every member scrape differente things, you can not use "cache". 

How it going to happen?

BlazingSEO

@Kaine - Hey, thanks for your questions and comments!

You're scraping 426 million URLs in one hour using one server of GScraper or Scrapebox? That blows my mind....

Can you explain more how the big-scrape-domain results are different? Are they faster results? More results? How so?

I'd rather not say what proxies or how many proxies/servers we will be using, but you bring up a good point about the first 100 users at the beginning. First off, we plan on running the system in beta testing for a good week, so with everyone that is a beta tester using our 24/7 scraper, our cache will build up fairly quickly. As for still giving a reliable speed for all users, our guarantee upon release will be that we are CHEAPER and FASTER than GScraper for most users - which will hopefully bring more customers to us instead of paying them (because who doesn't want cheaper and faster? ). As for the "not-most users", such as yourself that are scraping 430 million URLs an hour, I can't make such guarantees and maybe our service isn't for you. You have to remember also that we bring in a valuable 24/7 scraping feature, as well as auto-FTP, which will be two very desirable features that users will want to look at .

Brandon

@BanditIM one of the biggest problems with the way you're describing is that I remove dupe domains when scraping. Your service would need to give me the full list of it wouldn't be worth the money so having a little extra control is necessary for me.

BlazingSEO

@Brandon - Already ahead of you on that one . We offer removal of dupe-URLs and dupe-domains after the scraping has finished . Obviously, as time progresses, we will offer more of the features that Scrapebox/gscraper offer with their 'extra tools'... right now we just want a fast and reliable scraper, which is what our service is. We saw the need for dupe-url/dupe-domain removal, so we added that in as soon as possible. 

JudderMan

God dammit. Just bought Gscraper, proxies and a dedicated server like a week ago.

Interested.

Kaine

I meant 420 mo (ko/mo/giga) maybe 10 million by hour (Gscraper).

It is a good idea this beta testing for generating the cache

BlazingSEO

@JudderMan - No worries, we have some things to finish up yet, and still need to go through the beta testing. One thing we're making sure we really have down right is the proxy usage. Right now we're just getting as many DIFFERENT proxy providers in our system as possible so that if one goes down or becomes unavailable, there's plenty more to back it up.

@Kaine - Thanks . I'm sure I could help out your big scrape domain project if you're interested in working some deal between each other. I could scrape a lot of those domains for you . Shoot me a PM with some thoughts.

Kaine

Just time for cooking, i pm you after;)

tommy99

Sounds great you can count me in. I would be willing to pay between 50-100.00

BlazingSEO

@tommy99 - Thanks tommy for the feedback! I'll send PMs out to everyone when beta time is ready