Scrape Box Google scraping settings + Next Best engines after google to scrape own list or URLs

Ashish

Hi 

1) I am using Scrapebox with 20 Semi-dedicated proxies. Google is keep blocking to scrape with 2 threads and 120-second timeout in custom scraping and even in detailed scraping 300-second delay. I am just fed up with that.

If anyone here having any suggestion regarding setting then plz comment.

2) And one more thing, what if scrape URLs from other engines like yahoo or bing because they have low IP banned prob. If I am scraping from yahoo or bing and load them in SER then, Can I rank my site/post on my keywords on Google?

Give your opinions

Regards

Johan

1. Scraping Google will burn your proxies

2. Yes, scrape Bing or another search engine - you will be able to rank in Google if those sites are indexed in Google.

All you need to do is hit the 'check indexed' button in the side menu on Scrapebox and filter out the non Google ones.

Hinkys

Excuse the self-promotion but here's an extensive tutorial that's going to answer all your questions:

http://seospartans.com/scrapebox-scraping-tutorial-easy-56-million-links-day/

TL;DR;

1) It's really hard to scrape Google reliably these days, you need TONS of proxies for this and imo it's not worth it.

2) Scrape bing, you can get insanse speeds with just a few private proxies. If you're worried about getting too many links that are not indexed in Google (it's not a lot, it seems to be around 10% or less),here's what you do.

Create a test project and try to post to all sites you just scraped. Save the verified URLs, trim to root and then check which of those links are indexed in Google (and remove the ones that aren't).

710fla

@Hinkys thanks for the info! When checking to see if the domain is indexed, what do you set the delay too? I would be using Scrapebox Index Checker.

Do you use your own internet connection or do you use Google passed proxies?

Hinkys

@710fla
I don't do it anymore tbh, just use the list as is but I think it was something like 5 seconds with equal number of threads and private proxies.

Google seems to not mind checking if something is indexed as much as scraping.

Ashish

@710fla I have tested indexing with 20 semi dedicated proxies. 8 threads with 10 second delay it works without any problem. I haven't check it with more threads.

710fla

@Hinkys @Ashish thanks guys right now I have 50 shared proxies I was thinking of running 1 or 2 threads. See how that works out.

I also have GSA Proxy Scraper so I can gather some Google passed proxies too. I'm going to do some testing with them too, but I'm afraid they're going to end up dying quickly.

Tim89

You're going to need an abundance of proxies at your disposal to scrape lists effectively, trust me, I scrape my own lists too.