Best way to scrape targets for GSA search engine ranker ?

Repans

Hi ,

I used to scrape and feed GSA SER with Gscraper and now seems GScraper is totally dead! (after their proxy trouble) now i am looking for good scraper to scrape my self.

I have tried Scrapebox with 2000 footprints which merged with 200 unique different keywords. And i could scrape over 9 million targets at a time and i after remove dups it decrease to 100K anyone got idea to avoid this ?

totally tiring up with scraping

Thank you guys!

Hinkys

You need to use more unique keywords and less footprints. If you use too many similar footprints (or similar keywords for that matter), you will get similar results which leads to many duplicate urls.

So short version, much more keywords, less footprints.

If you want the longer version, I have a wonderful guide for you: 

http://seospartans.com/scrapebox-scraping-tutorial-easy-56-million-links-day/

Cheers!

1linklist

In a pinch taking your verified list, and using the scrapebox "External link extractor" to pull a bunch of targets - then repeating the process ad-inifinitum, can work.

Eventually you'll start getting junk though.

Beyond that, rigging up lots of custom footprints and keywords into GSA itself can help.

As fars as scrapers go, we use scrapebox for small-level stuff, and Hrefer (Which unfortunately requires you to purchase Xrumer first) for mid-level stuff.

If your any good at coding, you can also setup your own scraper to really pull in the targets, As far as that is concerned check out...

http://nutch.apache.org/
or
http://scrapy.org/