Skip to content

Search online for URLs

AshishAshish India
in Need Help
Hello everyone

I want to scrap URLs with GSA SER inbuild feature i.e. Search online for URLs (Option > Advance > Tools > Search online for URLs)

I am planning to do it with 10 Dedicated Buyproxies. (I know many of you will tell that is not enough, but I am with limited budget and newbie too). So here is my prob. how many thread should I have to run (Just for scraping no other task would have been done at that time). I am thinking about 2-3 thread with 90 sec. HTTP timeout. (I know it takes a lot of time, but I have a niche which has fewer keywords). But the main thing is I don't know that the thread limit which I am going to set will work on this feature too?

As far as proxies are concern I am thinking about ReverseProxies with 5 port monthly plan. They told me that I can use 50 threads with GSA. Any other Proxy suggession are also acceptable.

So can anyone guide me the 

1) What is the best practice for the Scraping URLs for GSA

And

2) What should I do from the above "proxies restrictions" and "thread with Http timeout" settings

All the suggestions are welcome.

Thank You
Sign In or Register to comment.