GSA Proxy Scraper - Parse extracted links from search-engine-tests
I am trying to uncheck Parse extracted links from search-engine-tests in the Provider section because I have the impression that I have no idea what it really does, and I wanted to see how it performs without it, but each time I restart the Proxy Scraper, the option is activated.
Is this a bug or a feature? ) Because it has a checkbox, which is usually used to check or uncheck things )
Let me however show you what this is doing:
When a Test of a proxy is performed against some search engine, it is doing a real search and not just opening the homepage. This real search is done with a known proxy IP/host in the hope that a new site is listed among the results where more proxies are listed.
The results get collected to a file and later, when this option is used, the program might be able to find new proxies. In the source column you will then read "Proxy-Search Links - <url>".
The program will only extract there if it's not from a known source.
However the time can differ a lot according to the sites it finds.
if you test with SER against anything else but bing you get them as not working.
the reason is some kind of bing-cache system where you can use those servers as proxies as well. i tried to understand this back than but didn't came far. i just accepted it and carried on.
to speed this up you can lower the timeout a bit...5seconds are not used much and if, then the proxy is probably unstable anyway. You can of course higher threads and also disable the provides with low success rate (sort by that column and disable the once at the end).