Proxy sources list - delete all sources with X results, remove source after X attempts
I would like to be able to remove proxy source sites in the sources list according to how many results they return. I have tried to be clear by proving an example in plainer english for each feature request.
Firstly an option to remove sites that have returned a number of proxies below or above a user defined value eg remove sources that returned 100 or less sites last scrape.
Would also like this to be switched to above X results eg. remove all sites that returned 10,000 or more sites last scrape.
And it would also be nice to have some kind of automated way of pruning the list of non-working sites implemented according to user values. eg.
automatically remove sites after 10 tries without proxies.
Having a "new proxies" or "doesn't matter" option here would also be very handy eg delete. remove site from the sources list if it fails to get any new proxies (regardless if it got proxies we already have) 3 scrapes in a row.
Firstly an option to remove sites that have returned a number of proxies below or above a user defined value eg remove sources that returned 100 or less sites last scrape.
Would also like this to be switched to above X results eg. remove all sites that returned 10,000 or more sites last scrape.
And it would also be nice to have some kind of automated way of pruning the list of non-working sites implemented according to user values. eg.
automatically remove sites after 10 tries without proxies.
Having a "new proxies" or "doesn't matter" option here would also be very handy eg delete. remove site from the sources list if it fails to get any new proxies (regardless if it got proxies we already have) 3 scrapes in a row.
Comments