Skip to content

different list of proxies for scraping and posting/account registrations/authorizations?

scrap3rscrap3r aus
edited August 2019 in GSA Search Engine Ranker
I was asking my proxy provider as they provide 3 kind of backbone reverse proxies: one type which changes with each http request (to use for scraping), one type which changes each 3 minutes (for authorizations) and 15 minutes how to input them into gsa. As there is just one list in the setting window can someone tell me how to do this?

i.e. my question is how to configure a different list of proxies for scraping and for posting/authorizations and is this possible at all? Thanks

Comments

  • DeeeeeeeeDeeeeeeee the Americas
    edited August 2019
    "...i.e. my question is how to configure a different list of proxies for scraping and for posting/authorizations and is this possible at all? Thanks" -scrap3r

    What's up, scrap3r? :) Actually, you can use all these proxies at once for different tasks.

    Firstly, don't EVER scrape with 3 minute and 15 minute reverse proxies! ONLY the proxies that randomly change with each request are OK for this!

    (Not a joking matter: If you disregard this rule, your provider will probably get very upset with you.)

    On your panel in your proxy provider web site, there will be different specific gateways (IP addresses) for each of those three sets of backbone reverse proxies.

    You can either enter the scraping proxy gateway IPs into your private list, and the 3 and 15 minute IPs into your public list, or vice-versa. That doesn't matter, but what does is that you set your GSA-SER program to scrape ONLY with the (public or private) set of gateways that changes with each request.

    Do this on the Options>>Submissions pop-up window,  button tog et there is top right pane of SER, between the RED delete (x) button and GREEN start (›)  button. They're check-boxes at the bottom of the (default) menu.

    --_--_--_--_--_--_

    You can also change the proxy location on (at least some) proxy services.

    But..this takes a few minutes for the changes to take effect AND the gateways are different.  So, each of the random, 3 min, and 15 min gateways will all need to be changed if you alter the proxy location (US/worldwide/EU/etc.)

    So, you can mix-and-match your use of random, 3 min and15 min proxies for your verification and scraping tasks, but you CANNOT mix-and-match those same 3 types of rotating backbones proxies for different locations.


    --_--_--_--_--_--_


    Honestly, I've enabled the 3 and 15 minute gateways and used alongside other proxies to post and verify, but I've never used them exclusively. Anyone can help me get more out of them? What specific types of sites do I need to keep the IP locked for 3 or 15 minutes to pass verification?

    So far, I've just used them as additional proxies, just because they're there, and I can access them.  Thanks!~
  • scrap3rscrap3r aus
    edited August 2019
    DeeeeeeeeDeeeeeeee thank you for your answer, that really explains it. Now looking at it again it just looks obvious - I never thought of the checkboxes regarding public and private proxies next to each task (as I have always used private proxies).
    Thanked by 2Deeeeeeee JasonParser
Sign In or Register to comment.