Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

A new Tool is born - GSA Proxy Scraper

SvenSven www.GSA-Online.de
edited July 2015 in GSA Proxy Scraper

GSA Proxy Scraper

imageimageimageimageimageimage

You can find details here: http://www.proxy-scraper.com/

I hope this will help everyone to find more reliable sources of proxies and manage to test them and keep them updated.

  • comes with over 800 sources to find proxies
  • lots of different things to find and locate new sources
  • proxy/port scanner 
  • many options to test against different websites (you can create custom test of course)
  • automatically check for anonymous level
  • give report of a suspicious proxy (in control of a spying company)
  • many filter options like google passing proxies
  • automatic upload of proxy reports (csv, text, html)
  • internal proxy server that can be used in any tool you want to directly use the proxies.

Any ideas, suggestions and bugreport (I hope there are no big bugs) are welcome!

ยซ13456

Comments

  • andrzejekandrzejek Polska
    edited May 2015
    First idea is to import ip ranges in format (like http://www.proxyfire.net/) - i think thats the only effective tool atm at the market to scan for open ports in windows.

    Also make ip ranges based on countries or many proxies. (so we can import them, not only using existing ones)

    image

    Here is proxyfire format:

    115.184.100.60 115.184.106.190 #1666 hosts
    115.184.108.20 115.184.114.195 #1711 hosts
    115.184.117.255 115.184.121.255 #1024 hosts
    115.184.183.191 115.184.187.191 #1024 hosts
    115.184.17.157 115.184.21.157 #1024 hosts
    115.184.217.158 115.184.223.161 #1539 hosts
    115.184.28.224 115.184.32.236 #1036 hosts
    115.184.39.17 115.184.45.20 #1539 hosts
    115.184.49.241 115.184.59.35 #2354 hosts

    Right now i got no idea how to import these ranges here image

  • SvenSven www.GSA-Online.de

    115.184.100.60 115.184.106.190 #1666 hosts

    would become...

    115.184.100.60-115.184.106.190

  • SvenSven www.GSA-Online.de
    edited May 2015
    Anyway next version will allow you to copy/paste this (just added it).
  • andrzejekandrzejek Polska
    edited May 2015
    Thanks a lot :) already what can i say, you guys make awesome software that beat everything that is at the market now thanks again :)

    Is there any randomization when testing proxies? So we can avoid netscan detection.

    Also whats the option Has to reply on ping? Does it ping the ip on specified port first?
  • SvenSven www.GSA-Online.de
    yes but i guess it can be improved some more.
  • What i am aiming for is,
    get ip ranges that generate tons of google passed proxies, (myself)
    loop the testing only for google passed
    and export into my ftp. (i know its possible)

    Is it already possible at this stage ?
  • SvenSven www.GSA-Online.de

    yes all of that thats possible.

    - export automatically to ftp is working including a filter for google passed only

    - re-testing proxies can be limited to google-passed proxies as well but I would use the filter on export only

  • andrzejekandrzejek Polska
    edited May 2015
    how can we make the loop of scanning ip ranges for google pass go automatically?
    Also is the port scanning limited somehow in demo? looks like it take 4 sec to test 900 targets (with 4sec timeout)
  • SvenSven www.GSA-Online.de
    oh you want to scan certain ip ranges all the time? Is that useful? Thats something I have not added. Maybe you can explain in pm why this is something you need...
  • I dont have to explain in PM, you know that getting google passed proxies is really important if you are scrapping right. There is a lot of search engines but they dont "know" so much that google does. Public proxies are dying fast, if they are avaiable to everyone - they are banned in google most of the time.

    There is many services where people sell port scanned google passed proxies.
    Some ranges are able to generate loads of google passed proxies, the key is to find these ranges and scan them 24/7 then export google passed to your scraping tool. 
  • SvenSven www.GSA-Online.de
    So much is clear but why is it so important to scan the same ranges again and again? Once scanned I would move on to the next and maybe scan something again weeks later?
  • andrzejekandrzejek Polska
    edited May 2015
    when i say ranges i mean maybe milions of ip's to scan you see, thats why loop is important, to make it automatic so my scraper can go for weeks afk

    some proxies are up only on certain hour's ... etc. etc.

    EDIT: Of course best solution would be to create projects like in gsa platform identifier.

    so project a) is scanning ip ranges for open ports (proxies)
    project b) is testing proxies from project a) for google pass (each lets say 60-180minutes)

    project a) runs only if there is less than 3000 google passed proxies in project b)
  • SvenSven www.GSA-Online.de
    well the only thing that is missing is a loop in proxy scanner
  • So let it be, thats my feature request :)
  • SvenSven www.GSA-Online.de
    will add it then ;)
  • andrzejekandrzejek Polska
    edited May 2015
    Thank you, btw. maybe your next product will be something to scrape search engines?:)

    btw. internal proxy server will randomize the proxy each time?

    EDIT: is it possible to use proxy port scanner via socks proxies? Would be cool, to prevent netscan detection :)
  • SvenSven www.GSA-Online.de

    - loop function added for next update

    - internal proxy server uses a new proxy on each request

    - using a proxy for proxy scanning/testing is something i can add as well, but maybe not for next update

  • Thank you, will be waiting specially for last thing beacuse hetzner is pain in the a. when it comes to loads of request on different ports to ips in same ranges
  • SvenSven www.GSA-Online.de
    just added ability to use a proxy for port scanning.
  • andrzejekandrzejek Polska
    edited May 2015
    Missing changelog in HELP my dear  8-}

    Thank you for the feature - actually we can use only 1 proxy, is it too much hassle to import many proxies and randomize it? I mean, too much hassle in coding?

    EDIT2: are you sure proxyfire format from clipboard is working? Looks like for me its not.
  • SvenSven www.GSA-Online.de

    - naw of course using many proxies is not a problem. can add that as well.

    - change log menu is indeed missing

    - will recheck proxyfire format (coded blindly and never checked as i thought it would work :/ )

  • What is the difference between GSA SER internal proxy checker and this proxy checker ?
  • Just a question - what about just checking proxies via proxy? (like checking if proxy passes google check via proxy).
  • andrzejekandrzejek Polska
    edited May 2015
    Cant edit post, i cant find option to import from file. I can add sources from urls, but files with proxies? (it will be reloaded each x minutes by other tool)

    Also why after adding proxies its automatically checking them?

    Did you test proxy scanning on 5k-10k-20k of threads? Looks like its stuck and slow, maybe theres a limit?
  • SvenSven www.GSA-Online.de

    @useruser1 the Proxy Scraper is a complete rewrite. It is way faster, offers a lot more functions and has many many new sources and ways to get proxies.

    ---

    Checking proxies via proxy...whats the use in it? It would cause a lot problems if that check.proxy is down

    ---

    it re-checks proxies in intervals. Youc an configure this to happen or not...or limit it to certain tagged proxies

    ---

    proxy providers can also get content from local files in next update (must have forgotten about this)

  • is there any "early bird" discount code? :)
  • halyosyhalyosy jakarta
    ^^
    count me in if there's any early bird discount code :D

    pretty please @sven
  • SvenSven www.GSA-Online.de

    Common, this tool is fresh and you want a discount already :/ It's better than any other proxy tool you will find...that should be enough to get your attention ;)

  • @Sven this is why I called it "early bird discount code" as it is fresh tool :)
  • Finally! However i did download it on my VPS and it just doesn't start.

    Any idea what's wrong?
    A discount would be great yes, if not at least remove the VAT from my purchase ha :).
    Otherwise i need to pay extra :).
Sign In or Register to comment.