Skip to content

Domain name in CONNECT request instead of IP

BarsikBarsik
in Feature Requests
Hello,

Would like to ask GSA SER to stop sending IP address instead of domain name in the CONNECT request. When I'm trying to add my proxies manually, it sends several requests to verify them. Having IPv4 address in CONNECT request does not allow me to use cheap IPv6 address space. This also results in using the IP resolved at the location where GSA SER is installed instead of the geographic location of the proxy server.

Instead of: 
CONNECT 216.58.209.174:443 HTTP/1.0\r\n\r\n

should be: 
CONNECT google.com HTTP/1.0\r\n\r\n
It is also very desirable to have HTTP/1.1 instead of HTTP/1.0

Comments

  • SvenSven www.GSA-Online.de
    There is an option for this you need to turn on called "Proxies should resolve domain to IP".
  • BarsikBarsik
    Sven said:
    There is an option for this you need to turn on called "Proxies should resolve domain to IP".

    Thanks for a fast reply, this feature seems not to be working. Even after the app restart, it continues to send connect with IP address.



  • SvenSven www.GSA-Online.de
    I have just reviewed everything and there really is no IP sent when this option is on. Can you investigate from what requests this is coming (e.g. submission, verification, search, proxy scraping) ?
  • BarsikBarsik
    edited August 2022
    v16.51 when I manually add a new proxy (Proxy List -> List tab -> Add Proxy -> Import from Clipboard host:port:login:password), the proxy is receiving several requests from GSA SER. All except GET and CONNECT are immediately denied by design, and my proxy processes the following requests:
    CONNECT 216.58.209.164:443 HTTP/1.0\r\n\r\n
    GET https://www.google.com/search?q=duckduckgo HTTP/1.0\r\n ....
    This is the proxy verification right after importing from clipboard. I answered "Mark private -> Yes, Verify -> Yes"

    GET fails too, but this is a different story. The same requests are sent when I use the previous menu item (login:password@host:port)
    CONNECT 216.58.209.164:443 HTTP/1.0\r\n\r\n
  • SvenSven www.GSA-Online.de
    Ahh ok thats the proxy test itself. OK there this is not used, but I will fix this in next update.
  • BarsikBarsik
    Sven said:
    Ahh ok thats the proxy test itself. OK there this is not used, but I will fix this in next update.

    v16.52 does not solve this, I'm still receiving IP instead of domain name in CONNECT request :)



  • SvenSven www.GSA-Online.de
    sorry, I can't seem to see any of that here. Can you send me the proxies you use and what exactly you click to get to this point?
  • BarsikBarsik
    My proxy server is in development and has no authentication right now. I run it privately in VirtualBox instance and my router has a dynamic IP. So, testing it from your side will be painful.

    The problem is when I manually add a new proxy (Proxy List -> List tab -> Add Proxy -> Import from Clipboard host:port:login:password), or import many proxies in a batch, the proxy checker in GSA SER sends pre-resolved IP instead of a domain name when it's trying to check the added one as Connect proxy.

    When I select Proxy List -> Options -> Test Proxies -> Test to perform -> Google (Search), it sends:
    CONNECT 216.58.209.164:443 HTTP/1.0
      instead of:
    CONNECT google.com:443 HTTP/1.0
    The Proxies should resolve domain to IP in Proxy List -> Option is not working for proxy checker feature. It seems to work for scraping, but not for the checker. I've tested GSA SER in scraping on Google and Bing today. Every CONNECT request comes with domain name as expected. But to add my proxies in the list I enabled only the Connect in Accepted Proxy Types and answered No when GSA asked me to verify them.

    When GSA goes to re-verify my proxies, it fails because my proxy server will deny any CONNECT request with IPv4 address. It needs a domain to resolve IPv6 target address. My idea is to scrap via huge and chap IPv6 subnets rather than buying IPv4 proxies.
  • SvenSven www.GSA-Online.de
    Thanks for the details. I was able to locate and fix the issue now. However, since this was "just" misbehaving on the proxy test, I will not update the product instantly. I hope you understand.
Sign In or Register to comment.