Weird side-effect from "down detection" update?
The down detection is working well, but it seems to have had some sort of unintended side-effect of making the internal server "lag out" after hours of running. The program seems to quit feeding SB proxies, and the Link Extractor instances just come to a halt. This is to be expected on public proxies (without GSA PS), as they go down and SB can't pull any working ones. However, one of the major benefits of using GSA PS + SB is to avoid this issue.
While investigating, I found a weird thing in the TCPView. Usually "localhost" is never shown in the connection manager from what I've seen (it's not a "remote address"). I think whatever happened here, is that GSA PS is "lagging out" somehow. It could also possibly be SB lagging out, but I ran this exact same setup weeks ago and never saw this issue with GSA PS. It only happened with public proxies without GSA PS.
When I logged into the server today, all instances of SB Link Extractor had stopped moving, as if there were no working proxies.. but there were 50k working in GSA PS. I often check the connections to clear them out using TCPView, but I've never seen "localhost" actually be listed in the connections.
While investigating, I found a weird thing in the TCPView. Usually "localhost" is never shown in the connection manager from what I've seen (it's not a "remote address"). I think whatever happened here, is that GSA PS is "lagging out" somehow. It could also possibly be SB lagging out, but I ran this exact same setup weeks ago and never saw this issue with GSA PS. It only happened with public proxies without GSA PS.
When I logged into the server today, all instances of SB Link Extractor had stopped moving, as if there were no working proxies.. but there were 50k working in GSA PS. I often check the connections to clear them out using TCPView, but I've never seen "localhost" actually be listed in the connections.
Comments
And actually EDIT: I just checked TCPView while everything is still moving, and those "localhost" entries are indeed there, I just never saw them before because too many ports were open and hard to see. But I sorted and they are there. (Makes sense because of the Server, but I just never saw them).
Hmm... I wonder what's making everything stop after 80,000 URLs scanned. Possibly something to do with the time-running instead of the threads etc. I tested 2000 threads tonight and although GSA PS proxies flew down quickly, I got 200k URLs scanned without any slowdowns at all.
One thing though, no matter what, this one server of mine will never get the update. It just never sees there’s an update available, even if I click check for update, it says I have the latest one but it’s not. No firewall enabled or anything. Both others receive all updates fine... same Windows, same updates, same TCP settings, etc. What could cause this? (Note that GSA server would be seeing these 3 installs on the same IP).
Far as the “stall” thing I’ll have to see in a bit (testing a large list now), I think maybe it’s an issue with Scrapebox, or possibly the TCP Windows time-outs I’m using. But the down detection is definitely way better now, I can leave everything on overnight and not worry about.
Thanks for the updates!!
Also it seems that 2.64 and 2.65 go down at about the same speed... was 2.64 a bit slower to go down (more retries) or should 2.65 be a bit slower?
Re: not getting update — Nope even if I turn everything else off... stop all programs, shut down all PCs, restart router and turn only the 1 on... I won’t receive the update. But all others will even if running at 100mbps there’s never a problem receiving updates.
Tonight I'm testing removing all TCPIP registry settings I added from a server to see if it does anything with the stall issue. I really don't remember it doing this before I added these settings... the settings actually helped a lot with errors and stuff though. It used to do this with public proxies in SB, but never when using GSA PS.