GSA PS + SB = Recommended

edited July 29 in GSA Proxy Scraper
So I'm finally getting the hang of using GSA PS and SB together. And I just wanted to say, the program is simply amazing for using Link Extractor, but you have to do it in the right way. Here's what I've learned:

1) GSA PS allows SB to use "connect" proxies, which results in dramatically higher success / completed URLs than just importing + checking proxies in SB.

2) Using 4x SB instances each at 100 threads only + GSA PS Internal server, results in a whopping 5x more completed URLs processed at 1/5th the threads, 100 threads vs 500 threads, in the same timeframe. For example, my 500 thread non-GSA PS SB's at 500 threads completed only 50-75k URLs before the public proxies died, whereas my SB's with GSA PS and only 100 threads completed 250k+ URLs in the same timeframe.

3) Using GSA PS with SB never "lags out" because it's constantly finding new proxies and importing them, opposed to just importing into SB and having proxies die mid-run.

- - - - - - -

There is unfortunately one issue. If you run GSA PS Internal server + SB for a while, GSA PS will be unable to close. Not sure what causes it, but it simply refuses to close the program, even when the SB's have been stopped for a long time. You're forced to ctrl+alt+delete, which removes all new proxies you've added.

The only way to combat this is to do auto-export, AND export all proxies before ctrl+alt+delete. I wish this issue could be fixed, I'm trying to send bug reports every time it happens.
Thanked by 1nycdude

Comments

  • DeeeeeeeeDeeeeeeee the Americas
    edited July 29
    hey seoaddict, thanks for the testing and data. :)  I do have a SB license, tho I don't use the software that often. So many other SEO ppl rely heavily on it, so I should probably look into using it more.

    I actually DLed and installed the demo of GSA PS just this morning!!!!!!!

    I have yet to read the GSA docs on this, but there are other tools built-in.

    It's a sick proxy manager. Can't say it isn't beyond cool.

    The program seems to take up little CPU usage, yet quickly collects a serious  number of (pre)sorted proxies and keeps them categorized and tested.

    Don't know why I never DLed this one to try before! Maybe b/c my old programs drive was always nearly full!? :p

    Is this a new interface? Asking b/c it looks very new.

    DEFINITELY on my to-buy list. ✔




  • edited July 29
    Yes it's great.

    @Sven

    On this note, I'm running 6x SB's with 100 threads each all going through Internal Server. Now I know the Internal Server seems to not work well with a lot of connections, but it's running fine... however, I just got a "Application Error" popup from GSA PS, and it said "continue running" or "close". I pressed Continue Running and my SB's still seem to be processing.

    I sent a bug report about the issue, please look! On that note, are the bug reports not saved anywhere that I can look at? I'd be interested to see what causes this issue, or rather, what the bug report shows.

    Is there any way you could tweak the Internal Server to work with more connections... even if it requires more CPU? Because as of now, it doesn't really use much CPU... but if it could use more when there's a lot of active threads, that would be great...

    Thanks!
  • SvenSven www.GSA-Online.de
    @seoaddictI got a bugreport from you this morning with the message of "not closing"....I didn'T find any suspicious things in it so I bag you to send them all again once you see this happening and I can debug maybe.
  • edited July 29
    Yes, check the one I sent about 30 minutes ago. It was during the application crash. It's separate from the one I sent this morning.

    Basically I was running 6x SB's at 100 threads each, and after a while GSA PS threw the "GSA Error" where the popup comes, and says Continue Running or Close Application. I pressed Continue Running, then submitting a bug report. It surprisingly kept working somehow... but, as with the memory leak/error I described when I first got the program...

    After this error. I decided to stop the SB's and restart GSA PS, and when I restarted (even though it closed properly) the "state" was reverted back to 10k proxies in the list, when there should've been only 7k.

    Now I'm trying 4x SB's at 100 threads, and 2x at 50 threads to see if it happens again. I also tried running the program as administrator to see if it makes a difference. However this is on Windows 7, which usually it doesn't matter either way.
  • SvenSven www.GSA-Online.de
    ok then it's logical...the crash happened without freeing the resources that it tried to wait for when closing now....next time please send the bugreport instead of letting it run.
  • Alright, I will. I just wanted to see if it would keep running if I pressed Continue Running, and indeed it seemed to still work. Do you think it would continue to work, or would've eventually just closed itself or had a crash where Windows had to force close it?
  • SvenSven www.GSA-Online.de
    I guess on one thread it got that exception and the amount of usable threads was one less...on a shutdown PS waited for that one thread to quit.
  • @Sven - New bug report sent. It was when I stopped the SB campaigns then tried to close GSA PS. I'm restarting + saving everything before I go to sleep. Hopefully you see something in the report. (I sent it from the actual error message box).
  • edited July 29
    Crashes are happening 100x more with the latest update... Every time I start 1 single SB campaign while also having search enabled on GSA PS (with only 5 threads), GSA PS instantly crashes. This was not happening before the update.
  • SvenSven www.GSA-Online.de
    but that report came up when you auto-exported something from main menu?
  • edited July 29
    No, the first one I simply tried to close GSA PS after running SB with internal proxy server for many hours no problems at all.

    Second one, I changed the search function from 60 -> 1 (just so it would start) -> then back to 60.

    Third one I didn't touch any settings at all. I just opened GSA PS, started 1 SB campaign, and it instantly crashed. I usually run my SB campaigns with GSA PS scraping proxies at only 5 threads, just to have a small drip of them coming in.

    Note that I do have auto-export set to every 30 minutes, mainly because I'm afraid it's going to crash and I'll lose all of the proxies that it's harvested while I'm asleep. But in either of these 2 cases, I did not try to export anything.

    Whatever the updated changed though (reading the changelog it seems insignificant) made the crashes happen instantly, within seconds of starting a SB campaign. I am back on the old version now, which at least doesn't crash instantly.
  • edited July 29
    Welp, on my 2nd install, after running GSA PS for hours no problems, I did indeed try to export proxies and it caused a crash. Another bug report coming, RIP.

    That does make me wonder if the auto-export is causing the crash now that you mentioned it above... but how else can I ensure the proxies are saved in case it crashes while I'm away?
  • SvenSven www.GSA-Online.de
    hmm hard to say where the bug is. However I fixed on in latest update...lets hope that was it.
  • @Sven Not sure what you changed but so far I haven't seen a crash, even after running SB for an hour, then closing GSA PS to save the current set of proxies, and re-opening. There was not any errors. So thanks for whatever you did! Will report back / submit bug reports for any future errors.
  • edited July 30
    Wait so @Sven, do you think this entire time this error was caused by me auto-exporting proxies every 30 minutes? I changed it to 720 minutes as a test and even on higher threads + Internal Server, I haven't seen a single crash. It could've very well been crashing after 30 minutes when the export tried to happen...

    I am testing 800 threads between 4x SB's now (200 each) and no crashes... It may have very well been the auto export causing the crashes the whole time.

    Update: I am fairly certain that it was the auto-export every 30 minutes causing the crash. Running at 800 threads on 2 different machines, haven't seen a single crash. We will see tomorrow when I wake up if it holds! But nice catch Sven and thanks a lot for the help!
  • SvenSven www.GSA-Online.de
    Accepted Answer
    maybe....if thats the case, we have something to work on :)
  • edited July 31
    @Sven Well I can confirm with 99.9% certainty that it was indeed the auto-export feature. I ran 800 threads on SB with GSA PS internal server on 2x installs for 12+ hours and zero crash. Closed all SB's today, zero problems closing GSA PS. Now we're talking... let's see how much more I can push it :).
Sign In or Register to comment.