Skip to content

GSA Keeps stopping? Need to manually stop/start project to get it to post again

This has been happening now for weeks...

Before I go to bed at night I duplicate some projects and import huge lists to them (either from sitelist or from textfile). At first, GSA blasts through them like a hot knife through butter.

But EVERY few hours, for some reason the threads go waaay down, cpu usage is only 2% and it's like GSA is doing nothing. When I check show urls-> remaining often there are still 100k urls left. All I need to do is stop/start project and it starts blasting fast @ max threads and 90%+ cpu....

Does anyone else have this? Am I missing something?

GK

Comments

  • same here....seems like gsa is doing it in chunks or something similar so its like when first one finished completely gsa starts new ...maybe
  • SvenSven www.GSA-Online.de
    Check logs please. Im sure it's doing email verification or something like that.
  • sure it isnt proxies and auto check? had that issue till i disabled the option of disable proxies when not working in proxy options.
  • edited May 2014
    I have this too.

    300 threads 100 proxies.
    Threads start at 200lpm and after an hour it's downl to 38 :/

    Autoatically disable proxies is unchecked

    Checking my logs i see alot of this

    20:31:25: [ ] Loaded 1/1 URLs from site lists
    20:31:26: [ ] Loaded 1/1 URLs from site lists
    20:31:26: [+] 1/1 matches engine MediaWiki - http://www.idiog.............
    20:31:26: [+] 1/1 new URL - http://www.idioga...................
    20:31:26: [+] 1/1 new URL - http://www.idio.........
    20:31:26: [ ] Loaded 1/1 URLs from site lists
    20:31:26: [-] 1/1 wrong engine detected - http://www.idi......
    20:31:27: [ ] Loaded 1/1 URLs from site lists
    20:31:27: [+] 1/0 matches engine MediaWiki - http://www.vue.....
    20:31:27: [+] 1/0 new URL - http://www.vue......
    20:33:20: [ ] Loaded 1/1 URLs from site lists
    20:33:20: [ ] Loaded 1/1 URLs from site lists
    20:33:22: [ ] Loaded 1/1 URLs from site lists
    20:33:22: [+] 1/1 matches engine .........
    20:33:22: [ ] Loaded 1/1 URLs from site lists

    when i start gsa it starts loading all the urls in sitelist, but after a while (30mins to 60mins) it looks like it only loads 1 url.


  • I think i know what it is but it's hard to explain

    I think it's related to a small list size. maybe sven can add his thoughts. i think it's related to this thread

    so i'm guessing it's a combination of too many threads combined with not enough links...
  • I import from sitelist only, always more than 100k urls per project...

    @ SVen, i check the logs and its not parsing emails. It just slows down after a while and maybe there is a new log entry every 30 sec or so. Threads and cpu go down to 1-2.

    Only way to get it to go fast is restart project
  • I'm having the same problem and have had it for weeks. I have to babysit GSA all the time ON MY VPS. It's really sad. Hope this gets fixed soon because there's really no reason for it to slow down and do nothing for hours.
  • @Sven ? It would be nice if this could be fixed. GSA keeps lowering the threads to about 30-130 and stays there. In the log it says the usual, posting etc. Nothing unusual and it's not doing any other tasks. It's just slow. Then if you stop it and start again it's acting normal for a while and falls down again. Hope this gets fixed soon because it's sad to babysit GSA on your VPS. 

    Regards,
  • ronron SERLists.com
    edited May 2014

    Guys, just a heads up, but SER is cranking faster than I have ever seen it. I wish I took some screenshots yesterday (I built over 400,000 links), but here's a quick screen shot from running at the 14 hour mark:

    image

    I have verification disabled in a lot (majority) of projects so that number is artificially low. Running v8.38. SER is running faster than I have ever seen it in 2 years. Seriously, it has to be the VPS or settings. Probably the VPS.

  • I'm running a VPS with very good specs. Very good processor, 32GB ram, unmetered bandwidth etc. What should I look for?
  • ronron SERLists.com

    I have been duped by the specs on VPS's. You know, 4 cores, 3.6Ghz, blah blah blah. But then it doesn't perform. And then you keep thinking it is SER or whatever. I would say the first rule is to always get a Xeon processor. But after that, I would treat yourself to a month run on a dedi. And just compare notes. Personally, I am getting tired of VPS's because of the poor performance.

  • I see. Well got the processor you are talking about. I'm using a dedicated server because I can't have my PC turned on all the time. Well, I could, but I could also risk the internet connection to fail sometimes which could lead to GSA not posting for hours etc. 
  • ronron SERLists.com
    Then it has to be settings. You should post some screenshots. Plenty of people here can help you out. Post General Main Option Settings and then project settings.
  • I just noticed this thread, and @ron got me thinking if it's indeed my VPS.

    I will say, however, that I have the EXACT same thing happening to me: I start my projects, and boom, everything's fine. Then it just dies in a couple of hours. When I stop it and start it again, boom, everything's working like magic again.
  • ronron SERLists.com
    edited May 2014
    @Artsi - There have been people that have had this complaint over the last 2 years (very few, but enough for me to notice the threads). My gut is that it is the VPS. I don't know why nor can I explain it. I just know that I have had VPS's and Dedi's, and I have never had this issue. And 99% of all users I know have not had this problem.

    So speaking as a SER user, I honestly do not believe at all that this is a SER issue. It is probably VPS related, but it may also be some weird setting problem. 
  • edited May 2014
    GSA SER has also been behaving the exact same way for me for months. I've switched provider 3 times already, and now I've got 3 dedis that are all experiencing this. It seems to mostly happen when I'm running a low amount of projects (1-5). When I'm just running one project, the thread usage will after about 20-30 minutes start to slowly decrease at 1-3 threads a time until it reaches 1 when it then increases very quickly again. Now I'm just using the scheduler to run all projects and restart after 30 minutes which seems to have helped on my speed. When I didn't use the scheduler, it was absolutely murdering the amount of links I could build in a day.

    Edit: oh, and of course it's not doing verification.
  • Fakenickahl, interesting. Maybe I should try using the scheduler to restart every 30 minutes like you. It is also happening to me running low amount of projects. I tried running a lot of projects then it worked. 
    But still annoying, this can't be the only alternative.

    Pictures of settings:


    Running 2k proxies, public, but google passed.
  • ronron SERLists.com
    It might be the low number of projects. I haven't had anything less than 100 projects per VPS since 18 months ago, so you guys may be on to something here.
  • So GSA is not able to run with low projects anymore maybe. Anyway, I'll have to set up more projects in order to make sure my threads stay up.......
  • KaineKaine thebestindexer.com
    edited May 2014
    Same problem with dialog box. Can't click for confirm, must use task manager for kill process.

    Problem start on same version (.41) and start after reboot.
  • this is what might be a reason ?
    lets say we have dedi and trying to run 400 threads
    if its 1 project all threads goes to 1 project it start with 0 and goes to 400..than gsa use another chunk of linklist and do from 0 to 400.
    now if there are 10 projects with same priority gsa divide threads by ten and do same as above but LPM sticks to same in second issue is because more porjects more WORKING threads
Sign In or Register to comment.