Skip to content

"Out of memory" error... but nowhere close to out of memory

seoaddictseoaddict
edited April 2019 in Bugs
Hi,

Since the most recent version of SER, I'm getting this message every day on a server I leave running 24/7.



This server has 16gb memory. SER is currently running at only 3gb. Total memory used by SER, CB, and all OS functions is only 5gb. The memory isn't even 50% used, yet I'm getting this error every day.

Any idea what could be causing it?

SER is only running at 75 threads. I've recently run this same server on 150+ threads without any problems.

If I press "no" I get the SER crash dialog.

It seems one of my recent updates resulted in a memory leak, that's the only thing that could explain this happening... I'm watching the memory now, after 5 minutes of running, SER is fluctuating between 200-300mb. How it goes from 300mb, to 3gb with the same projects running is the problem.

Comments

  • SvenSven www.GSA-Online.de
    Keep in mind that SER runs in 32bit mode. Meaning you reached the 3GB limit already and so it comes to this exception.
    I don't know what is causing it to use so much memory but I doubt it's about the new updates. Try the following:

    1. select all project (inactive)->right click->modify->delete unused accounts + delete target url history

    Also check the proxy setup. Maybe the public proxy scanner is finding too many data that wastes memory.
  • seoaddictseoaddict
    edited April 2019
    Using 105 private, dedicated proxies. No public proxies.

    I will try the "Delete unused accounts" because it could be related, as these are "scraping template" projects that have been re-used (cloned + re-used) many times.

    Before starting a new one (after cloning), I always do "Delete Target URL History," but I've never done "Delete unused accounts".

    Will let you know.
  • seoaddictseoaddict
    edited April 2019
    Well, there was ~70,000 unused accounts that it removed... so let's see if that fixes the issue.

    Edit:
    After ~15 minutes of running, the memory usage has gone from 250mb -> 800mb. Will see if it crashes later on.

    Barley any links have been submitted or accounts created. I leave these projects running with the SER search engine scraping running with all search engines (besides Google) enabled.

    Could it be related to the search engines / scraping? I've been using this exact method for 3+ years and never had a problem, never seen anything like this error... so it would've had to be something recent that changed to cause this.
  • seoaddictseoaddict
    edited April 2019
    SER is now at 3.14gb memory usage after ~12 hours of running but it didn't crash / have the error. This has to be something that changed in a recent update... because again, I have been running these projects in the exact same way for many many years and never seen this happen.

    I do have a large list of "don't post to domains containing" on some of the projects, but I don't see how that would cause such large memory usage increase... is there any way to see which projects are causing the issue?
  • SvenSven www.GSA-Online.de
    If you have a look at the changelog (which is really that and no other changes) it was not changed much lately what could have caused this.

    Maybe it would be possible to send me your project backup. If it grows on memory usage the same way here, I can probably spot the problem and fix it.
  • seoaddictseoaddict
    I sent a bug report... it's happening every single day now... never seen this in 6 years of running SER in this same way.
  • SvenSven www.GSA-Online.de
    again, send the project backup please. I really can't do anything as you seem to be the only person with the issue and I also was not able to locate any memory leak or whatsoever.
  • seoaddictseoaddict
    @Sven I've come to the conclusion that it must be my large "negative domain" lists that I use, because I don't want SER to find existing URLs. However, I just checked this list... It's of about 30k (root) domains. The list itself is only ~1mb... so I don't see how 1mb worth of root domains can translate into 500mb+ in SER. But otherwise, there is simply nothing that could be causing this issue.

    Could you try adding ~30k domains to your negative list and see if it causes the same issue for you?
  • SvenSven www.GSA-Online.de
    Maybe we can do this instead: Make a backup of one project + settings (global options->click backup on lower left) and send me that. I will check what might cause the memory lose here.
Sign In or Register to comment.