Max CPU at low threads

kijix84

I don't know why this started happening but;

1. doesn't matter which projects i'm running

2. i used to be able to run up to 1000 threads on this 4 core VPS without issue, now i can't even run 250 threads

i don't know what the issue is. i've restarted SER and cleared target url history for all projects but nothing has made a difference

Sven

it's due to a new thread managemant. All threads are proceed faster now and resulting in way better performance, but also more cpu usage.

kijix84

is there somewhere i can download older versions of SER? a database of sorts?

Sven

sorry no. Thats not possible. MAybe someone here can give you an older version. However the new threadding solution is using more cpu, ok...but thats also because your cpu is used better than before.

devmd

Same problem here. I know you say the CPU is used better now, but my performance has dropped significantly since the new update. I can not get pass 200 threads without maxing out the CPU. With the old threading I was using 500-600 threads without any problem. This is also affecting the number of links submitted, which has dropped to half of what it used to be. 

Sven

What CPU is that?

devmd

The problem is observed on Intel i7-2600 quad-core 3.4 GHz, Intel i7-3520M 2.9 GHz both of these machines have 8GB ram, also on  2 x Xeon 5570 with 48GB, when the machines set to submit only to verified site lists without scraping any URLs from search engines. I am using the same lists as before the issue started. Did a test with version 10.68 same site lists, same settings, on the i7 machine, with 550 threads and it did not have any serious spikes in load which dropped the threads (I have the setting at 80%). With version 10.86 I can go to max 250 threads before CPU spikes become problematic.

There is another machine with 12GB ram and single Intel Xeon X5570 running GSA without any CPU load issues when it scrapes the URLs from search engines without any problem.

BlazingSEO

In Sven's defense, CPU load is abstract in this case.... he said that threading was improved, so running 100 threads now is the same as running 500 threads before this version (as an example).

The only true argument that you could make to Sven that there is a problem is if your LPM is lower when comparing the SAME lists to each other.

devmd

I do not think that LPM is the realistic metric since it relies on many factors like captcha breaking which sites it would get from the list first etc... But just took a look at the results from both installs, with v10.86: 4.59 with v10.68: 11.73. I think that the LPM in the end is a results of the more threads. I did the test on the same verified list using only captcha breaker v3.28 and no proxies. Both GSA installs were left to the point of no more targets just to be sure that they have submitted to the whole list. Upon comparison of the verified links form both projects there was a 95% match so basically they both submitted to the same URLs successfully just one of them did it slower. 

kijix84

Well it's true since having this issue I have tried multiple ser versions on multiple vps and had the same issue. I then tried different lists and had better results. So my problem is not related to ser version as I loaded my settings on a version prior to the thread change and had the same issue

Problem is I don't know why ser would choke with this list. it's just a contextual list that I scraped and sorted with gsa pi.

kijix84

i want to update this thread with new info. i ended up getting a dedicated server, a xeon e3-1270 with 32gb ram and 120gb ssd. with this dedi, gsa no longer chokes and flatlines CPU. so i think gsa has just become very CPU intensive, for whatever reason moreso when it reads from the project settings site list files as opposed to directly importing the site list. maybe its because it has to constantly read from the files where manual import you are feeding it a list and then it just goes sequentially off of that, and sequential reading is always less CPU intensive. either way, if you are having this issue, i would recommend trying manual import of your site list or simply upgrading to a dedicated server.

PaulieP

Same problem here, i can't even run a 100 threads anymore without freezing up my server... and i don't see any benefits from this, because with low thread count comes low results... i am very sorry but this update really sucks imho, does anyone still have the older versions as this really doesn't work for me... 

GreenCloudServer

4 cores @ 3.4GHz / 3.8GHz Turbo

16GB RAM (DDR3)

2x1TB SATA3 Hardware RAID