Not using the full amount of threads
Hey,
I have a dedicated server where is SER so the CPU/RAM isn't a problem. I don't know from where this problem come, it's very annoying. I'm using 300 threads, only one project is running and I'm not scraping, only imported URL list.
When I start SER, it's running nice at 300 threads then when come the email checking state it's VERY VERY slow, the number of threads drop to between 5-40 and will never go up again unless I stop and restart SER. And when I have a list of 13,000,000 to parse... For example this night I parsed only 200k URL from 13,200,000 that is too slow.
I'm using 40 dedicated proxies, 1k yahoo email, Captcha Breaker with Recaptcha OCR, InstantLinkIndexer...
If someone can tell me anything that could help me to solve this matter...
Thanks.
I have a dedicated server where is SER so the CPU/RAM isn't a problem. I don't know from where this problem come, it's very annoying. I'm using 300 threads, only one project is running and I'm not scraping, only imported URL list.
When I start SER, it's running nice at 300 threads then when come the email checking state it's VERY VERY slow, the number of threads drop to between 5-40 and will never go up again unless I stop and restart SER. And when I have a list of 13,000,000 to parse... For example this night I parsed only 200k URL from 13,200,000 that is too slow.
I'm using 40 dedicated proxies, 1k yahoo email, Captcha Breaker with Recaptcha OCR, InstantLinkIndexer...
If someone can tell me anything that could help me to solve this matter...
Thanks.
Tagged:
Comments