I have 100 private proxies and set the Scraping/Searching option to 100 but it doesn't go higher than 10-20 when I run it. the CPU works around 5% and the ram is around 250MB max. what is wrong?
I notice the same problem, GSA just run full thread for a few minutes and then drop thread to very low and even stuck at a couple of threads, this behavior just happens in the latest patch 16.75. Can anyone help us please T.T @Sven please give us some idea on this problem thank you very much!
Depends on how many search engines and keywords you have. It is using one search engine + keyword combination only. If you use e.g. google and bing only and have 10 keywords added, it will open a max of 20 threads.
I have a lot of target sites and a dot use search engine, after the update it run for a few minutes full thread and then stuck at a very low thread, i have to shut down GSA-SER and turn it back on again to solve the problem
Depends on how many search engines and keywords you have. It is using one search engine + keyword combination only. If you use e.g. google and bing only and have 10 keywords added, it will open a max of 20 threads.
I didn't change the number of search engines so there are more than two. and the keywords I don't use more than two or three. so there should be something else. I want the software to use more CPU and RAM so to search faster. another issue is sometimes it starts so fast and gradually goes down to almost zero. i dunno why.
ok. half of the problem is resolved. here is the deal. the more keyword you use the faster the Scraping will be. BUT the problem is that scraping speed goes down to almost zero in a few minutes. I set 500 threads in the option it starts around 400 and goes below 10 in a few minutes. that is not the case with Sending. whatever the number i set in the option it sticks with it. I set it to 500 it stays there but I dunno if the higher the number does it affect the quality of Submitting or not. I mean do I get less green if I increase the number?
Scraping: It's normal that you see a decrease in activity if search was finished by a search engine/keyword combination. Also note that between each search, it waits a couple of minutes to not ban your proxy.
---
Checking/Sending: You can increase threads as much as you want but you need to make sure that your network can handle the load and that you stay below 3.8GB or RAM usage (32bit app).
Scraping: It's normal that you see a decrease in activity if search was finished by a search engine/keyword combination. Also note that between each search, it waits a couple of minutes to not ban your proxy.
---
Checking/Sending: You can increase threads as much as you want but you need to make sure that your network can handle the load and that you stay below 3.8GB or RAM usage (32bit app).
Scraping: Not necessarily because when I stop and start the operation the same thing happens. it goes up and then down. so basically if it was the case it should not go up again. it had to stay low.
Program not respond when writing temp file here - C:\Users\User\AppData\Roaming\GSA Website Contact\projects\file-name.datA736C7.tmp If number of lines > 1 million, this cause unstable connect
Hi Sven, I have a similar query. I have 25 private proxies which are working fine. I have around 8k keywords. In both cases below, CPU uses is around 2%, ram is below 1GB and I have a fast broadband i.e. around 50 mbps.
1- When I choose all international search engines + Google (US) and Bing (US) and select 200 threads, it works fine i.e. I can see 200 threads working and speed is good.
2. But when I choose only Google (US) and Bing (US) and choose 200 threads, its hardly 20-30 threads or even 10 and sometimes 7 or 6 threads. Note: I have 8k keywords so there are plenty of keywords to work on.
Question- When I have 8k key words and chose 200 threads in two search engines i.e. Goggle (US) and Bing (US), there should be 200 threads working??? Whereas, I see threads drops to 6-7 and per minute message sent are 5-10. Note: Changing threads doesn't have any effect in this scenario means if I chose 400 threads, its doesn't matter.
My opinion: Ideally when I have 8k key words and two most popular search engines like google (US) and Bing (US), enough private proxies, cpu is good (2%) memory is good (less than 1gb) with good internet speed, working threads should be what I choose and not otherwise.
it will at max use the amount of proxies * unique search engines ... as you have 25 proxies, the max amount of threads is just 25 for google only and 50 for google and bing.
There should be a dedicated videos explaining number of threads depends on what all factors and how to use it optimally. Information comes in bits and pieces and that too is not complete for many technically not very understating.
it will at max use the amount of proxies * unique search engines ... as you have 25 proxies, the max amount of threads is just 25 for google only and 50 for google and bing.
But as I told, I have 8k keywords and number of threads drop to 5-10 doesn't make any sense!! Something is not right. I observed number of threads increasing to like 175 and then dropping gradually to 5-10. Confusing.
It makes sense if you keep in mind that a search engine can not be queried by the same ip or proxy more than ones at a time and that you need to wait a certain time before querying the same search engine again with that same ip/proxy. If you do not take care of a certain "cool-down timing", you get all your proxies banned for excessive usage.
Comments
GSA web contact
I didn't change the number of search engines so there are more than two. and the keywords I don't use more than two or three. so there should be something else. I want the software to use more CPU and RAM so to search faster. another issue is sometimes it starts so fast and gradually goes down to almost zero. i dunno why.
the default. around 20 search engines. i havent touched that setting yet
nothing
this is it
If number of lines > 1 million, this cause unstable connect
1- When I choose all international search engines + Google (US) and Bing (US) and select 200 threads, it works fine i.e. I can see 200 threads working and speed is good.
2. But when I choose only Google (US) and Bing (US) and choose 200 threads, its hardly 20-30 threads or even 10 and sometimes 7 or 6 threads. Note: I have 8k keywords so there are plenty of keywords to work on.
Question- When I have 8k key words and chose 200 threads in two search engines i.e. Goggle (US) and Bing (US), there should be 200 threads working??? Whereas, I see threads drops to 6-7 and per minute message sent are 5-10. Note: Changing threads doesn't have any effect in this scenario means if I chose 400 threads, its doesn't matter.
My opinion: Ideally when I have 8k key words and two most popular search engines like google (US) and Bing (US), enough private proxies, cpu is good (2%) memory is good (less than 1gb) with good internet speed, working threads should be what I choose and not otherwise.