Skip to content

When using a keyword list, the system seems to not know when to move on to the next keyword

andredoumadandredoumad
in GSA Email Spider
Hi, I've recently begun using your email search spider to gather emails for my marketing campaign. Today I made a list with around 180 keyword strings to search for. It seems that it does methodically search for each keyword but there is a time where clearly the queue has no new urls and it'll sit there empty for around an hour before switching to the next keyword. Is there something i can do to speed this up, other than manually searching for each keyword myself? I read the manual and I think the "stop searching after set time" might work but I figure I shouldn't need to force the parser to move to the next keyword. Maybe you guys can help.... Thanks in advance, and great job on your products you guys seem to be some awesome engineers!
Tagged:

Comments

  • SvenSven www.GSA-Online.de

    Don't use that mentioned stop option. It will stop the whole parsing, not just the current keyword.

    If you see it not moving forward, it can have been banned by search engines for being used too much. Usually this should not happen but if you use other tools on the PC that use google e.g. it might happen that google sees this as a bot activity and blocks you.

    To overcome this situation you can either use proxies or use different search engines like bing/yahoo or alike.

  • andredoumadandredoumad
    edited March 2014
    It isn't blocked because the "processing search-result" indicator reports that all of the results have been processed; It doesn't stop until it reaches the end of the results list. It'll finish with all of the search results but then perhaps 200 or so threads remain, the threads slowly countdown to 0 then it moves to the next keyword. I just wonder why it takes so long for the last 200 or so threads to finish up. Those last few hundred threads seem to take way longer to finish than the rest. Seems odd, but it does function. In other words, It takes roughly 30 minutes to go through thousands of urls/threads and finally reach the end of the search results. Then 60 minutes is dedicated to just processing the last few hundred threads. It seems disproportionate. So really it gathers all of the data from the urls inside of 30 minutes and then sits and waits a little over an hour before moving onto the next keyword. If I were to stop it, and enter the next keyword myself, the software would finish the job in less than 30 minutes, however, I have 180 keywords and I can't babysit the program all day so I'm forced to let the program run essentially at a third of the rate that I know it could be processing data at.
  • SvenSven www.GSA-Online.de
    the max time a thread would wait is 200seconds (Depending on your thread number). However if you say you use 1000 threads, it is way to much. Most of the time it will occure a timeout processing it.
  • andredoumadandredoumad
    I'll keep it at a max of 50 threads for awhile and see what happens
Sign In or Register to comment.