Skip to content

what is thread in scraping?

Attock98Attock98 USA
in Need Help
threads you would like to use for scraping "Scraping/Search" and submitting contact forms "Checking/Submission". I normally use anywhere between 100 to 200 threads on my 20 core/256GD dedicated server with the most powerful video graphics card. I have tried using 500 and 1000 threads but even the most powerful dedicated servers do crash or become sluggish so I took the thread count down to 100-200.

what does thread mean? I am new to it and wanted to lean it?

thanks

Comments

  • SvenSven www.GSA-Online.de
    Think of one thread as you doing one thing (e.g. reading) and another thread doing other things like listening to music.
    Just that with our product it's 100-200 threads doing the same at the same time but for different websites. Meaning instead of submitting to one site after another, it is submitting to 100 sites at the same time.
    Using too many threads would mean you would also need a lot more memory and of course in our case much more network bandwidth. So keeping this to a reasonable number is important to not have the system crash.
  • Attock98Attock98 USA
    nice. thank U.
Sign In or Register to comment.