with a 4 core, 4gb ram, with just PI running the cpu is maxed out.
Tried 100 threads, reduced to 50 threads. 5 txt files loaded as test (total approx 600k url's) to identify platforms. The cpu usage remains above 98%
Please advise. Have tried limiting bandwidth as well, no effect.