Skip to content

Multithread for Import URLs and identify platform

AlexAlex
edited April 2013 in Feature Requests
I think this is ideal, if we can make multithreaded for import URLs and identify platforms. Because currently, importing and identifying takes very long to process... Many guys have told a trick that just import URLs to project then let it run, then import verified/submitted URLs to global sites. But running through those ones has just 1 benefit which is it can be multithreaded because it's running via "submission function". So if you guys could make the "identifying platform" function multithreaded, it would make more sense! And also it's for the purpose , if the user doesn't want to submit those URLs, just want to identify them.
Just my 2 cents.
Tagged:

Comments

  • SvenSven www.GSA-Online.de
    if we can make... In the end I have to make it. And it is already using your set number of threads to do this.
  • AlexAlex
    Lol if I could help you make it, I'm willing to. 

    I've just changed my thread settings to 500 to see if it helps. By the way, if you set your thread to 500, are all the submissions, verifications etc... AND this identifications are 500, or just the identifications are 500 threads individually? @Sven
  • SvenSven www.GSA-Online.de
    500 threads for all together
  • avielmedinaavielmedina
    hi
    i'm on 150 threads, not doing any submission/verifcation, only import and still very slowly like 200 urls per minute.. 
    thanks Sven
  • AlexAlex
    @avielmedina You can set your threads to 1200, beware of the CPU usage. I'm running on an OVH with Xeon CPU but it's still consuming 99% CPU xD but it's incredibly fast.
Sign In or Register to comment.