Skip to content

Limitation on tool "identify platform and sort in" ?


Is there a limitation on maximum URLs in the advanced tool "identify platform and sort in" (importing per file)? 
I got a file with about 1.8 million lines/URLs, and "identify platform and sort in" finished the job with only about 77.000 urls checked.

Thank you


  • SvenSven
    no limitations, but I guess there are a lot duplicates in it?
  • @Sven btw i have some question want to ask you.

    i follow this steps to import the new list feed SER.

    1: scrape the list by sb/gs and remove the duplicates by links(not by domains)
    2: identify platform and sort in the list by SER, i think them will come into identified folder right?
    3: make a test project only use the identified folder list.
    4: run the test project and choose to make the verified site list with options.

    is there any signals or other things can let me know the identified folder lists is runs out by test project?
  • SvenSven
    All is correct, except that you might want to import all the URLs from identified to the prohject directly. That way you know when the project runs out of targets.

  • razzaguhlrazzaguhl Deutschland
    @Sven No duplicates. All unique URLs (not domains) in the file.

    For testing purpose I splited the file in 50.000 line chunks and added three of them in the tool. So far, the progressbar "looks better" - as is not running that fast.

    Give you feedback as soon as the tool finished the job.

  • razzaguhlrazzaguhl Deutschland
    With splitted files the tool works. Three files with each 50.000 lines where processed:

    "Added 28930/126285 URLs to site list." Still wondering why its a total of 155.215 urls, thats 5.215 more than presents URLs from the files.

    But single file finished to soon (checked it twice).

  • I am having the same problem. ALthough I have removed the duplicate urls but still 300k links are only getting processed from a 3 million links file
  • As far as I know, there are no real limitations.
  • razzaguhlrazzaguhl Deutschland
    Thanks for your confirmation @AsimNawaz

    @ericspurs Sven also mentioned that there is no limitation. But it happen not only to me. So there might be a bug (or wrong usage)
Sign In or Register to comment.