Skip to content

Importing a mixed list of 730k URLs

AndyNAndyN
in Need Help
I used GScraper and scraped 1.5 days straight and got a mix of 730k URLs, various platforms including blogs, forums, web2.0 or whatever. Its a pretty big file and I would like to import them into a fresh GSA campaign, and limit it to 10k VERIFIED links per day. How can I do that?

How long does importing and let GSA sorting out such big list take? 

Please provide some guidelines. Appreciated!

Comments

  • Tim89Tim89 www.expressindexer.solutions
    edited June 2013
    it takes me around a couple of hours to sort through 100k lists, I believe you can speed this up by stopping all of your current projects and increasing the thread count to as high as you can go, it seems to me that the sorting url/platform function uses your threads.

    it'll take you about a day I think, depends on your machine and what else your doing.

    about the verified question, you can simply set up the project and set the max verified count to 10k... per day or am I missing something?

    I don't think you'll get 10k verifieds per day with a untouched unfiltered fresh scraped sitelist though.
  • komakinokomakino
    @Sven, can you please verify that. Does the URL import tool use the # of threads defined in Options-->Submission?

  • Tim89Tim89 www.expressindexer.solutions
    Test it yourself, import your list into the platform separator and then deactivate all of your projects, your threads will continue to run the limit you set it at, I tested this earlier today.
  • AlexAlex
    Also you can split those links, import to your projects and let it directly post to it, so all will be saved under indentified/submitted/verified automatically if you check the boxes in the Advanced settings...
  • rodolrodol
    REad the Announcement   @sven is on holidays
Sign In or Register to comment.