Skip to content

Import target URls limited to 18k?

hoolakhoolak
edited March 2013 in Need Help
I scraped some 80k urls and wanted to import in GSA. But it says total links as 18k..Is GSA limited to import only that many URL/s? I dont want to identify the URLs as that takes hell lot of a time
Tagged:

Comments

  • BrandonBrandon Reputation Management Pro
    How are you importing? I import 100k+ a couple of times per day.
  • hoolakhoolak
    Right click on project and click on "import target URL"s...It says 80k links imported but at the bottom in the console here it says like 1/18000. etc etc.
  • gamemerlingamemerlin
    Yes I think I have seen this too. Not sure what the issue is maybe be list is not formatted the correct way? Do they need to be trimmed to root?

    Also my lists are already filtered for dups.
  • ComputerEngineerComputerEngineer
    it reads partially that is why you see
    i am importing over 7m

    but problem is when you try to see how many left, it loads all which makes huge ram usage and slowness
  • sawa73sawa73
    It's normal, SER split your big list in small lists and store them in cache.
    When your reach 18000/18000 then after you should see 1/18000, and loop again until SER process all your urls.
  • SvenSven www.GSA-Online.de
    Correct. The program loads not all URLs from the file, just in batches to not waste memory.
  • danhoffdanhoff
    What software You use for scraping target urls ?
  • BrandonBrandon Reputation Management Pro
    @hoolak to verify you can right click Target URLs > Show Left URLs.  That will show you how many are remaining.  It's not always exact unless you click Stop first.

    @Danhoff I recently bought gscraper and it's so much faster that SB.
  • hoolakhoolak
    yea right got it..it imports in batches and I can see in Show left URLs..

    thanks
  • thisisalexthisisalex
    i too like gscraper.
Sign In or Register to comment.