Skip to content

Import target URls limited to 18k?

edited March 2013 in Need Help
I scraped some 80k urls and wanted to import in GSA. But it says total links as 18k..Is GSA limited to import only that many URL/s? I dont want to identify the URLs as that takes hell lot of a time
Tagged:

Comments

  • BrandonBrandon Reputation Management Pro
    How are you importing? I import 100k+ a couple of times per day.
  • Right click on project and click on "import target URL"s...It says 80k links imported but at the bottom in the console here it says like 1/18000. etc etc.
  • Yes I think I have seen this too. Not sure what the issue is maybe be list is not formatted the correct way? Do they need to be trimmed to root?

    Also my lists are already filtered for dups.
  • it reads partially that is why you see
    i am importing over 7m

    but problem is when you try to see how many left, it loads all which makes huge ram usage and slowness
  • It's normal, SER split your big list in small lists and store them in cache.
    When your reach 18000/18000 then after you should see 1/18000, and loop again until SER process all your urls.
  • SvenSven www.GSA-Online.de
    Correct. The program loads not all URLs from the file, just in batches to not waste memory.
  • What software You use for scraping target urls ?
  • BrandonBrandon Reputation Management Pro
    @hoolak to verify you can right click Target URLs > Show Left URLs.  That will show you how many are remaining.  It's not always exact unless you click Stop first.

    @Danhoff I recently bought gscraper and it's so much faster that SB.
  • yea right got it..it imports in batches and I can see in Show left URLs..

    thanks
  • i too like gscraper.
Sign In or Register to comment.