Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Removing duplicates


Sorry for spamming now with feature requests, but I just genuinely think these would make all our lives better.

Instead of manually having to remove duplicate urls and domains, why not at least make it some kind of global option? Like "remove duplicate urls and domains at ___ interval" or something.

I think it would be pretty neat, since it would make every guys and gals GSA run smoother, and again it would collectively save like 1000 000 00000 hours of human work in the long run :)

Best Answers

  • SvenSven
    Accepted Answer
    Sorry I don't undertstand that request as you have a "Remove Duplicates" already. And I don't see a reason for doing it automatically. It's not a problem to have duplicates in global site lists.
  • HinkysHinkys - Catchalls for SER - 30 Day Free Trial
    Accepted Answer
    +1 for automatic duplicate domain removal, especially on target URL lists for specific projects.

    For example, I have 3 campaigns which are just finding target sites all day and after around 20h of not deleting duplicate domains there was over 73mil target URLs (300k after deleting duplicates)

    Don't even wanna think about what would happen if I waited a few days...
    Doing this automatically would be a long way forward in the "no babysitting needed" direction. :)
  • Accepted Answer
    I don't see a purpose for this. Removing dupes takes like 5 minutes at most.


  • Justin, that's 30 hours per year of manually clicking on a button that could take care of itself.
  • Back in the old days, we had to remove duplicates by hand and walk uphill both ways to school.
Sign In or Register to comment.