Removing duplicates
Hi!
Sorry for spamming now with feature requests, but I just genuinely think these would make all our lives better.
Instead of manually having to remove duplicate urls and domains, why not at least make it some kind of global option? Like "remove duplicate urls and domains at ___ interval" or something.
I think it would be pretty neat, since it would make every guys and gals GSA run smoother, and again it would collectively save like 1000 000 00000 hours of human work in the long run
Sorry for spamming now with feature requests, but I just genuinely think these would make all our lives better.
Instead of manually having to remove duplicate urls and domains, why not at least make it some kind of global option? Like "remove duplicate urls and domains at ___ interval" or something.
I think it would be pretty neat, since it would make every guys and gals GSA run smoother, and again it would collectively save like 1000 000 00000 hours of human work in the long run
Comments
For example, I have 3 campaigns which are just finding target sites all day and after around 20h of not deleting duplicate domains there was over 73mil target URLs (300k after deleting duplicates)
Don't even wanna think about what would happen if I waited a few days...
Doing this automatically would be a long way forward in the "no babysitting needed" direction.