Skip to content

Check files against each other when removing duplicates

VMartVMart Natural SEO
edited July 2017 in GSA Platform Identifier
Check files against each other when removing duplicates

What is the use of this option, for what are your recommend this, can I tick this?

Comments

  • s4nt0ss4nt0s Houston, Texas
    Normally, Pi will only remove duplicates from a single file. It reads the file, see's if there are any duplicates, then removes them. If you enable this option and you have a folder with multiple files in it, Pi will look into all the files when seeing if there are duplicates.

    If you're using SER, you don't want to enable this option since some URL's can be identified by more than 1 engine. If the same URL is found in 2 files, it will remove one of them. 

    For SER use, its best to leave this unchecked.
Sign In or Register to comment.