Comparing lists

busek

Not sure if there is a free app - or an app for gsa users - but I saw that gsa remove dupes is much faster than the scrapebox one. I have a trouble when I have to remove urls containing smth or compare it on domain leavel when the list is bigger than 1 mln records - it's the scrappebox limitation. It's also a trouble when the first list is smaller than 1 mln - but the other lists we try to compare are like few million records - scrapebox is hanging all the time in such a case. Can you guide me an ap for this - I guess gsa module of remove duplicates as some separete addon could be a help here.

busek

Any ideas on that?

gooner

Scrapebox DupRemove addon can handle files larger than 1 million lines.

busek

Yes - I know that - but only for the purpose of removing duplicates (btw the gsa remove dupes is much faster). It cannot remove urls on domain level or containing some lines.

lemonzer

i'm using gscraper for remove dups. 

gooner

@busek - I'm not sure what you mean exactly, SB and GS do it and also SER dup remove i have used with sietlists of up to 7.5 million... It takes a long time but it works.

busek

Thank guys for answers, but as I wrote in previous 2 posts - I am not looking to remove dupes - but to compare two lists and remove urls from one of them.

gooner

@busek - Ohhhhh then you face a whole other challenge. As far as i am aware there is no commercially available software that can do that. I was searching for it as well.

idreesfarooq

Use compare plugin in notepad plus plus.

JudderMan

I think Text Wedge does this, but as ^^ says notepad++ is probably your best bet.

busek

But have you tried doing it in notepad++? I can barely compare two lsits containing of few thousand results. I also know that even opening few million results file is hard. But I run it with a list of under one million and on the second I have only one keyword to compare - when I tried to compare them I got: