I started a a new project and I wanted to use the verified list that I got from my other projects. A total of 30k links.
Just that 80% of those are no engine matches now. How is this possible?
I have 10 private proxy from buyproxies. 200 treads and 120 timeout.
Comments
When you do this kind of stuff, there are some things you should do first.
1) Understand that the majority (90% -95%) of all links in your verified folder are dups because of lower tiers
2) Actually remove duplicate domains and duplicate URLs (both)
3) Pay attention to your start number before the purge, and how much was deleted.
4) With that information, you now truly understand that most of your verified links are dups, whether Junk or even Contextuals
5) I haven't even gotten into using Sven's Clean-Up Tool, which does #2 above, plus it then verifies whether the remaining links are still there
So don't get disheartened or anything. This is normal.
Keep the 30,000 list in a safe place, and then clean a dup of it using the cleaning tool. See what happens.
What this is telling me is that (somehow) a bunch of links are gone. If they are mostly comments that run on to page 2, maybe I could understand. But something doesn't seem right.
Today on one of my servers I forgot to stick in the new proxies. So I ran with expired proxies and made almost no links using a list. So there are other factors to start diagnosing as well.