Captcha Breaker VS Captcha Sniper - A Definitive Case Study
I've been wanting to do this for quite some quite but never got around to it...until now!
After a while of running CS, I was sure that CB was better. The constant freezes and whatnot are annoying as hell as well as the fact that it seems like it gets every captcha wrong when I'm looking.
So I just downloaded the demo version of CB and started to figure out the details of the test. Also, I wanted to test the Spamvilla + CB combo and compare that to pay-per-captcha services. I've contacted spamvilla asking if I could borrow their API for a few days to include them in the test and got a positive response but never heard back from them after that. Oh well, if they want to pass on free promotion it's their loss.
But let's forget all that and dive right in to the test! I've tried to be as transparent as possible so anyone can replicate this test.
What exactly have I done here?
1. Compiled 5 lists using the "Identified" folder from my SER and loads of randomization:
- Random mix #1 (10k sites - all platforms)
- Random mix #2 (10k sites - all platforms)
- Social Bookmarks (2,5k sites - Pligg, Phpdug, Scuttle)
- Web 2.0s (2,5k sites - Dolphin, Elgg, Jcow, Oxwall, PHPFox, PHPizabi)
- Wikis (2,5k sites - DokuWiki, MediaWiki, TikiWiki, MoinMoin, WikkaWiki)
2. Created 4 projects in SER for each of these lists, the names are self-explanatory:
- "DeathByCaptcha" - Retry captcha 1 time
- Serves as a control project to see what is the max # of sites we can post to
- "Captcha Breaker" - Retry captcha 4 times
- "Captcha Sniper" - Retry captcha 4 times
- "No captcha" - Skips all captcha
- So we can know which sites don't use captchas and keep them out of the equation.
The general idea for each list (test) is the following:
- By using content scraped from very similar keywords using the same software, we minimize the chance that content is affecting the results
- We're working with "successfully submitted" numbers
- Take the links from "DeathByCaptcha" as the maximum number of links you could expect from other projects
- Subtract the "NoCaptcha" submitted links from the "DeathBycaptcha" links to get the number of sites that use captcha
- Calculate the success rate for CS & CB projects based on that number
The results are in...
Now while this all seems pretty close, let's put things into perspective with some math.
First we need to look and see that "DeathByCaptcha" submitted 7,074 while "NoCaptcha" submitted 4,446 links in total.
From this we can conclude that around 2,628 sites have captcha. Now we take our "Captcha Breaker" & "Captcha Sniper" projects and subtract those4,446 links that didn't require any captcha.
After doing so, we see that:
"Captcha Breaker" successfully posted to 2,097 sites with captcha = 80% success rate
(2,097 / 2,628)
"Captcha Sniper" successfully posted to 1,452 sites with captcha = 55% success rate
(1,452 / 2,628)
Also, I've used Test #2 as a speed test and here is how long it took each project to go through the entire list:
"No captcha": 30min
"Captcha Breaker": 33min
"Captcha Sniper": 42min
My 2 cents...
Take what you will from this test but if you add the fact that CS has some stability issues and CB is updated daily while CS is updated every few months, I think we have a clear winner. Even tho CB costs some $40 more than CS and you might be on a tight budget, I recommend spending those extra $40 here and saving yourself from loads of frustration down the road.
Also, I was amazed when I saw that CB isn't that far behind DBC (and in one of the specific tests I ran, it actually performed better than DBC lol). As the OCR gets even better, it could very well be the beggining of an end for all pay-per-captcha services out there. Cause why would you constantly have to buy new captcha packages when there's a one-time fee software our there that can handle 80%+ of captcha being thrown at it? (Even more when combined with Spamvilla which is yet to justify it's steep monthly price)
If you're interested in more tests including only specific types of links (Social Bookmarks, Web 2.0s & Wikis) you can find them in the original blog post.
On the other hand, if you're a CS user and want to repeat this test for yourself, just download the CB demo and go for it. All the details are here, if there's something I missed, comment below and I'll do my best to explain it.