Skip to content

Private vs. Semi-dedicated Proxies - 10 vs. 20 threads per proxy

edited April 2014 in Other / Mixed
I had some time on my hands to try running another technical test. This time it was about if there is a difference in how proxies behave when stressed and if there is a difference between semi-dedicated and private proxies.

I took 5 private proxies and 9 semi dedicated proxies (lost one on the way) running them at 100 threads (100/14 = 7.14 rounded down = 7 threads per proxy) and later at 300 threads (300/14 = 21.43 rounded down = 21 threads per proxy). All tests were done on a 20 / 20 schedule (20 threads changing every 20 minutes)

Results: at 7 threads per proxy the private proxies appeared in a fail message an average of 1.09 times per minute, there was not much variance, the highest being 1.25 and the lowest 1.00 times per minute.

shared proxies however participated in the failed messages an average of 2.43 times a minute with the highest being 2.8 and the lowest 0.9 (guess I am the only user of that "semi"-dedicated proxy, and it may have slightly lowered the number of fails for this kind of proxy ).

Even at 7 threads per proxy the private proxies produce more than twice the reliability of semi dedicated proxies (1.09 vs 2.43)

at 21 threads per proxy the private proxies participated in an average of 1.74 messages per minute, which is an increase of 60% compared to the 100 thread fail messages. compared to a 300% increase in stress this does not really seem like a lot AND private proxies still perform better than semi-dedicated at 100 threads (1.74 fail messages vs 2.43)

with 300 threads shared proxies average a 4.54 message per minute fail count which is an 85% increase as compared to running on 7 threads per proxy. Compared to private proxies at 300 threads semi proxies produce 261% of the participation in fail messages.

Conclusion: based on the above I think that private proxies perform much better than semi proxies, even when stressed. It would even be more beneficial to get dedicated proxies instead of semi. Stressing the private proxies with twice as many threads would 1.) produce a better outcome as far as reliability is concerned and 2.) keep the cost very much the same as with semi-dedicated proxies.

notes: I haven't gone above 300 threads per proxy as my test machine was already stressed at 95% capacity. A 400 or 500 thread test would have resulted in many fails not related to proxies.

Comments, thoughts and ideas? - post them below!

Comments

  • goonergooner SERLists.com
    You've done it again man! Awesome test :)
    Very thorough and very conclusive i would say.
    I'm a bit of a stats freak so i love these kind of split tests because there you have it in black and white, the numbers can't lie.

  • thanks :), was quite surprised at the findings. The only thing that worries me is that if you are a low end user e.g. when you only have 10 semi proxies and get 5 private ones the footprint may be too obvious (not sure how to even test this :/, but then again at 10 semi proxies the footprint is there as well.
  • KaineKaine thebestindexer.com
    How long so few proxies are banned ? 

    That is the more importante question for me ;)
  • ronron SERLists.com
    edited April 2014

    Alright @Ferryman, I'm trying to get to sleep, and first you throw a super intellectual video at me that took 19 minutes to watch, and now this. ROFL!

    I swear over a year ago (18 months ago when I tested) it made no difference. But you took the test in a different direction than I did, as I just ran each for a month and compared results. But it was not very scientific as I probably updated 60 ser versions in that 60 day period, lol. So it was completely invalid in hindsight.

    Your methodology was superior to my test. You did a much better job of freezing the other variables to isolate what you were testing. I would regard it as a very valid test. One thing I would want you to mention is how long you ran this test. 60 minutes? 12 hours? etc. And did you stop verification to ensure that was factored out? Regardless, I think you are really on to something.

    The team was talking (and I am not as smart as those guys so this wasn't me), and they are testing the theory that you can make more links at lower threads than higher threads - a similar concept that has to do with quality and stress related factors.

    Excellent analysis. I'm sold!   


  • +1 Ferryman - Very valuable data indeed
  • edited April 2014
    @ron - My LpM did not rise 3x with 300 threads as compared to 100 threads (not exactly sure though - just took a glance at the numbers) and it was not the proxies fault. Maybe CB, or captcha eater .. don't really know

    the test on 100 threads was run for 275 minutes, the 300 thread one for 81 minutes - the results are relative to the number of minutes run (that is why fail messages per minute). The length does not matter that much as results are constant within their type (private proxies and semi performed consistently during the tests - it is a split test, done at the same time with the same resources and the same list, not many variables here except the type of proxy. Afaik there is no IP in verification messages (at least i didn't find any when filtering the log). Also there is no IP in CB error messages, so that can be ruled out as well.
  • edited April 2014
    @Kaine - with the amount of links built I really don't think there is a difference if you have 10 semi or 5 private, the same goes for 100 semi or 50 private - semi dedicated means someone is using them with you (usually 2 other people) which means they do use them for more or less the same thing and the proxies will be banned as quickly (you just have more unknown variables in the equation with semi). That would be an interesting question to ask proxy providers though...

    @coneh34d Thanks :)
  • Nicely done Ferryman. Where's the affiliate link to dedicated proxies, though? 

    Kiddin' :D seriously this is a cool test as I was just about to push the button to buy a tonne of semis to go with my dedis, but not so sure now.
  • edited April 2014
    @JudderMan - sorry, all private proxies I set up myself, so no affiliate link :D
    Hmm, maybe buy 5 dedicated from a provider and test them against mine ... interesting idea. Thanks :)
  • Ferryman did you use any recaptcha solving service or just skipped all recaptchas. Private proxies are good, however with lower numbers you will get double blob captchas much sooner than with 2x more semidedicated proxies. On one server I run project with 40 dedicated proxies and even on 150 threads I get a lot of those hard to solve recatpchas at the end of the day.
  • @RayBan - I used a recaptcha service. the problem here is I will not be able to test this as there is no message on succesful captcha. From what I spoke to a person involved in solving captcha you get the difficult ones for one hour after 30 consecutive failed entries. For me it doesn't seem to matter as i always seem to get the dificult ones.
  • edited April 2014
    @ron - just went through the results again and removed the influence of "recaptcha url resolve failed" (the message has an ip in it), here are the results without recaptha url resolve failed:

    fmpm = fail message participation per minute

    100 threads priv - 0.79 fmpm, 300 threads priv - 1.21 fmpm, 55% increase
    100 threads semi - 1.73 fmpm, 300 threads semi - 2.98 fmpm, 68% increase
    100 threads semi fail rate compared to priv - 219%, 300 threads semi fail rate compared to priv - 246%, 12% increase
Sign In or Register to comment.