Skip to content

Old SER Install Double Amount of Verifieds

BrandonBrandon Reputation Management Pro
edited April 2013 in Need Help
I have a few SER installs, my oldest is on my slowest server but it typically does double the amount of submitted and verified links when compared to the newer servers with double the processor, triple ram and SSD.

I've copied the verified links text files into the new installs but that didn't help. 

Any idea why my one server is so much better than the others?  All settings are identical, they share the same proxies and the number of type of campaigns are very similar.

I have tested the number of threads and the old server typically can only run 100 threads while the others run 250.

TIA for any advice or suggestions.

Comments

  • BrandonBrandon Reputation Management Pro
    I don't know if it matters, but this good server also gets a lot of "Out of Memory" errors even when running just 3 total campaigns.  Memory is at 973MB right now running 100 threads, 3 campaigns and high CPU usage.
  • BrandonBrandon Reputation Management Pro
    Just a bump.  Right now my old slow server is nearly triple the verifieds when compared to my new high speed servers...
  • LeeGLeeG Eating your first bourne

    First red flag is the same proxies on all servers

    Why do people do this idiotic idea?

    Split the proxies into equal amounts between the servers

  • LeeGLeeG Eating your first bourne

    Another problem you might have is the engines you use

    If you use blog comments and your trying to hit those second time, they might have reached your obl limit of the twats that post profanities and use words in your bad filter have hit

  • BrandonBrandon Reputation Management Pro
    Thanks @LeeG Why would separating the proxies be beneficial?  Maybe I'm just missing the logic.  In my mind, while one server might be doing a lot of verifying, the proxies would be faster for the other servers.

    I've separated my engines out and use presets for the good engines that I've identified by having a good amount of verified links.  I typically don't use blog comments.
  • LeeGLeeG Eating your first bourne

    The logic. Sorry I seemed blunt with that and no explanation

    You never mentioned how many you use

    The explanation

    When you buy proxies, you look for the best deal on how many are sharing them

    Less people, less chance they are being rammed and banned (banned by the search engines, too many search queries at the same time by the same proxies)

    Proxy Hub who I use, say its a maximum of three people have access to each proxy

    Your running the same proxies on three machines at the same time

    So if your in a group of three using those proxies, you have now made it a group of five

    Two others plus your three servers ramming them

  • BrandonBrandon Reputation Management Pro
    I'm running 50 private proxies from BuyProxies across 3 servers.  Think there is a benefit in splitting them?

    Any other ideas why my slowest/oldest server does double the verifieds every day?
  • LeeGLeeG Eating your first bourne

    How many threads on each server?

    But, imo, split the proxies into chunks equal to your threads

    ie 100, 200, 200 The split 10,20,20

    Im not a fan of your proxy supplier in all honesty

  • BrandonBrandon Reputation Management Pro
    For my servers I run 75/150-250/150-250  I've been testing to see if I get better output with more than 150 threads and the only benefit seems to be during verification.

    I might give ProxyHub a try.  I'm still confused as to why my single 3.2 ghz server is running twice as fast as my other two which are dual 3.2 (6.4ghz) with SSD's and more ram.  Any idea on that issue?

    I can post screenshots if that might help but all of my settings are the same across the servers.
  • LeeGLeeG Eating your first bourne

    What are your search engine query times set to?

    Looking at that, if it was me, I would be running 30 to 50 proxies on each vps running 250 threads

    So I would have about 130 proxies

  • BrandonBrandon Reputation Management Pro
    Query times are 20 seconds on each server, timeout is 30 seconds.  The first server gets the same proxy usage as the other two.  I've tried importing the found/submitted/verifieds to the other servers but that didn't help the verifieds, and my old server still runs circles around the new ones despite running half the total threads and half the CPU.
  • LeeGLeeG Eating your first bourne

    Verified will be low with a 30 second time out

    Im on running 50 proxies on one machine and mine is set to 130 second timeout

    Set your search quires to default until you sort out the proxies

    And your hammering the proxies, you need to split them into groups for each server

    You have three separate machines hammering away on the same proxies with a 20 second delay between search queries. That's normally for a set of proxies on a single machine. Your queries could be less than a second under the right circumstances

  • BrandonBrandon Reputation Management Pro
    I picked up 100 semi-privates from proxy-hub.com.  I'll put those on one server and see if that does anything.  Thanks for that suggestion.
  • LeeGLeeG Eating your first bourne

    Good thing with proxy hub, their proxies last well and don't need 24/7 support

    Split them into 2 lots of fifty and then use them on two of your servers

    I run mine on about a 10 second time between search queries

    Bump up the time out on verified to about the 130 / 140 mark and your verified should start to jump up as well

  • AlexRAlexR Cape Town
    @LeeG - I think we started this discussion on another thread but it was never completed. ;-)

    "time out on verified to about the 130 / 140 mark"

    So what you're saying is wait up to 2mins + for the first byte to get sent from the website. Surely this is way too long and if you're waiting this long there is probably an error with the website anyway. Most sites load in under 30s, let alone send the first byte in well under 10s. 

    Why such a MASSIVE amount of patience with a website to send the first byte? Makes no sense to me. Surely it would be better to lower this and spend the 2 mins finding a new targets?
  • LeeGLeeG Eating your first bourne
    edited April 2013

    From my own testing, its mainly the verified that it affects

    I have set up a few peoples copies in the past and by simply altering that one time, made a massive improvement to the verified rate

    One friend I did it for, touching nothing else and his verified rate doubled

    Maximum time you can set it to is 180

    What I personally do is set it to 180 and then work the time down down while its going throught verifications. Too many timeouts and increase the time.

    The harder you push a machine, the more resources your using etc

     

  • AlexRAlexR Cape Town
    @LeeG - interesting that it affects verified rate. Do you have any idea why? It's just that 3 mins is a ridiculously long time to wait for a site to send the first data byte, so I can't get my head around why it makes sense to have this number high?

    @sven - what are your thoughts on this? Surely a site sends the first data byte within 10s in most instances, so why would setting it to 180 make a difference?
  • SvenSven www.GSA-Online.de
    if you increase threads you should also increase that setting. With many threads you got so much traffic that it is important to give each thread it's time to finish.
  • AlexRAlexR Cape Town
    @sven - so the HTML timeout is for all threads not on a per thread basis right? If so, surely a safe time to use would be around 5s for a website, so 5s per thread. So if you have 200 threads then set it to 1000s? 
  • LeeGLeeG Eating your first bourne

    Alex there are a number of different factors that affect this.

    Speed of machine, speed of proxies etc etc

     

    Don't question, just do, you will confuse yourself and hurt your head trying to understand a simple task as per normal. :O

    Im trying to help you, not be nasty here

     

    Max your time out to 180

    Then, when its verified link checking, drop the time in increments of 10

    170,160 etc until you find your sweet spot

    Average sweet spot is about the 130 to 140 area

    Watch for the amount of timeout errors you get.

    Too many, increase the timeout setting. Very few, drop it a bit more

     

    One trick to put ser into verify check mode is to stop it, restart the machine its running on and normally within fifteen minutes of running ser again, it goes into verify mode on everything

  • AlexRAlexR Cape Town
    @LeeG - thanks for the details. I have it set around 180. Will follow your advice and lower. I just like to understand things too. 

    Do you know how many times SER tries to verify? Isn't it as per interval until 5 days are up and then if not found gets removed? 
  • OzzOzz
    edited April 2013
    the interval of verification is dependend on the engine. just open some engine scripts and watch out for this line:
    verify interval=XXX

    XXX are the minutes between each verification attempt. you are able to change the verification interval on project basis like you prefer though if you don't want that SER manage this for you automatically.
Sign In or Register to comment.