Skip to content

How many proxies for PageRank checking?

CharybdisCharybdis
edited March 2013 in Need Help
Hi,

let's say I have X number of private proxies, and I use them also for PageRank checking. How many threads should I run in GSA without Google banning my proxies for excessive PageRank checking?

Is it possible to create a cloud based service for PageRank checking? If someone has already checked the PageRank of example.com, then he could upload that information into a cloud based service, so that everyone will know that example.com has PageRank X, so no need to waste proxies. Of course the whole database needs to be updated on every PageRank update.

Comments

  • rodolrodol
    so you want a cloud with all the scraped links of everyone, to let everyone spam all your links.... this is like a scraped list database and that's a big NO.... no one will share their whole list just to let you know the PR.
  • sagasaga
    Also, PR's do change (not often but they do) so that entire list would need to be changed and updated.
  • CharybdisCharybdis
    ok but how many proxies and how many threads do you use to avoid banning for excessive PR checking?
  • ambitionambition
    edited March 2013
    I have wondered if a PR cache server would be a possibility too.  Check the PR against the cache and if it already knows the PR it doesn't check Google.  If it doesn't know it it queries Google and updates the cache server.  The server could expire what it knows every 30 days or something.  I don't see why this couldn't be a highly secure server that only Sven and company could access, thus eliminating the exposure of the full list to anyone.  The other possibility would be to have the cache server check sites that aren't in the list when it is idle or below a certain usage percentage.  That way it eventually becomes a cache of Google and not a cache of our list.  At that point the security wouldn't matter and anyone could set up a cache server to help shoulder the load.  I would gladly volunteer one of my web servers for this project as long as a unix/linux variant could be written.
  • rodolrodol
    I understand that checking PR is annoying but your solution is hard to implement, expensive and is not the best way, save some money and buy some private proxies maybe 30-40, that can solve your problems.
  • ambitionambition
    edited March 2013
    I do use private proxies.  If I understand correctly each proxy check is a Google request.  That means you can't hit Google for another X number of seconds with that proxy.  You could speed up your searches, etc without having to use up those requests.  It's more efficient all around, plus it solves the problem for those who can't afford private proxies or can only afford 10 or so.  Whether it is worth the cost and time to develop is up to the GSA team.

  • CharybdisCharybdis
    Isn't caching PageRank against the Google Terms of Service?
  • TankTank
    edited March 2013
    This feature is NOT worth the cost and time to develop. What adding a cache feature would do would cost sven more time and money. He would have to pay more money for server bandwidth, resources and maintain the cache, because there would be tons of people querying his server.

    If this program had a month fee then I might agree, but its already cheap as hell, way to cheap.
Sign In or Register to comment.