How do I harvest domains on US Google?

MelloYello

I bought 10 private proxies and I have been having issues getting the software to work with Google in US. It will give me errors and it will start then finish within seconds and it won't render any of the content in the status logs. I switched it to Bing and it worked well but Bing isn't my favorite for what I am looking into because it has a lot of spammy websites. I even tried yellow pages with my keywords and that wasn't getting me anything close to what I was searching. 

So is my issue my proxies that I am using or is it how I have the software set up for google? Do I need to have a specific amount of threads? I am new to this so I apologize for any rookie questions. I just know that when I test my proxies against Bing they work, then when I test them against google search only 2/10 work and when I start with those it won't be able to submit the content in the logs. 

If it is a proxy issue where should I go to get more reliable proxies that work on Google in the US?

Thanks!

Deeeeeeee

Proxies are like a radio repeater. You>>>Remote
                                                         <<<System

Proxy:  Your<<---PROXY<<--Remote
            VPS -->>            -->>System

What error msgs are you getting? Likely, you proxies are NOT ok to "scrape",or get targets from Google.  Such proxies  are VERY specific. Reverse rotating proxies are good for this.  They hit Google (and other SEs) every time with a different procy out of a pool of many, many...


Don't hurt your private proxies scraping G, especially b/c if u aren't expressly permitted to do so,  u could lose privileges.

MelloYello

Gotcha. So the log looks like this:

starting to parse for keyword
checking/sending disabled do to scheduler(don't even have that option checked so I don't know why it's giving that error)
No content delivered, aborting(for all the keywords)

But when I switch from Google to Bing for the search engines Bing works very well while Google gives error messages. 

the_other_dude

thats because google banned your proxies. just don't scrape google and set a time delay between queries. I use 120 seconds.

MelloYello

Oh ok. Do you use Google as well or different search engines? I am just curious because Bing returns a lot of crap that surprises me at times. I am talking businesses not even in the same general location that I am searching for. 

Thanks I appreciate everyone answering these questions. 

the_other_dude

Yes, I scrape google. I also have 260 dedicated private proxies, that only I use, and I do not use SER to scrape for targets. I use scrapebox. I also scrape bing too, way less issues. 

Here's the deal, if your proxies are getting banned by google, you have to wait 48 hours to try again, or else everytime you try to connect to google the ban restarts. So just wait 48 hours before trying to scrape google again, and when you do set your delay to AT LEAST 120 seconds or you're just going to get banned again.

the_other_dude

Google has the best results, no question. But if you are going to try to scrape google you have to have a lot of proxies and do it right, meaning slowly. 

Deeeeeeee

"...But if you are going to try to scrape google you have to have a lot of proxies and do it right, meaning slowly.  ..."

So would you say that any private proxy could conceivably be used to scrape Google? I see your point; if u set interval long enough, seems like normal usage, not hammering G with search queries...

But still, some private proxies expressly forbid this, no?? 

the_other_dude

I suppose some companies might not want you scraping Google or using SER with their proxies. I've never dealt with any company that was against any SEO tool that I own. And I wouldn't, for obvious reasons. Google is who bans the proxies, they even will ban whole subnets from accessing their search results. 

Deeeeeeee

"Google is who bans the proxies, they even will ban whole subnets from accessing their search results. "

Interesting truth worth knowing. 

Do other SEs also ban proxies?