Okay so I went to sleep when this was happening and it looks like it is able to scrape, but at an EXTREMELY slow pace. Ran it all night and it only scrapped 2,000 sites. I'm pretty new to this and I'm only using the default proxies provided by website contact. Would I need a proxy service? I think my proxies could be potentially banned from search engines.
Comments