Email spider won't return search result in google uk

coin666

@sven, email spider was acting weird for me, it won't return any result when i use search engine with any keywords. I tried:

1. with proxies, without proxies, public proxies, private proxies, all return 0

2. with single keyword, with keyword file, with simple keyword, with keyword include advance operator such as inurl, no result

using the same proxies, same keywords i can get thousands of url by using Scrapebox. I can just import those URL to email spider and ask it to scrape email. but that is... if email spider can operate as what it should, i can save my time. 

please advice. thanks. 

Sven

An "it's working here" is very frustrating but It is...I have no idea other than your proxies and IP is all blocked. Or do you maybe have some wired filters that prevent it from scrapping? Try using Project->Reset Config and try using other search engines who do not block so fast (like MSN/Bing).

coin666

@sven, not sure if I found a bug. I reseted the config, and it works, but later when I right click the search engine list, choose none, then checked on google gb, it just won't scrape. If I don't right click and select none, just checked off google us, and checked on google gb, it will scrape. 

ok so i found a workaround on my problem, but not sure if this is a bug or what. Thanks!

coin666

@sven, just found out no, it's not a bug, but it just won't scrape google.co.uk, anything else I can do? Scrapebox can scrape with same proxies and keywords in google.co.uk... 

Sven

Just did a test here with "google GB" and keyword "software development". It's working fine for me. All I can suggest is using WireShark and see what is returned by google.co.uk

coin666

Thanks @sven, will try that. Another question about extra data using special tag. ES won't extract name field, here's the HTML of the field in the page: 

    <ul class="list-unstyled">

                            <li itemprop="employee" itemscope itemtype="http://schema.org/Person">

                              <p>

                                <span class="text-primary" itemprop="name">DAVID MUCHEMU</span><br/>

                                <span class="text-muted" itemprop="jobTitle">Founder and CEO</span><br/>

                              </p>

                            </li>

                            <li itemprop="employee" itemscope itemtype="http://schema.org/Person">

                              <p>

                                <span class="text-primary" itemprop="name"> </span><br/>

                              </p>

                            </li>

                        </ul>

The highlighted is the name field i wish to extract as extra data. In Extract extra data within special Tags:

Front: <span class="text-primary" itemprop="name">

Back: </span>

Here's the option page: 

What should i do to extract that data? 

Here's the example URL: http://www.manta.com/c/mrns5s7/cgmp-university-inc

Sven

found the issue and fixed it in next update. The problem was that there have been two tags with same name, one was empty one not.

coin666

Thanks @sven!