Question about search engine not found

ahmethann

Hello Sven,

Here the part that I took from logs:

[11:43:13] MSN US: [000/000] [PAGE 006] with "Denton steal" - http://www.bing.com/search?q=Denton steal+loc:US&filt=all&first=51&FORM=PERE

[11:43:13] MSN US: no results found in 3 tries, aborting.

It is making search on MSN US for above keyword at the page 6. And after that it says "no results found in 3 tries". However, there is search results on that page moreover there are other search result pages also.

So my questio is:
What is the reason that it can not find search results, even there are many pages and search results. Does search engine  block the certain IP because of repetitive queries and not show results for the certain IPs? Or any other reason?

Sven

it can have to do with bing blocking your IP or proxy

ahmethann

Ok, then is there a way to detect this (i mean if there is a block or if we really got to last page of search results)?

If there is a way to detect we got blocked it would be very great if it RECHECK the search results after the IP cooled down. So as far as I understand if we are blocked by the search engine we are skipping the result and we will not again turn back if there is result or not later in some time?

Sven

you have an option for that ...just enable the option "Repeat search when finished".

ahmethann

SO does it repeat the search when "the whole project finished" or when "currenct search term for the current search engine finished"?

Also what does "skip keywords that have been processed" do?

Sven

With this option on, the project will never stop on the search task. Once something finishes, it will repeat it.

"skip keywords that have been processed" < This will simply only use new keywords or those where the end-page of the search engine parsing was NOT reached.