Question on adding your own search engine
Hi Guys,
Just a quick question, i am scraping my own targets and am adding in real time to a database, i am adding the url for example:
http://www.site.com/crawler/api-crawler.php?s=1&page=1
I notice in the logs while running GSA is trying to search for a string like:
19:31:47: [ ] 007/250 [Page END] results on 24/7 Scraper for Joomla K2 with query "Create an account" "Forgot your password?" "Login Form" "Remember Me" "Log in" "Forgot your username?"
is this right enough? or can i turn the search query off?
cheers guys
Graham
Comments