Log: Same two urls

Hi @sven

Was wondering why in the log each line displays the same url twice? Is this a bug or how it's supposed to be? 

Tagged:

Comments

  • SvenSven www.GSA-Online.de
    Accepted Answer
    Thats no bug but on some log lines you have an url supplied to it, on some not. That line with two urls is a log line with url in message and the url behind it.
  • cusabio1cusabio1 www.cusabio.com
    Accepted Answer
    The site under http://foo.com can be the same as under http://www.foo.com, but it can be a totally different site, due to web server configuration. It depends on the DNS config too, which IP  kit points to www and which one to without www.

    If you want compare two sites, you need to fetch the content, and compare key parts (using nokogiri for example) about similarities.

    Nowadays due to sidebars and news, two consequent request to the same url, gives slight different html responses.

Sign In or Register to comment.