# robots.txt for http://inclusion.ngfl.gov.uk/ and http://tre.ngfl.gov.uk/ User-Agent: * Allow: / # disallow msn User-Agent: msnbot Disallow: / # disallow yahoo User-Agent: Slurp Disallow: / #disallow google User-Agent: Googlebot Disallow: /