# rules from http://www.robotstxt.org/wc/norobots.html # Make changes for all web spiders User-agent: * Disallow: /al-t/ Disallow: /nmr/ Disallow: /images/ Disallow: /cgi-bin/