# Disallows cgi-bin and images # Allows only major search engines and known friendly spiders # Major Search Engines and Known Friendly Spiders (allowed) User-agent: Googlebot Disallow: /cgi-bin/ Disallow: /images/ User-agent: MSNBot Disallow: /cgi-bin/ Disallow: /images/ User-agent: Slurp Disallow: /cgi-bin/ Disallow: /images/ User-agent: Teoma Disallow: /cgi-bin/ Disallow: /images/ User-agent: Twiceler Disallow: /cgi-bin/ Disallow: /images/ User-agent: Gigabot Disallow: /cgi-bin/ Disallow: /images/ User-agent: Scrubby Disallow: /cgi-bin/ Disallow: /images/ User-agent: Robozilla Disallow: /cgi-bin/ Disallow: /images/ # Everyone Else (NOT allowed) User-agent: * Disallow: /