User-agent: * Disallow: /404.htm Disallow: /400.htm Disallow: /500.htm Disallow: /go.asp Disallow: /register.asp Disallow: /login.asp Disallow: /search.asp /sitemap.asp