# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 # User-agent: * Disallow: */3055-* Disallow: */3001-* Disallow: *?sort=* Disallow: /download-launch?* User-agent: Brightbot 1.0 User-agent: ClaudeBot User-agent: juzi.bot User-agent: GPTBot Disallow: / Sitemap: https://download.cnet.com/sitemap-category.xml Sitemap: https://download.cnet.com/sitemap-index-platforms-alternatives.xml Sitemap: https://download.cnet.com/sitemap-index-platforms-app.xml User-Agent: OnetSzukaj User-Agent: SputnikBot User-Agent: Nigma.ru User-agent: WebAlta User-agent: WebAlta Crawler User-agent: http://Mail.Ru User-agent: Mail.RU_Bot User-agent: Rambler User-agent: Holmes User-agent: 360Spider User-agent: startsiden User-agent: SeznamBot User-agent: Yeti User-agent: Sosospider Crawl-delay: 1