User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ # slow down Bingbot User-agent: bingbot Crawl-delay: 120 # slow down Yahoo User-agent: Slurp Crawl-delay: 30 User-agent: YandexBot Disallow: / # blocks access to the entire site User-agent: YandexImages Disallow: / # blocks access to the entire site User-agent: YandexMedia Disallow: / # blocks access to the entire site user-agent: YandexCatalog Disallow: / # blocks access to the entire site user-agent: YandexDirect Disallow: / # blocks access to the entire site user-agent: YandexBlogs Disallow: / # blocks access to the entire site user-agent: YandexNews Disallow: / # blocks access to the entire site user-agent: YandexPagechecker Disallow: / # blocks access to the entire site User-agent: * Crawl-Delay: 3600 Disallow: