User-agent: * Disallow: /tmp/ Disallow: /cgi-bin/ Disallow: /webalizer/ Disallow: /modlogan/ Disallow: /administrator/ Disallow: /wp-admin/ Crawl-delay: 20 User-agent: AhrefsBot User-agent: Ezooms User-agent: MJ12bot User-agent: XoviBot Disallow: /