# Block Bytespider bot User-agent: Bytespider Disallow: / # Default MODX exclusions User-agent: * Crawl-delay: 10 Disallow: /assets/cache/ Disallow: /assets/docs/ Disallow: /assets/export/ Disallow: /assets/import/ Disallow: /assets/modules/ Disallow: /assets/plugins/ Disallow: /assets/snippets/ Disallow: /install/ Disallow: /manager/ Disallow: /cpcfiles/ Disallow: /old/ Disallow: /old # For sitemaps.xml autodiscovery. Uncomment if you have one: # Sitemap: https://yourdomain.com/sitemap.xml