silverstripe-webpack/robots.txt

7 lines
150 B
Plaintext
Raw Normal View History

2018-02-05 12:11:01 +01:00
User-agent: *
Allow: /
Crawl-delay: 10
Disallow: /admin
Disallow: /Security
Sitemap: https://www.bla-bla.org/sitemap.xml
Host: https://www.bla-bla.org