# https://www.robotstxt.org/robotstxt.html User-agent: * Allow: / # Sitemap Sitemap: https://gohorsejobs.com/sitemap.xml # Crawl-delay for polite bots Crawl-delay: 1 # Disallow admin/dashboard areas from indexing Disallow: /dashboard/ Disallow: /api/ Disallow: /login Disallow: /cadastro/