Robots.txt Generator
Generate robots.txt for your website
Rule 1
Rule 2
User-agent: * Allow: / Disallow: /admin/ Disallow: /private/ Disallow: /*.json$ User-agent: Googlebot Allow: / Allow: /blog/ Sitemap: https://example.com/sitemap.xml
About Robots.txt Generator
Generate a robots.txt file for your website. Control which pages search engines can crawl. Free.
Frequently Asked Questions
What is robots.txt?
robots.txt is a file in your website's root that tells search engine crawlers which pages not to index or crawl.