Generators
Robots.txt Generator
Generate a robots.txt file with common rules for search engines.
Source
Select your CMS platform
Crawl Policy (Optional)
Allow all crawlers. Add Disallow paths to restrict areas.
Paths
Disallow
Allow
Sitemap URL (Optional)
Crawl-Delay (Optional)
Output