robots.txt Generator

Build a robots.txt file by defining crawler rules, allow/disallow paths, and sitemap location.

Rule 1

User-agent

Disallow Paths

Allow Paths

Global Settings

Sitemap URL (optional)

Crawl-delay (seconds, optional)

Preview

User-agent: *
Disallow: /admin/
Disallow: /private/
Allow: /public/