← All tools
Robots.txt Generator
Generate a robots.txt file by choosing which paths search engines may crawl and adding your sitemap URL. A robots.txt file is a request, not a security control: well-behaved crawlers respect it, but it does not block people or hide private pages.
robots.txt
User-agent: * Disallow: /admin/ Disallow: /private/
Robots.txt is a request to well-behaved crawlers, not security. It does not hide private pages.
What this tool cannot do
This generates the file text only. You must upload it to your site's root yourself, and it cannot block malicious crawlers or protect private content.
How to use it
- Enter your sitemap URL if you have one.
- Add paths to disallow, one per line.
- Click Generate and copy the file.
- Upload it as robots.txt at your site's root and test it in Search Console.
Frequently asked questions
Does robots.txt hide a page?
No. Disallowed pages can still be linked to and may still appear in results. Use noindex or password protection for private pages.
Do I need a robots.txt file?
No, but it is useful to guide crawlers and to point them at your sitemap.
More SEO tools
Keyword Research HelperKeyword Density CheckerKeyword Suggestion ToolMeta Title and Description GeneratorMeta Tag AnalyzerSERP Snippet PreviewWebsite SEO AnalyzerXML Sitemap GeneratorXML Sitemap ValidatorSchema Markup GeneratorOpen Graph PreviewHeading Structure CheckerBroken Link CheckerURL Slug GeneratorRedirect CheckerWord CounterReadability CheckerCanonical URL CheckerInternal Link Checker