How to use it
Pick a ready template in simple mode, or define rule groups one by one in advanced mode. Copy the output or download it as .txt.
CRY Tools
Robots.txt Generator
Build crawler rules with correct syntax: multiple user-agent groups, allow and disallow rules, and sitemap declarations.
robots.txt
This tool runs entirely in your browser; your data is never sent to our server.
Examples
Blocking /admin for Googlebot while declaring your sitemap URL.
Use cases
New site launches, protecting staging, crawl budget management, e-commerce filter pages.
Privacy and data handling
Rules are generated in your browser; the paths and sitemap URL you type are never sent to our server or stored.
Frequently asked questions
Does Disallow remove my page from the index?
No. Disallow blocks crawling, it does not control indexing. Use noindex to keep a page out of the index.