Free tool

Robots.txt Checker

Paste any URL to read its crawler directives: groups, rules, and sitemap declarations.

What this tool checks

The robots.txt file is the first thing crawlers read. It declares which bots may visit which paths, and one stray Disallow: / can deindex an entire site overnight.

We fetch the live file, split it into per-bot groups, and flag dangerous patterns like site-wide blocks or missing sitemap declarations.