Result model

Robots.txt Checker result model

The report focuses on whether robots.txt is reachable, safe for crawling, and clear about sitemap discovery.

Severity ladder

Every report separates blockers, warnings, and healthy confirmations.

This is the reporting model behind the live tools, so operators know what to fix first instead of reading a flat list.

Critical

Fix these before relying on the current setup for monetization, indexing, or language targeting.

Warning

The page or file is reachable, but there is still technical risk worth reducing quickly.

Healthy

Public signals look aligned. Keep this state stable while you scale content or ads.

Signals included

  • File availability and response status
  • Risky global rules such as `Disallow: /`
  • Presence of a sitemap directive

Operator actions

  • Remove blanket blocking rules from production sites.
  • Keep important content and verification paths crawlable.
  • Add a clean sitemap line pointing to the canonical host.
01

Run the public check

Use the live checker to see what is publicly reachable and technically visible today.

02

Fix the highest severity first

The result model is intentionally ordered so blockers come before informational clean-up.

03

Re-run after deployment

These tools are built for before-and-after verification, not one-time screenshots.