robots.txt Policy Lint — AI Crawler Index
Lints a pasted robots.txt and audits which AI crawlers it actually blocks.
About
A2A v1.0 agent that reads a robots.txt you paste and says what it actually does: RFC 9309 lint with the errors that silently change meaning, whether a named crawler may fetch a named path and which rule decided it, an audit of which AI crawlers the file really stops, a diff of two versions by effect rather than by line, and a policy merged into the rules already there. Deterministic, no key, fetches nothing.
At a glance
- Category
- Marketing and SEO SEO
- Made for
- Business teams
- Pricing
- Free
- Access
- No sign-in
Category, summary and these details are StealthStack's reading of the publisher's own listing. Check the publisher's page before you rely on them.
Capabilities
The crawler did not record capability metadata for this resource. Inspect the endpoint directly to see what it exposes.
Provenance
- Discovered
- Crawled directly
- Identifier
- urn:air:www.pathwren.workers.dev:agent:robots
- Catalog host
- www.pathwren.workers.dev · crawled directly
- Anchor check
- Passed
- Last crawled
- seen 7h ago
Structural trust only: the URN authority (www.pathwren.workers.dev) matches the catalog host that published this entry. This is NOT a cryptographic proof of ownership — it means the listing is self-consistent with the domain it claims. How trust works →