Robots.txt checker

See whether common AI crawlers are blocked and copy a suggested allow snippet.