Why Won't AI: AI Crawler Checker

See which crawlers a site's robots.txt allows, and sync your ChatGPT Business Hub insights to Why Won't AI.

Click the icon to see what the robots.txt file of the current site allows and blocks. The extension shows a list of common crawlers, whether each one is allowed or blocked, and the exact line in the file that decides it. You can also type the address of another site and check it. A crawler that is allowed can still be stopped by a firewall, or get an empty page from a site built with JavaScript. The free scan at whywontai.com shows what crawlers get from your pages. No account. No sign-in. No tracking. ChatGPT Business Hub sync (optional): if you use Why Won't AI and click Connect ChatGPT on your dashboard, the extension sends the insights your own Business Hub page shows (mentions, website clicks, discovery topics and business agent stats) to your Why Won't AI account each time you open Business Hub. It does not read your chats, cookies or passwords, and it does nothing until you connect.