RoboX - Live Robots & Technical SEO Checker

Instantly view status code, meta robots, X-Robots, canonicals & robots.txt. Run deep dive analysis & validate robots.txt & sitemaps.

Simplify your technical SEO analysis with RoboX by Wingmen. RoboX is the ultimate technical SEO companion for developers and SEO professionals. Validate the crawlability and indexability (HTTP Status, Canonicals, Meta Robots, X-Robots, Robots.txt & Sitemaps) of any website – for search engines and AI crawlers, individually or in bulk – in real-time directly in your browser – fast, secure, and privacy-focused. See critical SEO signals at a glance in the popup or the new sidebar, seamlessly switch your crawler perspective, and bulletproof your technical setup. 🆕 New in 5.0: Sidebar mode, AI crawlers by purpose, AI directives (Content-Signal, Content-Usage, License), AI visibility per platform, canonical chain to the end, smarter Link Analyzer, honest results behind bot protection & logins, backup & sync, built-in handbook. 🚀 Key Features: 1️⃣ Global Crawler Perspective & User-Agent Switcher • Set Primary Crawler: Define in the settings which primary crawler perspective (e.g., Googlebot, Bingbot, GPTBot) RoboX should use to view the visited URLs. All integrated features evaluate data dynamically from the exact perspective of this selected bot. • Integrated User-Agent Switcher: Load URLs directly with a simulated User-Agent of your choice – only in the current tab, all other tabs stay untouched. When using this simulation, the entire technical setup of the page is seamlessly evaluated from the exact perspective of this UA. Typical bots are pre-installed, and custom UA strings can easily be added. • Honest Simulation: If a bot protection (Cloudflare & co.) answers your simulated bot with a check page, RoboX tells you – and explains that real crawlers are recognized by their IP address, so a blocked simulation does not mean the real bot is blocked. 2️⃣ HTTP Status Checker & Advanced Headers • Instant Status: See the raw HTTP status code of the current page immediately – and a clear "?" instead of a guessed 200 if no server response was observed (e.g. back/forward cache). • Redirect Inspector: Traces the full redirect chain of the current page, documenting status codes for every single hop. • Robots.txt Block Detection: Warns you instantly if any intermediate URL in a redirect chain is blocked by the robots.txt for your simulated bot. • Raw HTTP Headers: Inspect the raw HTTP response headers individually for every hop. • Smart Header Warnings: Automatically detects critical SEO headers like hidden Refresh redirects, crawl-budget wasting Cache-Control directives, No-Vary-Search, or Vary: User-Agent. 3️⃣ Meta Robots & X-Robots-Tag Checker • Dual Analysis: Checks both HTML tags and HTTP X-Robots-Tag headers simultaneously. • Crawler-Specific Evaluation: Instantly detects if specific bot directives override general rules and calculates the final Index/Follow status exactly for your selected crawler. • Smart Conflict Detection: Warns you if tags conflict or if JavaScript injects/modifies meta tags after the initial page load (DOM vs. Raw HTML). • AI Visibility: Shows per platform (Google AI Overviews & AI Mode, Microsoft Copilot, Apple Siri & Spotlight) whether the page can appear in AI answers – based on nosnippet, max-snippet, data-nosnippet, noindex, noarchive/nocache and paywall markup. 4️⃣ Canonical Checker • Dual Analysis: Analyzes both HTML and HTTP Header canonicals to detect mismatches or conflicts. • Canonical Chain Deep-Check: Follows canonicals and redirects to the very end (up to 5 steps) and validates every step's HTTP status, robots.txt allowance, and indexability. Flags loops, errors, noindex targets, blocked targets, redirects, and canonicals pointing to other canonicals. 5️⃣ Robots.txt Checker & Bulk Testing • Instant Status: See immediately if the current URL is allowed or disallowed for your selected bot. • RFC 9309 Validation: Automatically checks syntax against official standards. Detects critical errors, typos, and deprecated directives. • AI Bots Dashboard: See at a glance if the current URL is blocked or allowed for more than 70 AI crawlers (like GPTBot, ClaudeBot, Perplexity, etc.) – grouped by vendor or, new, by purpose: training, AI search, user fetch, and agent. Crawlers that may ignore robots.txt for user requests are clearly marked. • AI Directives: Reads Content-Signal (Cloudflare), Content-Usage (IETF draft), and License (RSL) and shows which AI usage preferences apply to your bot – with line numbers. • Google Testing Tools: Warns you if Google's Rich Results Test or URL Inspection are blocked by the robots.txt although Googlebot is allowed. • Bulk Tester: Check up to 100 URLs simultaneously against the live robots.txt or your own custom code snippet directly in the extension. 6️⃣ Blocked Resources Analyzer & Rendering Simulation • Risk Assessment: Automatically identifies CSS, JS, API, or Font files blocked by the robots.txt and estimates their potential risk level to the page's rendering. • True Rendering Simulation: Opens the current page in a new tab, temporarily overrides your browser's User-Agent with that of the selected bot, and strictly blocks all resources (like CSS or JavaScript) forbidden to it via API according to the robots.txt. This lets you visually verify exactly if important layout elements remain invisible to the machine. 7️⃣ Link Analyzer & Visualizer • Powerful Extraction: Scrapes all on-page links, retrieving their exact HTTP status, Canonical targets, and Robots directives – dynamically evaluated from your current crawler perspective. • Link Attributes & Problem Targets: Detects nofollow, ugc, and sponsored and flags internal links pointing to noindex pages or pages with a foreign canonical. Filter the list by problems, broken links, redirects, blocked URLs, and more. • Visual Highlighting: Paint links directly on the webpage! Colors indicate their HTTP status, dashed outlines instantly reveal URLs blocked by the robots.txt, and badges show rel attributes and problem targets while you scroll. 8️⃣ Technical SEO Change-Monitoring • Save Baselines: Save the exact technical SEO status (Status code, Canonicals, Meta/X-Robots) or the complete robots.txt file of any URL directly in your browser. • Smart Alerts: When revisiting a monitored URL, RoboX instantly warns you via the extension icon if any live directive deviates from your saved baseline. Robots.txt versions are compared line by line. • Backup & Sync: Save your baselines and robots.txt archives as a file and load them on another computer or share them with colleagues. Your settings sync through your browser account. 9️⃣ XML Sitemap Validator • Deep Validation: Audits file size, XML structure, HTTP status codes, logic errors, hreflang attributes, and natively unzips GZIP (.gz) sitemaps. 🔟 Bulk Technical SEO Checker • Scale Your Analysis: Evaluate HTTP status, robots.txt allowance, canonicals, Meta/X-Robots tags, and crucial HTTP headers for up to 200 URLs at once. • JS-Rendering Support: Toggle JavaScript-rendering to extract DOM-injected SEO signals using background ghost-tabs (processes up to 20 URLs safely in the background). • Easy Export: Export your complete bulk analysis directly to a CSV file for further processing. 🛡️ Honest Results – even behind bot protection & logins • Bot Protection Detection: Recognizes check pages of Cloudflare, Akamai, DataDome & co. instead of reporting them as your page – no false "robots.txt missing", broken links, or baseline alarms. • Logins & Staging: Sites behind Cloudflare Access or a server password (.htaccess) are checked with your login as soon as you have them open in a tab. • Correct Character Encoding: Pages in ISO-8859-1 and other encodings are read correctly, including URLs with umlauts. ⚙️ Your Tool, Your Rules: Highly customizable & workflow ready. • Popup or Sidebar: Use RoboX as a popup or as a sidebar that stays open next to the page and follows every tab change. • On-Demand vs. Live Monitoring: Set the extension to continuously monitor the status quo in the background or to only run checks when you actively click the extension icon. • SPA Support: Smartly detects Single Page Application (SPA) navigation and prompts a quick refresh to ensure your data is never outdated. • Dark Mode & Snapshot Overlay: Switch seamlessly to Dark Mode or detach the analysis into a floating window – perfect for screen sharing. • Built-in Handbook: The 📕 button opens a handbook with a legend of all icon states. 🔒 Privacy First: Your data stays yours. • Client-Side Only: RoboX operates 100% locally in your browser. • No Tracking: We do not track your browsing history, analyze your traffic, or send data to external servers. Made with ❤️ by Wingmen -- Bug Reports & Feedback: Found an issue? Please contact us via the form in the "Support" section. -- Legal: Imprint: https://wngmn.com/en/imprint/ Privacy Policy: https://wngmn.com/en/privacy/