SEO Scrubbox

Technical SEO & live visual crawling: AI checks, canonicals, rendering diffs, redirects, sitemaps, hreflang, JSON-LD, CrUX.

SEO Scrubbox is a lightweight yet powerful Chrome extension built for SEO professionals, developers, and technical marketers who need real-world insight into how a page behaves for search engines and AI crawlers. It blends instant on-page analysis with deeper HTTP-based audits and live visual crawling, showing both the raw view-source and fully rendered states. This means you can see exactly what Googlebot, Bing, and modern AI systems actually see when they hit your site. Designed to fit smoothly into modern SEO and AI search optimisation workflows, SEO Scrubbox helps you diagnose canonical inconsistencies, rendering differences, structured data quality, crawl signals, redirect behaviour, and sitemap visibility—all without leaving the page you are checking. Core Capabilities Interactive Visual Crawler (ScrubberCrawl) - Live Node Graph: Maps site architecture in real-time using a dynamic visual graph as the crawler discovers new pages. - Multiple Layouts: Switch instantly between standard Directory routing and Crawl Depth visualisations. - Dynamic Metric Mapping: Customise node size and colour to instantly spot issues, mapping metrics like status codes, indexability, unique inlinks/outlinks, word count, and file size. - Live Filtering & Search: Filter the visible graph by indexability or HTTP status, and use the search bar to highlight specific URLs or metrics instantly. - Crawl History & Snapshots: Automatically saves your crawl history, allowing you to load, review, and delete previous crawl snapshots without re-running them. - CSV Import & Export: Export full crawl data and metrics to CSV for external reporting, or import a previous CSV to visualise it in the graph. - Free Crawl Cap: Safely crawls up to 500 URLs per run straight from your browser. Fast Checks with Rendered Awareness - Live Indexability Icon: The extension icon automatically changes colour based on live indexability signals. - On-Page Nofollow Highlighter: Toggle a visual overlay that highlights rel="nofollow" links directly on the page with a dashed outline so you can spot them instantly. - On-Page Data-Nosnippet Highlighter: Toggle a visual overlay to instantly spot data-nosnippet elements blocking text from AI summaries and search snippets. - Redirect Tracking: Captures full redirect chains from real browser navigation events. - Canonical & Meta Diagnostics: Compares canonical tags, meta robots, and Googlebot directives before and after JavaScript rendering, flagging any changes introduced by JS. - Text & Heading Analysis: Identifies JS-caused changes to titles and meta descriptions (normalising text to ignore harmless entity differences) and checks your H1 structure, validating if the H1 is genuinely the first heading on the page. - Signal Detection: Highlights noindex, nofollow, none, and X-Robots-Tag HTTP header signals. - JS Toggle: Features a quick per-site JavaScript toggle for controlled render testing. Deep Technical Audit - Staging Mode Ready: Toggle on staging mode to pass your active session cookies and authorisation headers through to the crawler. This lets you easily audit pre-production sites and password-protected staging environments. - Granular Crawl Controls: Dial in your audit with interactive sliders for maximum concurrent threads, request delays (speed), internal URL limits, and sitemap check limits. - Smart Sitemaps: Recursively discovers sitemaps from robots.txt, identifies index vs. URL set structures, pulls lastmod dates, and automatically skips massive sitemaps to preserve performance. - Internal Link Auditing: Checks internal links for status codes, redirect chains, and nofollow flags, grouping issues logically by HTTP status code. - Link Frequency Counting: Automatically calculates the frequency of discovered internal links and provides a one-click copy function for spreadsheet-ready exports. - JS-Injected Link Isolation: Automatically isolates and flags links injected exclusively by client-side JavaScript. - Hreflang Validation: Validates hreflang status codes and checks for reciprocal return links. - Smart Caching & Cancellation: Uses session caching for lightning-fast re-runs and features an instant abort button to safely cancel deep audits mid-flight. Server, Security & Caching Diagnostics - Edge Security Checks: Audits critical security headers including Content-Security-Policy (CSP), Strict-Transport-Security (HSTS), and X-Content-Type-Options (nosniff). - Cache & Compression Analysis: Evaluates Cache-Control policies, ETags, Vary headers, and confirms modern compression formats like Brotli, Gzip, or Zstd. Rendered vs Non-Rendered Comparison - Visual Diffing: Uses sleek half-wheel visualisations to compare internal link counts and total word counts before and after JS rendering, helping you instantly spot JS-injected links or dynamically loaded text. - JS SEO Debugging: Easily spot hidden content, DOM rewriting issues, and rendering bottlenecks. Structured Data Inspection - Modern Schema UI: Detects all JSON-LD scripts on the page and organises them into clean accordions. - Instant Validation: Automatically parses the schema to flag it with a clear "Valid" or "Not valid" status pill. - Entity Identification: Extracts the @type or @graph to clearly label each script at a glance. - Raw Code Access: Neatly formats and indents the raw JSON-LD payload so you can easily review the exact data being fed to search engines and AI crawlers. - One-Click Exports: Copy raw JSON-LD schema or grouped link errors directly to your clipboard in spreadsheet-ready formats. Redirect Intelligence - Canonical Differentiation: Differentiates between actual HTTP redirects and client-side canonical rewriting. - Final Destination Mapping: Detects internal links that resolve via redirects during deep audits and displays their final destination status codes alongside the original link. Indexability Clarity - Explicit AI Crawler Access: Automatically parses robots.txt to verify Allowed/Blocked status for specific AI agents like GPTBot, ClaudeBot, PerplexityBot, and Google-Extended. - Snippet Control Auditing: Flags snippet shielding directives (like nosnippet or max-snippet:0) across meta tags and X-Robots-Tag headers. - Google-Style Precedence: Parses robots.txt using Google-style precedence logic. - Comprehensive Evaluation: Evaluates X-Robots-Tag headers alongside DOM meta tags for a complete indexability picture, maintaining consistency between the instant quick check and the full deep audit. AI Search Optimisation Ready SEO Scrubbox is built with modern AI-driven search in mind. It helps you optimise for: - Google Search Generative Experiences and AI Overviews. - Bing AI and emerging answer engines. - LLM-based crawlers and AI agents. - Entity clarity, structured data consistency, and canonical stability. - Clean, machine-readable crawl signals and rendering consistency. Core Web Vitals (CrUX Integration) - Hook up your own API key to pull in the Google Chrome UX Report. - Displays real-world field data for LCP, INP, and CLS, instantly classifying them as Good, Needs Improvement, or Poor. User-Agent Control - Emulate Googlebot Smartphone or Desktop. - Set custom user agents and dynamically modify request headers to test server responses. Why use SEO Scrubbox? - Privacy-First: Runs entirely locally in your browser. No external servers or data logging. - Frictionless: No page reload required for most checks. - Built for Real Work: Fast enough for daily spot-checks, precise enough for deep technical investigations. - Actionable: Focuses on signal clarity over noise, helping you understand exactly what search engines and AI crawlers see, how your page is interpreted, and why it behaves the way it does.