AI Source Scraper

Capture cited web links and source metadata from Claude, Gemini, and ChatGPT for longitudinal digital methods research.

# AI Source Scraper *A research tool for capturing and comparing sources surfaced by AI search interfaces.* Designed for digital methods and AI search research, the extension turns source and activity panels from supported AI platforms into structured datasets while preserving information about the conditions under which each response was generated. Rather than collecting links alone, AI Source Scraper lets researchers annotate each capture with methodological metadata: - **Capture mode:** Longitudinal or Comparative - **Search condition:** AI search or Agentic AI search - **Prompt condition:** Isolated, Initial, or Follow-up - **Prompt framing:** Underspecified, Program, Anti-program, or Ambiguous - **AI fieldwork observations:** researcher notes recorded during data collection Captures accumulate within a research session and can be exported as **CSV, JSON, and TXT**, allowing researchers to merge, filter, compare, and analyse source-selection patterns across prompts, platforms, search conditions, and moments in time. This research tool operationalises digital methods such as the [**Longitudinal Prompt Method for Understanding AI Search**](https://www.researchgate.net/publication/408580170_Longitudinal_Prompt_Method_for_Understanding_AI_Search), supporting projects that seek to advance source studies of single or cross-AI models by turning the citation layer of AI chatbots into structured data. ## Two research designs *Longitudinal* captures are designed to investigate how AI responses, sources, or interfaces develop and change over time or across conversational sequences. *Comparative* captures support research designs in which conditions are deliberately varied, for example by comparing AI search with Agentic AI search or testing different prompt framings. The same metadata structure is available in both modes, making it possible to build comparable datasets while keeping the methodological conditions of each capture explicit. ## Supported research contexts AI Source Scraper is intended for research involving AI-mediated web search, source selection, citation practices, conversational search, and the changing relationship between prompts, search conditions, and retrieved web sources. **ChatGPT and Gemini can be studied in logged-in or logged-out conditions** where their interfaces permit anonymous use. [**Claude.ai**](https://claude.ai/) requires an account to conduct conversations. ## Data and privacy The extension processes captured research data locally in the browser. Researchers decide when to export or clear their session data. ## Citation Omena, J. J. (2026). *AI Source Scraper* [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.20945556 **Source code:** https://github.com/jannajoceli/ai-source-scraper