Fabricated Stats Survived Seven Months in ChatGPT, Reaching Academic and Investment Research
What happened
A researcher at Peec AI published a blog post titled Publish stats, get cited: The tactic that put my data all over the web documenting a deliberate experiment in citation laundering. The researcher created a website containing 50,000 fabricated statistics and tracked how AI systems retrieved and cited that content over time. ChatGPT continued surfacing the false data for more than seven months after Google removed the site from its search index. The fabricated figures appeared in an academic paper and in investment research published by a prominent Qatari financial institution, both of which treated the invented numbers as factual. Repeated citation by credible sources is sufficient to make false data appear authoritative to AI retrieval systems. This holds regardless of the underlying source's validity or continued existence.
Why it matters
- ·Enterprises using AI tools for investment research, regulatory submissions, or competitive intelligence have no standard control to detect fabricated statistics that appear credible due to wide citation. This directly affects fiduciary and regulatory accuracy obligations in financial services, legal, and professional services sectors.
- ·The seven-month persistence gap between Google deindexing a source and an AI system stopping retrieval of its content means organizations cannot rely on web-source removal as a remediation step. Compliance programs that assume AI tools track source validity in real time must revisit that assumption.
- ·Once false data reaches a credible publication, such as an investment research report or an academic paper, downstream AI systems and human reviewers alike treat it as verified. This citation-laundering dynamic means that a single successful injection into a reputable outlet can propagate false information indefinitely through AI-assisted research workflows.
Governance controls affected
What to do now
- ☐Identify every internal workflow where employees or AI tools use AI-generated research summaries, competitive intelligence, or market statistics as inputs to investment decisions, regulatory filings, or client deliverables, and flag those workflows for additional source verification requirements.
- ☐Ask your research, legal, and compliance teams whether they have a step-by-step process to trace any statistic surfaced by an AI tool back to a primary source, such as a government database, peer-reviewed publication, or named organization's original report, before that figure is used in a consequential document.
- ☐Review vendor contracts for any AI research or data intelligence tools your organization subscribes to, and confirm whether the vendor discloses how long content remains in its retrieval system after the original source is removed or retracted.
- ☐Establish a pre-publication verification step for any AI-assisted document that will be shared externally or used in regulatory submissions, requiring that all statistics and factual claims be cross-checked against primary sources rather than AI-retrieved summaries.
- ☐Brief your board or audit committee on the citation-laundering risk: the finding that fake data published on a purpose-built site reached investment research at a major financial institution demonstrates that this is not a theoretical risk but an active one requiring program-level controls.
What to watch next
Regulators overseeing investment research quality, including securities regulators and financial stability bodies, have not yet issued explicit guidance on citation-laundering risk in AI-assisted analysis. However, the pattern documented here creates clear exposure under existing accuracy and suitability obligations. Compliance teams should monitor whether the EU AI Act (Regulation (EU) 2024/1689) general-purpose AI transparency provisions, as enforced by the European Commission under the AI Act Governance and Enforcement Framework, begin to address retrieval accuracy and source-persistence gaps as part of systemic risk assessments. The incident also reinforces the need for organizations to follow guidance from NIST AI 600-1 Generative AI Profile on output verification. This is particularly important for high-stakes research workflows where AI hallucinations and citation laundering have overlapping effects.
Stay ahead of stories like this
Get every Global AI governance development like this one, plus the rest of the week's developments. Every Thursday.
Recent issues
- Three separate incidents this week reveal that AI vendors cannot be trusted to self-police, and that harm from deployed agents can run unchecked for months before anyone inside the organization notices.15 Oct
- AI systems built to extend your reach are now extending attackers' reach too, and regulators in California and South Korea are making clear that containment failures belong to deployers, not just vendors.8 Oct
Free every Thursday. Unsubscribe anytime.