Question 7 of 24
How do we handle AI-generated content and hallucinations?
Defining responsibility when AI produces inaccurate outputs used in contracts, reports, or customer communications, and the controls that prevent harm.
Hallucinations are a design characteristic, not a bug
Large language models generate plausible-sounding text by predicting likely continuations of input sequences. They do not retrieve verified facts. They can produce outputs that are grammatically fluent, contextually appropriate, and entirely false. This is not a temporary limitation that will be engineered away. It is a fundamental characteristic of how current generative AI systems work.
Organizations that deploy generative AI without controls to detect and prevent hallucinations are accepting liability for outputs they cannot predict or verify. When those outputs appear in contracts, regulatory filings, customer communications, or legal documents, the exposure is significant.
Responsibility and liability
The legal question of who bears responsibility for AI-generated errors is still being resolved in courts and regulators' offices. The emerging consensus is that the deploying organization, not the AI vendor, bears primary responsibility for outputs used in its operations. Terms of service for most major AI platforms disclaim liability for output accuracy and prohibit reliance on AI outputs in high-stakes decisions without human review.
In professional services, attorneys who submitted AI-generated court filings containing fabricated case citations have faced sanctions. In financial services, AI-generated research that contains material errors may implicate securities regulations. The professional and regulatory standards that apply to human-generated work generally apply equally to AI-assisted work product.
Controls that reduce risk
For high-stakes use cases, require human review of all AI-generated content before it is used externally. Define what "review" means in practice: a reviewer who scans for obvious errors is not the same as one who verifies every factual claim against source documents.
Use retrieval-augmented generation (RAG) architectures that ground model outputs in specific, verified documents rather than relying on parametric memory. Implement output filtering that flags content making specific factual claims for additional review. Maintain logs of AI-generated content used in significant decisions or communications.
Train employees to treat AI outputs as drafts requiring verification, not finished work product. The cultural norm that AI output is "good enough" is one of the most significant sources of hallucination-related risk in enterprise settings.