Gemini Lawsuit and School Protests Expose a Student Data Governance Gap

What happened

Why it matters

• ·School districts and other institutions deploying AI to minors face acute procurement liability if vendor contracts contain no binding terms on data handling, consent, or user safeguards. Even large, multi-year contracts with major AI vendors may be silent on these obligations. That silence creates regulatory and litigation exposure under state and federal child privacy frameworks.
• ·The lawsuit against Google signals that product liability claims tied to AI-related harm to minors are moving from theory to active litigation. Compliance teams at any organization deploying consumer-facing AI to users under 18 should treat vendor safety assurances as insufficient without independent verification and contractual accountability.
• ·The CPS situation illustrates a broader gap in public-sector AI governance. AI tools can be embedded in institutional infrastructure without a board-approved policy, without staff or parent awareness, and without any formal risk classification. This pattern mirrors the shadow AI inventory problem documented across enterprises and is now drawing enforcement and legislative attention.

Governance controls affected

What to do now

• ☐Pull all contracts with AI vendors serving student or minor users and confirm whether each contract includes explicit, enforceable terms on how student data is collected, stored, and shared with the vendor's subprocessors.
• ☐Ask your legal and procurement teams whether any AI product deployed to minors was reviewed and approved through a formal institutional policy process, or whether it was adopted at the department or technology level without board or leadership sign-off.
• ☐Verify whether parental or guardian consent processes exist for each AI tool accessible to minors, and document where those processes are absent or untested.
• ☐Request written confirmation from Google, Anthropic, and OpenAI (or any AI vendor with an education contract) specifying which student data is ingested, retained, or used for model training, and under what contractual restrictions.
• ☐If your organization has an AI acceptable-use policy for external or consumer-facing tools, confirm that it explicitly covers tools deployed to or used by minors, and that it has been reviewed by legal counsel familiar with child data privacy obligations.

What to watch next

Federal and state legislative attention to AI in schools is likely to intensify following the Emma Doe lawsuit and the multi-city protests. Several states have enacted or proposed restrictions on AI tools in educational settings. Bills addressing minor-facing AI chatbot liability are also advancing in Congress, as seen in bipartisan bills targeting AI agent liability and minor-facing chatbots. Compliance teams should monitor whether Chicago Public Schools adopts a formal board-approved AI policy, as the outcome could become a reference model for other districts. Any expansion of child privacy regulations to cover AI-generated interactions or behavioral profiling of minors would directly affect vendor contract requirements.