Benchmark Scores Are Not Enough: Brookings Finds Agentic AI Evaluation Must Extend to System Behavior and Real-World Workflows
What happened
The Brookings Institution published How can we best evaluate agentic AI? on October 14, 2025, presenting original research on the gap between current AI evaluation practice and the demands of agentic system deployment. The paper argues that standard model benchmarks, which measure isolated capability on fixed tasks, cannot capture the behavior of agents operating in dynamic, multi-step workflows or interacting with other agents. Brookings identifies the emergence of unpredictable properties in multi-agent systems as a core measurement challenge that current testing designs are not equipped to address. The research calls for evaluation frameworks that prioritize predictive validity, meaning that test results must reliably forecast real-world system behavior, not just in-lab performance. The authors further argue that standardized evaluation methods are necessary to support regulatory decisions, implying that fragmented or proprietary evaluation approaches will not satisfy future compliance requirements.
Why it matters
- ·Regulatory exposure: Regulators in the EU, US, and Singapore are moving toward requiring documented pre-deployment evaluation of high-risk AI systems, and the Brookings findings signal that benchmark-only evidence is likely to be viewed as insufficient for agentic deployments, creating conformity assessment risk for organizations that rely solely on vendor-provided scores.
- ·Operational impact: Compliance teams that have built evaluation gates around model-level benchmarks must now assess whether those gates actually capture system-level behavior in production workflows, particularly for multi-agent pipelines where emergent interactions can produce outcomes no single model test would predict.
- ·Organizational risk: Organizations deploying agentic AI without socio-technical evaluation are accumulating governance debt, because the absence of validated, real-world behavioral evidence will become a material gap if an incident occurs and regulators or litigants demand proof that adequate testing was conducted before deployment.
Governance controls affected
What to do now
- ☐Audit your current pre-deployment evaluation process for agentic systems and document whether it tests system behavior in realistic multi-step workflows or relies solely on model-level benchmark scores.
- ☐Identify any multi-agent pipelines in production where emergent interaction behaviors have not been tested and schedule a structured behavioral assessment that includes adversarial scenarios and cross-agent delegation paths.
- ☐Review your deployment readiness criteria under AGT-016 to determine whether evaluation requirements reference predictive validity or real-world generalization, and update those criteria if they do not.
- ☐Engage your AI vendors to request evaluation documentation that addresses system-level and socio-technical impacts, not just model accuracy or benchmark rankings, and flag gaps in vendor responses for escalation.
- ☐Assign ownership within the governance function for tracking emerging evaluation standards from bodies such as NIST, ISO, and the EU AI Office, since standardized agentic evaluation methods are under active development and will likely become compliance requirements.
What to watch next
The Brookings paper signals a broader shift in regulatory expectations toward system-level and socio-technical evaluation evidence, a direction that aligns with ongoing NIST AI RMF implementation guidance, EU AI Act conformity assessment technical standards under development, and Singapore's IMDA agentic AI governance framework published in 2026. Compliance teams should monitor NIST's forthcoming work on agentic AI measurement and any EU AI Office technical specifications that reference evaluation validity requirements for autonomous systems. Enforcement actions against organizations that deployed agentic systems without adequate behavioral testing are a credible near-term risk as high-risk AI use case registrations accumulate under the EU AI Act.