AI Safety Evaluation

AI safety evaluation refers to the structured testing of AI models before and after deployment to find harmful behaviors, such as generating dangerous instructions, leaking private data, or acting unpredictably in high-stakes settings. For enterprise governance teams, these evaluations provide the evidence needed to justify deployment decisions, document due diligence, and respond to regulators who increasingly ask how a company knows its AI systems are safe. Evaluation results also help set internal risk thresholds and determine whether a model needs added controls before it touches customers or sensitive operations.

1 item

Research2026-10-28

DualChem Benchmark Exposes a Gap Between AI Risk Recognition and Model Behavior

Scale AI, working with NIST's Center for Advancing Innovation and Standards for Super Intelligence (CAISSI), has published DualChem. It is a 600-question benchmark testing how frontier AI models handle ambiguous chemistry queries with dual-use potential. Testing across roughly 30 frontier models found that models frequently answer queries they simultaneously flag as potentially dangerous. The benchmark gives government agencies and enterprises a new tool for assessing AI risk in chemistry and related sensitive domains.

Scale AI NIST CAISSI DualChem dual-use risk AI benchmarking AI safety evaluation export controls