What is our explainability standard for AI decisions?

The appropriate level of explainability depends on the stakes of the decision, the regulatory requirements that apply, and the audience who needs the explanation. A data scientist debugging model behavior needs a different kind of explanation than a customer asking why their application was denied. Both needs are legitimate and require different technical and procedural approaches.

Build your explainability standard around a risk-tiered framework. For minimal-risk systems, basic documentation of the model's purpose and general behavior may be sufficient. For limited-risk systems, transparency about the fact that AI is being used and its general logic is typically required. For high-risk systems, full technical documentation, audit logs, and the ability to provide individual-level explanations are necessary.

When a model's reasoning has to be explained after the fact, SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) are the most widely used techniques. Both show which inputs, such as income or payment history, most influenced a specific decision. They work with most types of model but have limits. For very complex models, the explanation is an approximation and may not accurately reflect how the model actually reached its answer.

For the highest-risk applications, consider whether a simpler model whose reasoning can be read directly would do the job. Scorecard-style statistical models, decision trees (a series of yes/no rules), and rule-based systems can all be explained by showing how they work. For data held in tables, the accuracy gap between these and more complex models has narrowed. The legal and operational benefits of a model you can explain can outweigh a small loss in accuracy.

GDPR Article 22 and the Fair Credit Reporting Act (FCRA) both require that individuals who are subject to automated decisions receive meaningful explanations. "Meaningful" has not been precisely defined by courts or regulators, but the emerging standard is that the explanation must be specific enough for the individual to understand and potentially contest the decision.

Build explanation delivery into your decision workflow, not as an afterthought. For credit decisions, this means FCRA-compliant adverse action notices that identify the principal reasons for the decision in plain language. For employment decisions, it means documentation that can be provided if a candidate requests an explanation. For customer decisions, it means a customer service process that can retrieve and communicate the relevant information.