AI Security and Identity
Access control, data protection, and identity management for AI systems and agents. As AI models gain access to sensitive data and enterprise systems, security controls designed for human users are no longer sufficient. This topic covers the security frameworks, data governance obligations, and identity controls that apply specifically to AI.
Key board-level questions
- 1.What data can AI systems access, and how is data lineage and consent tracked?
- 2.Do we enforce fine-grained access control for AI models and agents?
- 3.How do we mitigate risks of data leakage or unintended exposure?
- 4.Are AI systems integrated into our identity and access management strategy?
Regulatory frameworks
OWASP Top 10 for Large Language Model Applications
The OWASP Top 10 for Large Language Model Applications identifies the ten most critical security risks in LLM-powered systems, including prompt injection, insecure output handling, training data poisoning, model denial of service, and supply chain vulnerabilities. It is the most widely referenced security framework for AI applications and is used by development and security teams globally to prioritize controls.
EU Cyber Resilience Act
The EU Cyber Resilience Act establishes mandatory cybersecurity requirements for products with digital elements placed on the EU market, including hardware and software incorporating AI components, covering the entire product lifecycle from design through end-of-life.
UK ICO Guidance on Artificial Intelligence and Data Protection
The UK ICO's guidance on AI and data protection establishes how the UK GDPR and Data Protection Act 2018 apply to the design, development, and deployment of AI systems that process personal data.
EU Data Act
The EU Data Act establishes harmonised rules on access to and use of data generated by connected products and related services across the EU, addressing both personal and non-personal data. It creates new obligations for data holders to share data with users and third parties, and sets conditions for public sector bodies to access privately held data in exceptional circumstances.
EU Digital Operational Resilience Act
The EU Digital Operational Resilience Act (DORA), Regulation (EU) 2022/2554, establishes a comprehensive ICT risk management, incident reporting, operational resilience testing, and third-party risk oversight framework for EU financial entities, with direct implications for AI systems deployed in financial services and the technology providers that supply them.