AI regulatory compliance is moving from general guidance to specific scrutiny. The ICO has published guidance on AI and data protection. The FCA has addressed AI risk in regulated firms. Sector regulators across financial services, healthcare, and professional services are increasingly incorporating AI governance into their supervision frameworks.
When an AI-related incident triggers regulatory contact – whether through a mandatory breach notification, a supervisory review, or a complaint that surfaces AI involvement – the questions that arrive are predictable. They are not primarily technical questions about what the AI system did. They are governance questions about what the organisation knew, what controls were in place, and whether the organisation was exercising adequate oversight of its AI usage.
Understanding what those questions look like, and what good answers require, is relevant preparation for any organisation using AI in contexts where regulatory obligations apply.
Did you know what AI systems were in use? The AI regulatory compliance starting point
The foundation question in almost every AI-related regulatory enquiry is whether the organisation had an adequate picture of its AI usage before the incident occurred. Not a theoretical awareness that AI tools existed, but an operational inventory: which systems were in use, what data they processed, who was accountable for them, and what controls governed their use.
This question is deceptively straightforward. Most organisations have partial visibility – they know about the formally approved tools and have limited awareness of everything else. Under regulatory scrutiny, that partial visibility becomes evidence of inadequate governance rather than a reasonable operational position, the same gap covered in what is shadow AI risk.
The ICO’s approach to data protection impact assessments is instructive here. Where AI systems process personal data in ways likely to result in high risk, a DPIA is required before deployment. An organisation that cannot demonstrate it identified its AI systems, assessed their data processing implications, and conducted required DPIAs is in a weak position regardless of what happened in the specific incident.
What due diligence did you conduct on AI suppliers?
Where an AI incident involves data processed by a third-party AI tool, the regulatory question is whether the organisation conducted adequate due diligence before using that tool for that purpose.
Under UK GDPR, using a third-party data processor requires a data processing agreement that meets specific requirements. For AI tools handling personal data, the DPA needs to address what the processor does with the data, under what terms, for what purposes, and with what sub-processors. The fact that a supplier provides an AI tool does not exempt the organisation from its obligations as controller.
The practical question regulators will ask is whether the organisation reviewed the supplier’s data handling terms before using the tool with personal data, and whether those terms were adequate. An organisation that adopted a tool without reviewing its terms, or that relied on a standard privacy policy rather than a data processing agreement, will find that position difficult to defend.
This extends to the ongoing dimension. Supplier terms change. An organisation that conducted initial due diligence but has no process for monitoring subsequent changes – the gap covered in the hidden supplier risks in enterprise AI – is not maintaining the standard of ongoing oversight regulators expect.
What did your AI acceptable use policy say?
Where staff behaviour contributed to the incident – a data submission that should not have happened, a use case outside intended boundaries, an output relied upon without adequate review – the regulatory question is what the organisation’s policies said about that behaviour and how those policies were communicated and enforced.
A policy that did not address the relevant scenario, or that was not communicated to staff in a way that would reasonably influence their behaviour, provides limited protection. The question is not whether a policy existed, but whether it was fit for purpose and operationally embedded – the distinction covered in what should an AI acceptable use policy cover.
This is where the gap between generic downloaded templates and organisation-specific governance becomes a regulatory issue rather than just a governance quality issue. A policy clearly written for a different organisation’s context, that does not reflect the tools actually in use or the data categories actually relevant, is evidence of form over substance.
Who was accountable for AI governance decisions?
Accountability is a central theme in regulatory scrutiny of AI governance. Regulators want to understand not just what controls existed, but who owned them, who made governance decisions, and whether accountability was clear and exercised.
Diffuse accountability – where multiple functions have a role in AI governance but none has clear ownership – tends to produce governance that looks complete in documentation and fails under operational pressure. When a regulator asks who was responsible for ensuring adequate AI governance, “it was a shared responsibility across IT, legal, compliance, and the business” is not a satisfactory answer, as covered in AI governance for boards.
The organisations in the strongest regulatory position are those that can point to named accountabilities, evidence those accountabilities were exercised, and a clear governance structure connecting operational AI usage to board-level oversight.
What have you done since the incident?
Regulatory enquiries do not end with what happened before the incident. They also assess what the organisation has done in response – whether the incident has been treated as a governance learning opportunity or managed primarily as a PR and legal problem to be contained.
Organisations that can demonstrate a genuine governance improvement programme in response to an incident – updated policies, improved supplier governance, clearer accountability, enhanced monitoring – are in a materially better position than those that have focused mainly on narrative management.
The governance questions regulators ask after an AI incident are the same questions good governance requires organisations to ask themselves before one occurs. The difference is the context in which those questions are being answered.
If you want to ensure your AI governance is ready for regulatory scrutiny before an incident forces the question, Black Chili’s AI Incident Review service provides the independent assessment you need.
If you are not sure what AI tools are in use inside your organisation, an AI Exposure Review gives you a clear, independent picture - what is being used, what data it touches, and where the real risks are.
