Your incident response plan was written for incidents caused by people. Agentic AI changes that assumption entirely. When the actor is an AI system operating within its sanctioned parameters, the investigation, containment, and remediation playbook looks fundamentally different.
Incident response is one of the most mature disciplines in enterprise security. Organisations invest significantly in playbooks, tabletop exercises, retainer agreements with specialist firms, and technical capabilities that allow them to detect, contain, investigate, and recover from security incidents. Most of that investment was made with a specific assumption in mind: that incidents are caused by human actors, whether malicious insiders, external attackers, or users making mistakes.
Agentic AI breaks that assumption. When an AI agent takes an action that causes harm, the characteristics of the incident are materially different from anything that existing playbooks were designed to address. The actor was not a person. The action may have been technically authorised. The intent, in the conventional sense, did not exist. And the trail of evidence looks nothing like a traditional attack.
This piece works through what agentic AI incidents look like, why they are hard to detect and investigate, and what organisations need to build into their incident response capability before those incidents occur.
What an Agentic AI Incident Looks Like
An agentic AI incident does not begin with a phishing email or a stolen credential. It begins with an AI agent doing something it was not supposed to do, or something it was technically permitted to do but that produces an outcome the organisation did not intend.
The triggering event might be a prompt injection attack, where malicious instructions embedded in content the agent processed caused it to take an unauthorised action. It might be a misconfiguration, where the agent's access scope or action permissions were broader than intended. It might be an edge case in the agent's reasoning, where an unusual combination of inputs produced an output that was unexpected and harmful. Or it might be an entirely authorised action that produced an unintended consequence because the agent's understanding of the goal was subtly different from the human operator's.
In each case, the incident may have occurred without any human making a decision, without any externally visible attack traffic, without any credential compromise, and without any of the indicators that traditional security monitoring is designed to detect. The first sign of the incident may be an operational anomaly, an unexpected email sent to an external party, a record modified in an unexpected way, or a data exposure that surfaces through a routine review rather than an alert.
Why Investigation is Different
Traditional incident investigation starts with a question: who did this? Investigators trace activity back through logs, authentication records, and network traffic to identify the human actor responsible. The investigation is structured around establishing a chain of human decisions and actions.
An agentic AI investigation starts with a different question: what did the agent do, why did it do it, and was that within the scope of what it was authorised to do? Answering those questions requires a different kind of evidence and a different investigative methodology.
The evidence required includes the original instruction given to the agent, the full sequence of reasoning steps the agent took in response to that instruction, every tool call the agent made including inputs and outputs, every intermediate result the agent produced, and every action the agent took in external systems. Without this evidence, reconstructing what happened and why is extremely difficult.
This is why agent-level audit logging, discussed in the agentic AI governance piece earlier in this series, is not merely a compliance requirement. It is the foundational prerequisite for any meaningful investigation of an agentic AI incident. Without it, the investigation starts from a position of near-complete evidential blindness.
The investigation must also address whether the agent's actions were within its authorised scope. This requires clear documentation of what the agent was designed to do, what access it was granted, and what actions it was permitted to take. Where this documentation does not exist, establishing whether the agent acted within or outside its parameters becomes a matter of inference rather than comparison against a defined baseline.
Containment: Stopping an Agent Mid-Flight
Containing a traditional security incident involves isolating affected systems, revoking compromised credentials, and blocking attacker access. Containing an agentic AI incident requires stopping an autonomous system that may be mid-way through a complex, multi-step task.
This introduces a containment challenge that does not arise with conventional incidents. Simply stopping the agent may leave a partially completed task in an inconsistent state. If the agent has taken actions in multiple systems, some of which are reversible and some of which are not, stopping it mid-process may create additional problems. The containment decision requires an assessment of what the agent has done so far, what it would do next if allowed to continue, and which of those future actions would be more or less harmful than the current state.
Practical containment options for agentic AI systems include revoking the agent's credentials or API access to prevent further actions, disabling the agent at the platform level, applying network-level controls to prevent the agent from communicating with external systems, and in Microsoft 365 environments, using conditional access policies to block the service account or identity under which the agent operates.
Each of these options should be pre-documented as part of the incident response playbook for each agentic system, with the specific steps required to implement them and the person authorised to make the containment decision identified in advance. Discovering how to stop an agent during an active incident is not the right time to be working through the options.
Remediation: Reversing What the Agent Did
Remediation after a conventional security incident typically involves removing malware, restoring from backup, patching vulnerabilities, and resetting compromised credentials. Remediation after an agentic AI incident may involve attempting to reverse actions that the agent took, which introduces a set of challenges that have no direct parallel in conventional incident response.
Some agent actions are straightforwardly reversible. A record modified by the agent can be restored from a previous version. A document created by the agent can be deleted. A meeting booked by the agent can be cancelled. These reversals are straightforward if the audit log is sufficient to identify exactly what was changed and when.
Other agent actions are not reversible. An email sent externally cannot be unsent. Data submitted to an external API cannot be recalled. A communication made on behalf of the organisation to a client or counterparty creates a record that exists independently of the agent's logs. Where the agent has taken irreversible actions, remediation shifts from reversal to damage limitation: understanding what was disclosed or communicated, assessing the legal and regulatory implications, and managing the consequences.
This is why the human checkpoint model discussed in the agentic AI governance piece is so directly relevant to incident response. The most effective remediation for an irreversible agentic action is preventing it from happening in the first place. Human approval requirements for high-consequence actions are both a governance control and a remediation cost avoidance measure.
The Accountability Question
Following a conventional security incident, accountability is relatively straightforward to establish. A human made a decision or took an action that caused or enabled the incident. That human, and the organisation they work for, bears accountability for the consequences.
Following an agentic AI incident, accountability is more complex. The AI did not make a human decision. It executed a process within parameters that were set by humans. The accountability question distributes across the person who instructed the agent, the person who configured its access, the person who designed its behaviour, the team that deployed it, and the vendor whose model underpins it.
This distribution of accountability has direct implications for how incidents are reported internally, how they are communicated to regulators, and how liability is apportioned if the incident results in legal proceedings. Organisations that have not thought through the accountability model for their agentic AI systems before an incident occurs will find themselves working through these questions under pressure, with incomplete information and external scrutiny.
The accountability framework should be established as part of the governance design for each agentic system, not constructed retrospectively after an incident. It should identify who is accountable for the agent's design, who is accountable for its access configuration, who is accountable for monitoring its behaviour, and who has the authority to make decisions about containment and remediation when something goes wrong.
Building the Agentic AI Incident Response Playbook
Every agentic AI system in production should have an incident response playbook that is specific to that system. Generic AI incident response guidance is a starting point, not a substitute for system-specific preparation. The playbook should cover the following.
System description and authorised scope. A clear description of what the agent is designed to do, what access it has been granted, and what actions it is permitted to take. This is the baseline against which anomalous behaviour is assessed.
Detection indicators. The specific indicators that should trigger an investigation: anomalous action patterns, access attempts outside the authorised scope, unexpected external communications, and performance or behavioural anomalies that may indicate compromise or manipulation.
Containment procedures. The specific steps required to stop the agent, the person authorised to make the containment decision, and the assessment framework for deciding whether and how quickly to contain given the risk of leaving the agent running versus the risk of stopping it mid-task.
Evidence preservation. The logs, outputs, and system states that need to be preserved immediately upon identification of a potential incident, before containment actions that might overwrite or lose evidence.
Investigation methodology. The sequence of investigative steps for reconstructing what the agent did, why it did it, and whether its actions were within or outside its authorised scope.
Remediation options. A categorisation of the actions the agent can take by reversibility, with the specific remediation steps for each category and the person responsible for executing them.
Notification and reporting. The conditions under which the incident must be reported to regulators, clients, or other third parties, and the accountable person responsible for making those notifications.
Test the Playbook Before You Need It
An incident response playbook that has never been tested is a document, not a capability. Tabletop exercises for agentic AI incidents are unfamiliar territory for most security teams, but the discipline of working through a scenario before it happens is exactly as valuable for agentic AI as it is for ransomware or data breach scenarios.
A useful starting scenario for a tabletop exercise: an email processing agent has sent a summary of internal communications to an external address. The summary contains commercially sensitive information. It is not clear whether the agent was manipulated through a prompt injection attack or whether a misconfiguration caused it to include the external address in its output. The agent is still running. What do you do?
Working through that scenario with the relevant stakeholders, security team, legal, data protection, and the business unit responsible for the agent, will surface gaps in the playbook, ambiguities in the accountability framework, and technical limitations in the investigation and containment capability. Surfacing those gaps in a tabletop exercise is significantly less costly than discovering them in a live incident.
If you are deploying agentic AI systems and want to develop incident response playbooks that are fit for purpose, or if you want to run a tabletop exercise that tests your current preparedness, we can help.
