Prompt Injection Explained in Plain English

Prompt injection is one of the most significant security risks associated with AI systems built on large language models, and one of the least understood by the people responsible for governing them. The technical literature is dense. The practical implications are significant. The gap between the two creates real exposure for organisations deploying AI without adequate security review.

This article is not a technical deep dive. It is a plain-language explanation of what prompt injection is, why it matters, and what it means for organisations using AI systems in their operations.

What prompt injection actually is

A large language model works by receiving instructions – a prompt – and generating a response based on those instructions and its training. In a well-governed deployment, the instructions come from the organisation or the authorised user. The model does what it is told, within the boundaries set by whoever configured the system.

Prompt injection subverts this arrangement. An attacker introduces instructions into content the AI system processes – not through the authorised prompt channel, but somewhere the model will read as part of its context. If the model cannot reliably distinguish between the trusted instructions it has been given and the untrusted content it is processing, the attacker’s instructions can override, modify, or supplement the original ones. The OWASP Top 10 for LLM Applications ranks this as the leading security risk for applications built on large language models.

The result is that the AI system behaves in ways the organisation never intended, and in many cases specifically tried to prevent. The model follows the attacker’s instructions instead of, or alongside, the organisation’s own.

Direct and indirect prompt injection

There are two main variants of the attack, and both are relevant to organisational AI deployments.

Direct prompt injection involves an attacker submitting malicious instructions through the normal user input channel. If an AI customer service tool asks users to describe their issue, an attacker might submit a carefully crafted message designed to override the system’s instructions, causing it to reveal its configuration, produce prohibited content, impersonate a different service, or exfiltrate information it has access to.

Indirect prompt injection is the more insidious variant and the harder one to defend against. Here, the malicious instructions are not submitted by the attacker directly. They are embedded in content the AI system retrieves and processes as part of its normal operation – a document the system is asked to summarise, a web page it browses, an email it reads on the user’s behalf, a database record it queries. The attacker’s instructions sit in that content, waiting for an AI system to process it and act on them.

The indirect variant matters particularly for AI systems with tool use capabilities – those that can browse the web, read documents, query databases, send emails, or interact with other systems on behalf of users. Each of those capabilities represents a potential injection surface, and instructions embedded in the content those systems process can direct them to take actions far beyond what the authorised user requested.

Why prompt injection is harder to prevent than it sounds

The intuitive response to prompt injection is to build better filters – to detect malicious instructions before they reach the model and block them. This is a reasonable approach with some value, but it is not a complete solution.

Large language models are generative and context-sensitive. They do not process instructions as discrete, identifiable objects that can be cleanly separated from benign content. The same underlying capabilities that let a model understand nuanced natural language instructions also make it difficult to build reliable filters that catch all malicious instructions without generating unacceptable false positive rates on legitimate content.

Obfuscation makes the problem harder still. Malicious instructions can be expressed in ways that evade pattern-matching – through unusual phrasing, through instructions split across multiple pieces of content, through encoding or formatting that the model interprets as it processes the text. Each of these techniques reduces the reliability of filter-based defences.

None of this means defences are pointless. It means prompt injection needs addressing through a combination of technical controls, architectural decisions about what AI systems can access and do, and governance around where AI is deployed and with what level of oversight.

What prompt injection means for organisations using AI

For most organisations, the practical implications of prompt injection depend on what AI systems they are running and what those systems can do.

AI tools that are essentially read-only – where the model generates text that a human then reviews and acts upon – carry relatively limited prompt injection risk. Even if an attacker influences the model’s output, the human review step provides a meaningful check.

The risk increases significantly where AI systems have agency – where they can take actions, access systems, retrieve data, send communications, or interact with other tools on behalf of users. In those contexts, successful prompt injection can result in actions the organisation never authorised, data being accessed or exfiltrated through the AI system, and the AI being used as an attack vector against the systems it is connected to.

For organisations deploying or evaluating AI systems with tool use capabilities, security review needs to explicitly address the prompt injection surface: what content sources the system processes, what actions it can take, and what controls limit the impact if an injection attack succeeds.

The governance dimension of prompt injection

Prompt injection is frequently categorised as a technical security problem, and it is. But it also has a governance dimension that is easy to miss.

Organisations deploying AI systems often do so without a clear picture of what those systems can be made to do by a motivated attacker. The risk assessment conducted before deployment may not have considered injection as an attack surface – a gap covered in more depth in what an AI risk assessment should actually cover. The supplier may not have been asked what testing they have conducted against injection attacks. The incident response process may not include AI-specific scenarios.

Good governance of AI systems with significant capabilities includes understanding the injection risk surface as part of the pre-deployment assessment, ensuring supplier security practices include injection testing, and building the monitoring and incident response capability to detect and respond to injection-related incidents if they occur.

If your organisation is deploying AI systems with significant capabilities and wants to ensure the security risks are properly understood, Black Chili’s Continuous AI Assurance service provides the ongoing independent oversight you need.