Prompt injection
Prompt injection denotes a class of attack on applications using large language models in which input is used to try to override or circumvent the system's original instructions. Because a language model processes instructions and data in the same stream of text, manipulative text can lead the model to carry out unwanted actions, disclose confidential information or ignore protective rules. Prompt injection counts as one of the most important security risks in developing AI solutions and can only be contained effectively through a bundle of coordinated safeguards.
Also known as: prompt injection attack, LLM injection
What is prompt injection?
Prompt injection arises from a fundamental property of language models: they do not reliably distinguish the operator's trustworthy instructions from arbitrary content reaching the prompt as data. If the model receives text phrased like a new instruction, it can follow it although it comes from an untrustworthy source.
The problem is structural and cannot be solved by a single setting. Unlike classic security holes, there is no sharp boundary between code and data to work from. That is exactly why prompt injection is so persistent and requires several layers of defence.
This entry describes the risk purely for information and defence. The aim is to understand the underlying dangers and take suitable protective measures, not to enable actual attacks.
Attack types and safeguards
Prompt injection comes in several forms. In direct injection a user themselves tries to override the system rules through their input. In indirect injection the manipulative text reaches the system through an external source, a web page or a document the application processes for instance.
The table below sets typical risks against the most effective safeguards. It is meant as guidance for securing AI applications.
| Attack type | risk | safeguard |
|---|---|---|
| Direct injection | user overrides the system instruction | Mark input as data, limit what the model is allowed to do |
| Indirect injection | Malicious text from external sources acts | Filter and isolate external content, and never treat it as a command |
| Jailbreak | Safety rules are bypassed | Multi-stage review, Guardrails and output filters |
| Data exfiltration | Confidential data is siphoned off | Minimise access, keep sensitive content out of the context |
Why prompt injection is so dangerous
The danger of prompt injection grows with an application's capabilities. As long as a model only returns text, the damage is limited. As soon as it can use tools, send emails, query databases or run code, though, a successful injection can trigger real actions going far beyond a mere wrong answer.
The indirect variant is particularly critical. When an application processes external content, summarising web pages or documents for instance, hidden text there can contain instructions the model follows unintentionally. The application's user notices nothing at first, although unwanted actions run in the background.
On top of that, prompt injection cannot be ruled out entirely. It is an open research topic where, realistically, the aim is reducing risk, not preventing it completely. Security concepts therefore have to assume an injection can succeed in an individual case and limit the possible damage.
Effective safeguards
Effective protection rests on the principle of several lines of defence. No single measure suffices, which is why technical, organisational and design precautions are combined. At the centre stand the strict separation of trustworthy instructions from untrustworthy input and the consistent limiting of what the model may do at all.
One central principle is least privilege. A model should be able to reach only the tools and data strictly necessary for its task. Actions of high consequence, such as sending messages or changing data, should additionally be safeguarded by human confirmation.
The following measures have proven themselves in practice for reducing the risk of prompt injection considerably.
- Mark user input and external content clearly as data and keep it apart from instructions.
- Give the model only the tools it needs, following the principle of least privilege.
- Check and clean input and output with guardrails and filters.
- Safeguard high-impact actions with human approval.
- Never treat external sources as commands, and limit their influence.
- Check security continuously through tests and red teaming.
Prompt injection in enterprise use
For companies, prompt injection is a core topic of AI security and governance. Anyone who LLMapplications productively should account for the risk in the design rather than adding safeguards afterwards. A security review belongs in every AI project from the start.
In practice that means defining clear responsibilities, documenting data flows and setting which actions a system may carry out on its own. The more autonomy an application gets, the stricter the controls have to be. With agents that have tool access, particular care is needed.
Ultimately, prompt injection shows that security in AI solutions must not be an add-on but part of the architecture. Through the interplay of thoughtful design, technical protective layers and organisational rules, the risk can be reduced to an acceptable level, even though it can never be ruled out entirely.
Frequently asked questions
What is prompt injection?
Prompt injection is an attack on LLM applications in which manipulated input overrides or circumvents the system's original instructions. Because a language model processes instructions and data in the same stream of text, harmful text can lure the model into unwanted behaviour. It counts as one of the most important security risks in AI applications.
What sets direct apart from indirect prompt injection?
In a direct injection a user tries to override the system rules through their own input. In an indirect injection the manipulative text reaches the system through an external source, a web page or a document the application processes for instance. The indirect variant is often more dangerous, because it stays invisible to the user.
Can prompt injection be prevented entirely?
No. Prompt injection is a structural problem of language models and an open research topic. The realistic aim is reducing risk, not preventing it entirely. Security concepts should therefore assume an injection can succeed in an individual case and contain the possible damage consistently through limited permissions.
How do you protect an AI application against prompt injection?
Through several lines of defence: input and external content are clearly marked as data and separated from the instruction part, the model gets only the permissions it needs, input and output are checked with guardrails, and powerful actions are safeguarded by human approval. Regular testing and red teaming round out the protection.
Why is prompt injection especially risky with AI agents?
Agents can use tools, query data or carry out actions. If an injection succeeds here, it does not stop at a wrong answer: real actions can be triggered, such as sending messages or changing data. The more autonomy a system has, the stricter the controls have to be.
What is the difference between prompt injection and a jailbreak?
A jailbreak aims to bypass a model's safety rules so it produces content it should refuse. Prompt injection is the broader term for overriding or redirecting the system instructions through input. A jailbreak is therefore a particular form of prompt injection.
Related terms
Protecting AI systems and their data against risks such as prompt injection and data leaks.
Technical and organisational guardrails that secure and control how AI systems behave.
A framework of policies, roles and controls for responsible and compliant AI.
Prompt engineering is the craft of phrasing AI instructions so that language models return better results.
An LLM is an AI language model that understands and produces text by predicting the most likely next word.
An AI hallucination is output from an AI model that sounds plausible but is factually wrong.
Put AI to work for your business?
We help you integrate artificial intelligence into your processes, your marketing and your website — strategically and securely.
Your contact
Stefan
I look forward to hearing about your project and finding the best solution together.
