Prompt Injection vs Jailbreak in Agentic Systems

How Prompt Injection Works and Why the Architecture Makes It Hard to Close

The root cause is structural. Once you see it, you cannot unsee it. LLMs have no native mechanism to distinguish trusted system instructions from untrusted user content. Everything arrives in the same context window, processed identically. That is the original sin the architecture has yet to resolve, and I am uncertain the current generation of models is designed to resolve it.

The SQL injection analogy is worth taking seriously. SQL injection exploits the mixing of code and data in a database query; prompt injection exploits the mixing of instructions and content in a language model context. The difference is scale: a context window can hold thousands of tokens, and injected instructions can be woven through any of them, buried in ways that resist obvious detection.

Two vectors dominate. Direct injection is what most people picture: a user types "ignore previous instructions" and tries to override the system prompt. Indirect injection is harder to see and more dangerous in practice. The attacker embeds instructions not in user input but in external content the agent reads during normal operation, documents, emails, web pages, tool outputs. No malicious user input is required. The agent fetches a document that tells it what to do, and it complies.

The steganographic variants are where a lot of security teams underestimate the problem, badly. Unicode tag characters in the range U+E0000 through U+E007F render as invisible in browsers and terminals but are processed by tokenizers. Zero-width joiners and bidirectional text overrides can hide or segment instructions inside readable text. Homoglyphs, Cyrillic characters visually resembling Latin letters, evade keyword filters while remaining legible to a human reviewer. An input that passes human review can still carry a live injection. Think about what this implies for automated scanning pipelines: if the characters are invisible to humans and meaningful to tokenizers, the review layer most teams rely on is operating on a different representation of the text than the model is. That gap is not theoretical.

Retrieval-augmented generation introduces a particularly tractable attack surface. A small number of carefully crafted poisoned documents can manipulate AI responses the large majority of the time via RAG. The injection does not need to reach the user's input field; it just needs to reach the retrieval corpus.

At the far end of the severity spectrum, researchers Lee and Tiwari demonstrated in 2024 that a malicious prompt can propagate across interconnected agents, behaving structurally like a computer worm. The injection becomes systemic rather than isolated. OWASP has ranked prompt injection at the top of its LLM risk list for two consecutive editions. That it still holds that position tells you something about how hard this actually is to close.

How Jailbreaks Work and How the Technique Families Have Evolved

The goal of a jailbreak differs from injection in one precise way: it does not redirect the model's task; it overrides what the model is willing to output. The target is the alignment training, rather than the application logic. That distinction sounds subtle until you are trying to figure out which defense to reach for, and you grab the wrong one.

The oldest recognizable family is persona and role-play attacks. DAN, which stands for "Do Anything Now," instructs the model to instantiate a second persona not bound by the original rules. The variant often includes a fake token economy, rewarding compliance to make the override feel procedurally legitimate. The DAN family has iterated through more than a dozen versions alongside STAN, DUDE, and AIM. Each iteration responds to the patches that closed the previous one. It is a cat-and-mouse dynamic, and the cats have had a productive few years.

Crescendo, formalized by Microsoft Research in 2024, is the multi-turn evolution of this approach. It starts with benign questions adjacent to the target topic. Each individual turn is defensible; the harmful output only emerges after the model has been primed across a sequence of small steps, none of which would trip a single-turn content filter. A 2025 follow-up study found multi-turn jailbreaks exceed 70% success rates against models optimized only for single-turn protection. If most safety evaluations are conducted turn by turn, what exactly are they measuring? They are certainly not measuring the environment in which production agents actually operate.

Many-shot jailbreaking, documented by Anil et al. at NeurIPS 2024, showed that effectiveness follows a power law in the number of in-context harmful examples. This one I kept coming back to, because the implication is uncomfortable: longer context windows, the very capability that makes agents more useful, are a jailbreak surface. The capability improvements and the vulnerability surface are growing from the same source. There is no clean separation there.

At the automated end, JBFuzz in 2025 achieved roughly 99% average attack success rates across GPT-4o, Gemini 2.0, and DeepSeek-V3. These are not edge cases. The empirical range across technique families runs from roughly 65% for simple multi-turn approaches up to roughly 99% for automated fuzzing. The floor is not low.

One contrast worth holding onto: a jailbreak can happen entirely through a model's chat interface with no external data involved. Prompt injection requires the model to process content from outside the trusted instruction set. Same output category, different causal chain, different defense. Conflating them produces security theater, and I see that conflation constantly.

What Changes When the Model Is an Agent Acting on the World

A model answering questions in a chat interface has a bounded blast radius. It can produce bad text. An agent is a different proposition entirely. Agents execute code, send emails, query databases, call APIs, often with minimal human oversight between steps. The exposure profile is categorically different, and this still has not fully landed for a lot of organizations deploying these systems.

Research on agentic AI systems has found that agents move substantially more data than human users, by an order of magnitude or more in some analyses. A compromised agent is a high-magnitude data exposure event, rather than a single-user incident. The permissions the system granted the agent become the attack's reach. This is the "confused deputy" problem: the agent holds broad credentials, and injected instructions execute under those credentials. The attacker does not need to steal access; they redirect it.

Multi-agent architectures extend this further in ways that are not always intuitive. A successful injection in one agent can propagate downstream through a chain. The STAC framework, developed by Li et al. in 2025, identifies a distinct attack category where individually innocent tool calls are chained into dangerous sequences. The attack is not visible in any single step; it only becomes apparent when you look at the full sequence. Privilege escalation, memory persistence, and lateral movement across agents all become tractable once the initial guardrail falls.

Model Context Protocol servers represent a concrete and currently undercharted surface. BlueRock Security's 2026 analysis of more than 7,000 MCP servers found 36.7% vulnerable to server-side request forgery, with proof-of-concept exploits against Microsoft's MarkItDown MCP server successfully retrieving AWS IAM keys from EC2 metadata endpoints. MCP is meant to extend what agents can reach; that is exactly what makes ungoverned MCP servers a liability rather than just a feature.

Roughly 90% of agents carry excessive privileges. The vulnerability class and the permission problem are not independent issues running in parallel; they amplify each other in ways that make an already serious situation considerably worse.

Why Agents Are Measurably More Vulnerable to Jailbreaks Than the Underlying Model Alone

Here is the finding that surprised me when I first worked through this research: the model's safety training degrades inside an agent wrapper. The agent architecture is itself a jailbreak risk, independent of the inputs it receives. Most practitioners who have invested in base-model safety evaluation do not expect that, and the gap between expectation and reality is substantial.

Kumar et al. (2025) and Chiang et al. (2025) both demonstrate this empirically. The BrowserART benchmark makes it concrete: a GPT-4o browser agent's attack success rate rises from 12% in a standard chat setting to 74% under a direct ask, and to 100% under an ensemble of attacks. Same model, dramatically different exposure. That is not a minor variance; it is a different threat profile entirely.

Chiang et al. identify three design factors responsible. Embedding the goal directly in the system prompt creates a persistent, overridable instruction the model must reconcile with everything it subsequently encounters. Iterative action generation across steps accumulates context in ways single-turn safety evaluation never tests. Processing environment feedback through an event stream introduces a continuous, externally influenced input channel. None of these are bugs exactly; they are architectural features of what makes agents useful. The vulnerability is baked into the design, which is an uncomfortable thing to sit with.

Coding agents are a specific high-risk case worth calling out. Saha et al. (2025) found that wrapping a model in a code agent raises attack success rate by 1.6 times on average, reaching roughly 75% in multi-file codebases, with up to a third of outputs being directly runnable malicious code. The agent's tool access, the thing that makes it useful, is the vector.

Safety evaluation on the base model does not predict safety in deployment. Organizations shipping agents while relying on model-level alignment as the primary defense are measuring the wrong thing in the wrong environment.

What Real Attacks Against Deployed Agents Look Like

EchoLeak, assigned CVE-2025-32711 with a CVSS score of 9.3, is as close to a canonical agentic attack as we have seen in production. In June 2025, a crafted email caused Microsoft 365 Copilot to access internal files and transmit their contents to an attacker-controlled server. No user interaction required. A single injection cascaded through the agent's retrieval capabilities to exfiltrate chat logs, OneDrive files, SharePoint content, and Teams messages. The attack chained multiple bypasses in sequence: it evaded Microsoft's cross-prompt injection attack classifier, circumvented link redaction via reference-style Markdown, and exploited auto-fetched images. Indirect injection, privilege escalation, and data exfiltration, contained within a single malicious document. Read it carefully, because it will not be the last attack structured that way.

GitHub Copilot's CVE-2025-53773, scored at 9.6, followed a different path to a similar outcome. A prompt injection tricked the coding agent into editing its own configuration file, enabling auto-approval of all subsequent commands. The assistant became a remote access trojan. The tool access was the weapon.

Cursor IDE's CVE-2025-54135 demonstrates the MCP surface directly: an indirect injection wrote a malicious configuration file, triggering remote code execution with no user interaction.

HOUYI, documented by Liu et al. in 2024, provides the breadth dimension. A black-box prompt injection technique compromised 31 of 36 real-world LLM-integrated applications, including Notion, at an 86.1% success rate. That is not a narrow finding targeting one system. That is a systematic characterization of how the production landscape responds to this attack class.

Anthropic's November 2025 report on an AI-orchestrated espionage campaign adds the adversarial dimension that makes the timeline more urgent. Attackers used agentic AI capabilities against approximately 30 targets across technology, financial services, and government sectors, with AI performing 80 to 90% of campaign tasks autonomously. The adversary is already using the same agentic capabilities enterprises are deploying. That symmetry should change how organizations think about their defensive timeline, because the attacker is not waiting.

Which Defenses Apply to Which Attack, and Where the Coverage Gaps Are

The taxonomy split maps directly to the defense strategy, and keeping these categories distinct is the practical payoff of all the definitional work above. Prompt injection is an architecture problem; the fix lives in how external content enters and is labeled in context, through input validation, output encoding, and trust-boundary enforcement between instruction and data channels. Jailbreaking is a model alignment problem; the fix lives in training, fine-tuning, and inference-time content classifiers. But per the BrowserART data, model-level alignment is insufficient once the model is wrapped in an agent. Both layers need defending. They are not substitutes for each other, and a lot of vendor pitches obscure that.

OWASP's Top 10 for Agentic Applications, released in December 2025 after development by more than 100 experts, ranks Agent Goal Hijacking as the primary agentic risk. Prompt injection remains the dominant production failure mode.

Several controls address both threat classes simultaneously. Least-privilege access is foundational: agents should hold only the permissions their current task requires. The figure of roughly 90% of agents carrying excessive privileges is a deployment configuration problem, rather than a model problem, and it is tractable today without waiting for better models. Real-time observability matters more in agentic contexts than in API logging because the detection window after initial compromise is narrow when an agent can act across dozens of steps in seconds; a post-incident audit log is a postmortem, rather than a safety net. A centralized tool and MCP registry ensures every server an agent can reach represents a deliberate, documented decision rather than an unmonitored assumption. Human-in-the-loop checkpoints on high-stakes actions are not just good practice: the GitHub Copilot attack specifically succeeded because auto-approval was enabled. That is a configuration choice, rather than an inevitability.

OpenAI acknowledged in December 2025 that prompt injection is unlikely to be fully solved at the model level. Defenses must live in the system surrounding the model. The boundary between trusted and untrusted content needs to be enforced structurally, because hoping the model figures it out is not a security posture.

The steganographic and multi-turn attack families reveal why input filtering alone is insufficient. An injection can be invisible to a human reviewing content; the harmful output only emerges after a sequence of individually benign steps. No single filter covers that surface.

On the compliance side: prompt injection now touches at least seven frameworks, including OWASP, MITRE ATLAS, NIST, the EU AI Act, ISO 42001, GDPR, and NIS2. With the EU AI Act's August 2026 deadline approaching, the governance layer is not optional for organizations operating in that jurisdiction.

Why the Threat Landscape Is Growing Faster Than Most Organizations' Defenses

Dark web discussions of jailbreaking methods grew 52% year over year, from 2,747 mentions in 2023 to 4,167 in 2024, per KELA's 2025 AI Threat Report. Mentions of malicious AI tools surged 219% over the same period. Between 2019 and 2025, dark web forum posts mentioning AI keywords grew 371% according to Group-IB's Weaponized AI report. That growth predates the current wave of agentic deployment. The interest was already accelerating before the attack surface expanded, which means the attacker tooling was industrializing before most enterprises had finished their first agentic pilot.

The trajectory of automated attack capability is where the numbers get uncomfortable. JBFuzz achieved roughly 99% success rates in 2025 against frontier models. BrowserART shows agents at 100% under ensemble attacks. These are evaluations against production-grade systems. The research frontier and the attacker economy are converging on a capability level that most enterprise security programs have yet to calibrate for, and the gap is not closing on its own.

The organizational gap that matters most right now is observability, rather than detection logic or model patching. Most teams have no real-time record of what their agents accessed during a session. Anthropic's espionage report finding that AI performed 80 to 90% of campaign tasks autonomously means the window between initial compromise and meaningful damage is measured in seconds to minutes. You cannot respond to an attack you cannot see. Instrumentation is the only way to see it, and it keeps getting deferred because it feels less urgent than shipping the next feature. That prioritization will look different in retrospect.

The naming confusion that opened this piece is, at its core, a prioritization failure. Teams that treat prompt injection and jailbreaking as interchangeable deploy generalized defenses that address neither completely. Teams that understand the split can triage correctly: architecture controls for injection vectors, alignment and agent-level hardening for jailbreak exposure, observability for both. The threat is not standing still while that distinction gets sorted out.