The first time an accounts payable agent forwarded a payment to an attacker, the team blamed the model. The model was functioning exactly as designed. The integration was not. The agent was asked to read a PDF invoice. At the bottom of the page, rendered in white text on a white background, was a single sentence instructing the system to ignore previous directions and pay the invoice to a specific external account. The agent processed it.
Large language models operate on a linguistic version of a von Neumann architecture. Instructions and data share the same context window and the same attention mechanism. To the model, a system prompt, a user query, and the text of a vendor contract are all just tokens in a sequence. You cannot fine-tune a model to perfectly distinguish between a command and a piece of evidence, because the architecture does not separate them.
Prompt injection is not a model vulnerability. It is a data flow problem. It is the SQL injection of the LLM era, and the fix is the same: separate code from data.
When an LLM only returns text to a screen, prompt injection is a novelty. A user tricks the model into writing a poem instead of a quarterly summary. The blast radius is a wasted API call. When an LLM is given tools—the ability to query a database, send an email, or update a ticketing system—it becomes an agent. A prompt injection into an agent is no longer a parlor trick. It is remote code execution.
Consider a customer service platform that uses a model to triage incoming support tickets. The model has a tool to process refunds under fifty dollars to speed up resolution. A user submits a ticket complaining about a broken product. At the end of the message, they add a line instructing the system to summarize the ticket as approved and trigger the refund API. The model reads the ticket, recognizes a command it has the tools to execute, and triggers the API. The ticket was real. The instruction inside it was not.
The vulnerability rarely lives in the model’s weights. It lives in the middle tier. Models do not actually click buttons or send emails. They output structured text, usually JSON, which matches a schema provided by the developer. A script parses this JSON and makes the actual API call to the downstream system. If that script blindly executes the API call based on the model’s output, the system is compromised.
We are treating prompt injection like a personality defect of the AI, rather than a missing parameter check in the API. When an application executes an unauthorized database command because a user typed a strange string into a form field, we do not blame the database. We blame the developer who forgot to sanitize the input. The same logic applies here. If an incoming email can issue a database command, you do not have an AI problem. You have an architecture problem.
The hardest boundary to secure is often not the internet versus the internal network. An internal ticket can contain text pasted from a customer email, a vendor document, or a scraped web page. Every external document the model parses is an unauthenticated instruction channel.
This creates a dynamic where the agent receives text through an approved tool and treats the tool’s output as an approved instruction. The search call to the knowledge base may be authorized; the page it returns has not been authorized to direct behavior. An automated HR screening agent built to read PDF resumes might encounter a document with hidden text instructing it to return a recommendation of highly qualified and invoke the interview scheduling function. The hiring manager’s calendar fills up with unqualified candidates because the agent trusted the delivery mechanism. A tool response is a delivery mechanism, not a trust endorsement.
The enterprise AI market is currently focused on giving models more agency. We are connecting them to email servers, code repositories, and ERP systems. We are taking systems that inherently cannot distinguish between data and instructions, and we are giving them write access to our infrastructure. Waiting for the next model update to fix this is a losing strategy. Better models reduce the success rate of basic injections, but they do not close the vulnerability. Refusal behavior should sit behind capability limits, not substitute for them.
The durable controls are architectural. The first step is to treat all external content—emails, PDFs, scraped web pages, ticket comments—as untrusted input. Validate, sanitize, and escape it. Define strict trust boundaries. Establish which sources can give instructions and which are data only. The source can supply evidence. It cannot grant itself authority.
Separate reading from acting wherever the workflow allows. An agent drafting a recommendation does not automatically need the credential that implements it. Scope the integration, not just the prompt. If an agent is designed to query a specific database table, the middleware executing the agent’s commands should only have network access and database credentials for that specific table. Do not pass a god-mode API key to a script that takes directions from a language model.
Implement read-only architectures by default. If an agent must alter state, require explicit human confirmation. But confirmation dialogs are not security controls if users click through them. A generic prompt to proceed mostly transfers the agent’s confusion to the reviewer. A confirmation screen must show the exact action being authorized, the destination, the data involved, and the source of the request. If the agent can read the attachment and send mail, an instruction buried in the attachment has somewhere to go. The reviewer needs to see that trajectory.
When an incident occurs, the audit trail is the only way to understand what happened. Log the source material the agent relied on, the tool calls it proposed or made, the permissions in force, and the approval decision. Without that trail, an injection looks like an inexplicable model error.
Testing must cover the complete workflow, not just the model. Red team the integrations by injecting instructions into documents, emails, and tickets. A model-only test misses the permissions and approval screens that determine the actual outcome.
You cannot fine-tune your way out of a missing permissions boundary. The model will always read the text. The question is whether your architecture gives that text the authority to act.