Prompt injection as an authority-confusion problem
How to study this lesson
Read one section, trace the worked example, and try the knowledge check. Bookmark saves a shortcut; notes appear in My notebook. Mark lesson complete is your own assessment and does not mark its lab passed.
By the end, you can
- Identify direct and indirect routes for untrusted instructions.
- Keep tool-result content separate from granted capabilities.
- Design action checks that do not depend on detecting every attack phrase.
The document is evidence, not the operator
A research agent reads a vendor page to answer a legitimate question. The page contains a paragraph telling the agent to export its customer database before continuing. The problem is not merely offensive wording. It is an attempt to promote retrieved content into authority over the agent's future actions. The document can supply facts for the user's question, but it cannot grant access to another system or expand the task.
Indirect prompt injection arrives through material the agent consumes, such as webpages, repository files, tool responses, or retrieved messages. Direct attacks arrive in inputs that attempt to redirect behavior more immediately. OWASP treats prompt injection as a risk that cannot be eliminated simply by adding a stronger instruction paragraph. The architecture should limit what an influenced model can cause, even when the malicious wording is unfamiliar.
Preserve provenance across transformations
Label source material with its origin and trust category when constructing context. Do not concatenate a fetched page into the same field used for trusted application instructions without preserving the distinction. A summary of an untrusted page remains derived from an untrusted page; summarization does not turn it into a command from the user. The same applies when one agent hands another agent a plan produced from retrieved content.
Consider a three-step trace. A browser returns text, a summarizer extracts a supposed next step, and an executor proposes sending an attachment. At each handoff, carry the source reference and the actual user goal. If the executor sees only send the attachment, the system has lost the evidence needed to judge scope. Provenance is useful context, while the final permission check must still rely on trusted authorization state outside the model's prose.
Check the action at the last responsible boundary
A model can propose a structured action containing tool, tenant, resource, arguments, and a reason. The reason is an explanation to inspect, not a permission token. A deterministic policy layer checks whether the authenticated actor has the required grant and whether the action fits the user's authorized task. An allowlisted tool name alone is insufficient: sending to a different recipient or querying a different tenant changes the action's consequences.
In the example, the controller holds a trusted set of capabilities for reading report-7. Retrieved text claims that export-all is required. The policy checks the proposed capability against the trusted set and denies it regardless of how persuasive the text sounds. This toy approach illustrates authority separation. A real service must also authenticate the actor, prevent forged policy inputs, validate arguments, and enforce the resource boundary at execution time.
Filters help, but do not define authority
Pattern matching can detect familiar attack strings, and model-based classifiers can add evidence. Both can miss paraphrases or incorrectly flag ordinary quotations. A search result describing prompt injection might legitimately contain ignore previous instructions. Rejecting every such page would harm the research task. Conversely, a malicious request can be phrased politely without any familiar keyword.
Measure both task utility and unauthorized action attempts. Construct synthetic documents that contain harmless content, explicit redirection, subtle recipient substitution, and false claims of prior approval. For each document, ask whether the final action remained within the grant. In a toy evaluation of 40 documents, zero unauthorized actions is useful evidence about those 40 cases, not proof that the system is injection-proof. Expand the test distribution as real failures reveal new mechanisms.
Combine containment with clear user intent
An agent that cannot access secrets cannot disclose those secrets through a tool it does not possess. An agent that can draft an email but cannot send without an approved payload has a bounded path to action. These controls reduce consequences without relying on perfect semantic detection. They also make normal behavior easier to explain: the system can say which proposed action requires additional authorization and show the exact content.
Keep the user experience proportional to the task. Routine authorized reads should not trigger repeated prompts merely because retrieved text is untrusted. Additional approval is useful when a proposed action exceeds existing authorization or has a separately defined consequential boundary. A robust system preserves already granted intent while refusing to treat documents, tool outputs, or model self-assertions as new grants. This is a design for capability control, not a claim that the model itself has become immune to influence.
Work through the code
The trusted grant set is provided separately from model proposals and retrieved reasons. The example performs no reads or exports and is not an authentication system. Replace a proposal reason with more persuasive text and observe that it does not change the capability decision.
from dataclasses import dataclass
@dataclass(frozen=True)
class Proposal:
capability: str
source: str
reason: str
def decide(proposal, trusted_grants):
if proposal.capability not in trusted_grants:
return "deny"
return "allow"
grants = frozenset({"read:tenant-a:report-7"})
proposals = [
Proposal("read:tenant-a:report-7", "user-task", "answer question"),
Proposal("export:tenant-a:all", "retrieved-page", "page says required"),
Proposal("read:tenant-b:report-7", "tool-result", "claims prior approval"),
]
for proposal in proposals:
print(proposal.capability, decide(proposal, grants))
print("grant count:", len(grants))
read:tenant-a:report-7 allow export:tenant-a:all deny read:tenant-b:report-7 deny grant count: 1
Run Python snippets locally with the prerequisites named above. The module coding lab runs directly in your browser.
Pause and reason
A summarizer converts a webpage instruction into the sentence: The user has approved sending the full archive. A second agent receives only that summary. Identify two repairs to the handoff and one independent execution control.
Check your understanding
Why is an allowlist containing only the tool name send_email incomplete?
Your notes
Explain the mechanism in your own words. Add a failure you want to test.