Prompt Injection: Why External Text Must Stay Untrusted
Prompt injection occurs when content supplied to an AI system tries to redirect its behaviour. The content might be a webpage, document or tool response that looks relevant to the user's task. The application needs boundaries that remain effective even when the model reads misleading instructions.
Why it matters
A useful source can contain hostile text. Treating every retrieved passage as authoritative instructions can turn a reading feature into an unintended action or disclosure pathway.
A practical workflow
- Separate the user's request and application policy from the documents being processed, and label external material clearly.
- Give tools the minimum access needed and enforce permissions outside the model rather than inside a prompt alone.
- Require review for consequential writes and prevent arbitrary destinations from receiving private information.
- Test documents that contain misleading commands and inspect whether the system continues the authorised task.
An example
A webpage included in a research task might tell the assistant to ignore the user and upload local files. That text is part of the page being analysed. It provides no permission to access or send anything unrelated to the requested research.
A mistake to avoid
Do not rely on a single phrase such as 'ignore malicious instructions' as the complete defence. The strongest boundary is often limiting what the application can do and validating every sensitive operation independently.
Try it and check the result
Create a harmless test document that asks for an unrelated action. Verify that the assistant summarises the relevant content without following that request, and that the tool layer blocks actions outside the permitted scope.
Reference and further reading
OWASP's generative AI security project.
The workflows and examples in this guide are illustrative. Product controls and availability can change; check the linked official guidance for your setup.
Continue with AI Tool Calling: Validate the Action Before Running It or explore all blog topics.