RAG Poisoning
Type: technique
Description: Adversaries may inject malicious content into data indexed by a RAG system to contaminate a future thread through RAG-based search results. This may be accomplished by placing manipulated documents in a location the RAG indexes (see Gather RAG-Indexed Targets). The content may be targeted such that it would always surface as a search result for a specific user query. The adversary’s content may include false or misleading information. It may also include prompt injections with malicious instructions, or false RAG entries.
Version: 0.1.0
Created At: 2025-03-04 10:27:40 -0500
Last Modified At: 2025-03-04 10:27:40 -0500
External References
- Retrieval-augmented generation., Wikipedia
- BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models., arXiv
- ConfusedPilot: Confused Deputy Risks in RAG-based LLMs., arXiv
- PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models., arXiv
Related Objects
- --> Initial Access (tactic): An adversary can gain initial access by injecting malicious content into a publicly available data source indexed by RAG.
- --> Persistence (tactic): An adversary can gain persistence by creating or modifying an internal data source indexed by RAG that users interact with.
- --> Johann Rehberger (entity): Demonstrated by
- --> Michael Bargury (entity): Demonstrated by
- --> Tamir Ishay Sharbat (entity): Demonstrated by
- --> Ayush RoyChowdhury (entity): Demonstrated by
- <-- Retrieval Content Crafting (technique): Targeted RAG Poisoning is a form of RAG Poisoning, crafting malicious content to surface for a specific user query.
- <-- Resource Poisoning (technique): Resource Poisoning is a form of RAG Poisoning, targeting future threads of the same user.
- <-- Shared Resource Poisoning (technique): Shared Resource Poisoning is a form of RAG Poisoning, leveraging acquired intra-company access for lateral movement.
- <-- Google Gemini: Planting Instructions For Delayed Automatic Tool Invocation (procedure): Adversary creates an email with embedded instructions and sends it to victim. The victim interacts with that email through Gemini.
- <-- Microsoft Copilot: From Prompt Injection to Exfiltration of Personal Information (procedure): Copilots gets access to malicious data via email or shared document.