False RAG Entry Injection
Type: technique
Description: Adversaries may introduce false entries into a victim’s retrieval augmented generation (RAG) database. Content designed to be interpreted as a document by the large language model (LLM) used in the RAG system is included in a data source being ingested into the RAG database. When RAG entry including the false document is retrieved, the LLM is tricked into treating part of the retrieved content as a false RAG result. By including a false RAG document inside of a regular RAG entry, it bypasses data monitoring tools. It also prevents the document from being deleted directly. The adversary may use discovered system keywords to learn how to instruct a particular LLM to treat content as a RAG entry. They may be able to manipulate the injected entry’s metadata including document title, author, and creation date.
Version: 0.1.0
Created At: 2025-03-04 10:27:40 -0500
Last Modified At: 2025-03-04 10:27:40 -0500
External References
Related Objects
- --> Defense Evasion (tactic): An adversary can inject false RAG entries that are treated by the AI system as authentic.
- --> Tamir Ishay Sharbat (entity): Demonstrated by
- <-- Financial Transaction Hijacking With M365 Copilot As An Insider (procedure): Inject a new RAG result to ensure copilot does not reference or confuse the malicious email with the newly provided malicious data.
- <-- Copilot M365 Lures Victims Into a Phishing Site (procedure): Inject a new RAG result to ensure copilot does not reference or confuse the malicious email with the newly provided malicious data.