Citation Manipulation
Type: technique
Description: The adversary manipulates citations provided by the AI system to add trustworthiness to their social engineering attack. Variants include providing the wrong citation, making up a new one or providing the right citation for the wrong data.
Version: 0.1.0
Created At: 2024-10-03 22:24:49 +0300
Last Modified At: 2024-10-03 22:24:49 +0300
External References
Related Objects
- --> Impact (tactic): An adversary can social engineer by providing trustworthy sources to maliciously-crafted messages or data.
- --> Citation Silencing (technique): An adjacent technique which also includes adversary control over citations.
- --> Michael Bargury (entity): Demonstrated by
- --> Tamir Ishay Sharbat (entity): Demonstrated by
- --> Gal Malka (entity): Demonstrated by
- <-- Data Exfiltration from Slack AI via indirect prompt injection (procedure): Slack cites the message from the private channel where the secret was found, not the message from the public channel that contained the injection. This is the native behavior of SlackAI, and is not an explicit result of the adversary's attack.
- <-- Financial Transaction Hijacking With M365 Copilot As An Insider (procedure): Force copilot to output a reference to the legitimate Vendor Details excel spreadsheet and not to the malicious email.
- <-- Copilot M365 Lures Victims Into a Phishing Site (procedure): Force copilot to output a reference to the phishing website.