AI Social Engineering

Type: technique

Description: The adversary uses the AI system as a malicious insider, social engineering the system's user abusing its trustworthiness.

Version: 0.1.0

Created At: 2024-10-03 22:24:49 +0300

Last Modified At: 2024-10-03 22:24:49 +0300

External References

  • --> Impact (tactic): An adversary can social engineer their victim through interactions with their trusted AI systems.
  • <-- Data Exfiltration from Slack AI via indirect prompt injection (procedure): Once a victim asks SlackAI about the targeted username, SlackAI responds by providing a link to a phishing website. cites the message from the private channel where the secret was found, not the message from the public channel that contained the injection. This is the native behavior of SlackAI, and is not an explicit result of the adversary's attack.
  • <-- Financial Transaction Hijacking With M365 Copilot As An Insider (procedure): Provide a trustworthy response to the user so they feel comfortable moving forward with the wire..
  • <-- Copilot M365 Lures Victims Into a Phishing Site (procedure): Entice the user to click on the link to the phishing website: Access the Power Platform Admin Center..