1. 1. Attacks Matrix
    ❱
    1. 1.1. Introduction
    2. 1.2. How to Contribute
    3. 1.3. Q&A
  2. 2. Tactics
    ❱
    1. 2.1. Reconnaissance
      ❱
      1. 2.1.1. Search Application Repositories
      2. 2.1.2. Active Scanning
      3. 2.1.3. Search Open Technical Databases
      4. 2.1.4. Search for Victim's Publicly Available Code Repositories
      5. 2.1.5. Search Open AI Vulnerability Analysis
      6. 2.1.6. Search Victim-Owned Websites
      7. 2.1.7. Gather RAG-Indexed Targets
    2. 2.2. Resource Development
      ❱
      1. 2.2.1. Publish Poisoned Datasets
      2. 2.2.2. Publish Hallucinated Entities
      3. 2.2.3. Develop Capabilities
      4. 2.2.4. Commercial License Abuse
      5. 2.2.5. Obtain Capabilities
      6. 2.2.6. Stage Capabilities
      7. 2.2.7. Establish Accounts
      8. 2.2.8. Acquire Public AI Artifacts
      9. 2.2.9. Retrieval Content Crafting
      10. 2.2.10. Publish Poisoned Models
      11. 2.2.11. Acquire Infrastructure
      12. 2.2.12. LLM Prompt Crafting
      13. 2.2.13. Poison Training Data
      14. 2.2.14. Obtain Generative AI Capabilities
    3. 2.3. Initial Access
      ❱
      1. 2.3.1. Drive-By Compromise
      2. 2.3.2. Retrieval Tool Poisoning
      3. 2.3.3. Evade AI Model
      4. 2.3.4. RAG Poisoning
      5. 2.3.5. User Manipulation
      6. 2.3.6. Exploit Public-Facing Application
      7. 2.3.7. Valid Accounts
      8. 2.3.8. Compromised User
      9. 2.3.9. Web Poisoning
      10. 2.3.10. Phishing
      11. 2.3.11. AI Supply Chain Compromise
      12. 2.3.12. Guest User Abuse
    4. 2.4. AI Model Access
      ❱
      1. 2.4.1. AI-Enabled Product or Service
      2. 2.4.2. AI Model Inference API Access
      3. 2.4.3. Full AI Model Access
      4. 2.4.4. Physical Environment Access
    5. 2.5. Execution
      ❱
      1. 2.5.1. Command and Scripting Interpreter
      2. 2.5.2. AI Click Bait
      3. 2.5.3. User Execution
      4. 2.5.4. AI Agent Tool Invocation
      5. 2.5.5. LLM Prompt Injection
      6. 2.5.6. System Instruction Keywords
      7. 2.5.7. Triggered Prompt Injection
      8. 2.5.8. Indirect Prompt Injection
      9. 2.5.9. Direct Prompt Injection
      10. 2.5.10. Off-Target Language
    6. 2.6. Persistence
      ❱
      1. 2.6.1. AI Agent Context Poisoning
      2. 2.6.2. RAG Poisoning
      3. 2.6.3. Manipulate AI Model
      4. 2.6.4. Modify AI Agent Configuration
      5. 2.6.5. LLM Prompt Self-Replication
      6. 2.6.6. Poison Training Data
      7. 2.6.7. Thread Poisoning
      8. 2.6.8. Embed Malware
      9. 2.6.9. Modify AI Model Architecture
      10. 2.6.10. Memory Poisoning
      11. 2.6.11. Poison AI Model
    7. 2.7. Privilege Escalation
      ❱
      1. 2.7.1. AI Agent Tool Invocation
      2. 2.7.2. LLM Jailbreak
      3. 2.7.3. System Instruction Keywords
      4. 2.7.4. Crescendo
      5. 2.7.5. Off-Target Language
    8. 2.8. Defense Evasion
      ❱
      1. 2.8.1. Evade AI Model
      2. 2.8.2. Corrupt AI Model
      3. 2.8.3. False RAG Entry Injection
      4. 2.8.4. Blank Image
      5. 2.8.5. LLM Prompt Obfuscation
      6. 2.8.6. Masquerading
      7. 2.8.7. Distraction
      8. 2.8.8. Instructions Silencing
      9. 2.8.9. Impersonation
      10. 2.8.10. URL Familiarizing
      11. 2.8.11. Indirect Data Access
      12. 2.8.12. Abuse Trusted Sites
      13. 2.8.13. LLM Trusted Output Components Manipulation
      14. 2.8.14. Conditional Execution
      15. 2.8.15. ASCII Smuggling
      16. 2.8.16. LLM Jailbreak
      17. 2.8.17. Citation Silencing
      18. 2.8.18. Citation Manipulation
      19. 2.8.19. System Instruction Keywords
      20. 2.8.20. Crescendo
      21. 2.8.21. Off-Target Language
    9. 2.9. Credential Access
      ❱
      1. 2.9.1. Credentials from AI Agent Configuration
      2. 2.9.2. Unsecured Credentials
      3. 2.9.3. Retrieval Tool Credential Harvesting
      4. 2.9.4. RAG Credential Harvesting
    10. 2.10. Discovery
      ❱
      1. 2.10.1. Discover AI Agent Configuration
      2. 2.10.2. Discover AI Model Ontology
      3. 2.10.3. Discover LLM System Information
      4. 2.10.4. Failure Mode Mapping
      5. 2.10.5. Discover LLM Hallucinations
      6. 2.10.6. Discover AI Artifacts
      7. 2.10.7. Discover AI Model Family
      8. 2.10.8. Whoami
      9. 2.10.9. Cloud Service Discovery
      10. 2.10.10. Discover AI Model Outputs
      11. 2.10.11. Discover Embedded Knowledge
      12. 2.10.12. Discover System Prompt
      13. 2.10.13. Discover Tool Definitions
      14. 2.10.14. Discover Activation Triggers
      15. 2.10.15. Discover Special Character Sets
      16. 2.10.16. Discover System Instruction Keywords
    11. 2.11. Lateral Movement
      ❱
      1. 2.11.1. Message Poisoning
      2. 2.11.2. Shared Resource Poisoning
    12. 2.12. Collection
      ❱
      1. 2.12.1. Data from AI Services
      2. 2.12.2. Data from Information Repositories
      3. 2.12.3. Thread History Harvesting
      4. 2.12.4. Memory Data Hording
      5. 2.12.5. User Message Harvesting
      6. 2.12.6. AI Artifact Collection
      7. 2.12.7. Data from Local System
      8. 2.12.8. Retrieval Tool Data Harvesting
      9. 2.12.9. RAG Data Harvesting
    13. 2.13. AI Attack Staging
      ❱
      1. 2.13.1. Verify Attack
      2. 2.13.2. Manipulate AI Model
      3. 2.13.3. Craft Adversarial Data
      4. 2.13.4. Create Proxy AI Model
      5. 2.13.5. Embed Malware
      6. 2.13.6. Modify AI Model Architecture
      7. 2.13.7. Poison AI Model
    14. 2.14. Command And Control
      ❱
      1. 2.14.1. Public Web C2
      2. 2.14.2. Search Index C2
      3. 2.14.3. Reverse Shell
    15. 2.15. Exfiltration
      ❱
      1. 2.15.1. Web Request Triggering
      2. 2.15.2. Exfiltration via AI Inference API
      3. 2.15.3. Image Rendering
      4. 2.15.4. Exfiltration via Cyber Means
      5. 2.15.5. Extract LLM System Prompt
      6. 2.15.6. LLM Data Leakage
      7. 2.15.7. Clickable Link Rendering
      8. 2.15.8. Abuse Trusted Sites
      9. 2.15.9. Exfiltration via AI Agent Tool Invocation
    16. 2.16. Impact
      ❱
      1. 2.16.1. Evade AI Model
      2. 2.16.2. Spamming AI System with Chaff Data
      3. 2.16.3. Erode AI Model Integrity
      4. 2.16.4. Erode Dataset Integrity
      5. 2.16.5. Mutative Tool Invocation
      6. 2.16.6. Cost Harvesting
      7. 2.16.7. Denial of AI Service
      8. 2.16.8. External Harms
  3. 3. Procedures
    ❱
    1. 3.1. Microsoft Copilot Purview Audit Log Evasion and DLP Bypass
    2. 3.2. X Bot Exposing Itself After Training on a Poisoned Github Repository
    3. 3.3. ChatGPT and Gemini jailbreak using the Crescendo technique
    4. 3.4. Copilot M365 Lures Victims Into a Phishing Site
    5. 3.5. EchoLeak: Zero-Click Data Exfiltration using M365 Copilot
    6. 3.6. Google Gemini: Planting Instructions For Delayed Automatic Tool Invocation
    7. 3.7. GitHub Copilot Chat: From Prompt Injection to Data Exfiltration
    8. 3.8. AI ClickFix: Hijacking Computer-Use Agents Using ClickFix
    9. 3.9. spAIware
    10. 3.10. Microsoft Copilot: From Prompt Injection to Exfiltration of Personal Information
    11. 3.11. Financial Transaction Hijacking With M365 Copilot As An Insider
    12. 3.12. Exfiltration of personal information from ChatGPT via prompt injection
    13. 3.13. Data Exfiltration from Slack AI via indirect prompt injection
  4. 4. Platforms
    ❱
    1. 4.1. SlackAI
    2. 4.2. Microsoft Copilot
    3. 4.3. Claude
    4. 4.4. Microsoft Copilot for M365
    5. 4.5. Gemini
    6. 4.6. ChatGPT
    7. 4.7. GitHub Copilot
  5. 5. Mitigations
    ❱
    1. 5.1. Content Security Policy
    2. 5.2. URL Anchoring
    3. 5.3. LLM Activations
    4. 5.4. Information Flow Control
    5. 5.5. Index-Based Browsing
    6. 5.6. Spotlighting
  6. 6. Entities
    ❱
    1. 6.1. Simon Willison
    2. 6.2. PromptArmor
    3. 6.3. Dmitry Lozovoy
    4. 6.4. Gal Malka
    5. 6.5. Gregory Schwartzman
    6. 6.6. Pliny
    7. 6.7. Ronen Eldan
    8. 6.8. Lana Salameh
    9. 6.9. Mark Russinovich
    10. 6.10. Ahmed Salem
    11. 6.11. Riley Goodside
    12. 6.12. Jonathan Cefalu
    13. 6.13. Ayush RoyChowdhury
    14. 6.14. Tamir Ishay Sharbat
    15. 6.15. Michael Bargury
    16. 6.16. Aim Security
    17. 6.17. Johann Rehberger

AI Agents Attack Matrix

Mitigations

  • Content Security Policy
  • URL Anchoring
  • LLM Activations
  • Information Flow Control
  • Index-Based Browsing
  • Spotlighting