1. 1. Attacks Matrix
    ❱
    1. 1.1. Introduction
    2. 1.2. How to Contribute
    3. 1.3. Q&A
  2. 2. Tactics
    ❱
    1. 2.1. Reconnaissance
      ❱
      1. 2.1.1. Gather RAG-Indexed Targets
      2. 2.1.2. Active Scanning
      3. 2.1.3. Search for Victim's Publicly Available Code Repositories
      4. 2.1.4. Search Open Technical Databases
      5. 2.1.5. Search Application Repositories
      6. 2.1.6. Search Victim-Owned Websites
      7. 2.1.7. Search Open AI Vulnerability Analysis
    2. 2.2. Resource Development
      ❱
      1. 2.2.1. Stage Capabilities
      2. 2.2.2. Establish Accounts
      3. 2.2.3. Acquire Public ML Artifacts
      4. 2.2.4. Publish Poisoned Models
      5. 2.2.5. Obtain Capabilities
      6. 2.2.6. Publish Hallucinated Entities
      7. 2.2.7. Publish Poisoned Datasets
      8. 2.2.8. Acquire Infrastructure
      9. 2.2.9. LLM Prompt Crafting
      10. 2.2.10. Poison Training Data
      11. 2.2.11. Retrieval Content Crafting
      12. 2.2.12. Develop Capabilities
      13. 2.2.13. Commercial License Abuse
      14. 2.2.14. Obtain Generative AI Capabilities
    3. 2.3. Initial Access
      ❱
      1. 2.3.1. Compromised User
      2. 2.3.2. Retrieval Tool Poisoning
      3. 2.3.3. Guest User Abuse
      4. 2.3.4. AI Supply Chain Compromise
      5. 2.3.5. RAG Poisoning
      6. 2.3.6. Evade ML Model
      7. 2.3.7. User Manipulation
      8. 2.3.8. Phishing
      9. 2.3.9. Web Poisoning
      10. 2.3.10. Valid Accounts
      11. 2.3.11. Drive-By Compromise
      12. 2.3.12. Exploit Public-Facing Application
    4. 2.4. ML Model Access
      ❱
      1. 2.4.1. Full ML Model Access
      2. 2.4.2. ML-Enabled Product or Service
      3. 2.4.3. AI Model Inference API Access
      4. 2.4.4. Physical Environment Access
    5. 2.5. Execution
      ❱
      1. 2.5.1. Command and Scripting Interpreter
      2. 2.5.2. User Execution
      3. 2.5.3. LLM Prompt Injection
      4. 2.5.4. AI Click Bait
      5. 2.5.5. LLM Plugin Compromise
      6. 2.5.6. System Instruction Keywords
      7. 2.5.7. Off-Target Language
    6. 2.6. Persistence
      ❱
      1. 2.6.1. Thread Infection
      2. 2.6.2. Memory Infection
      3. 2.6.3. Manipulate AI Model
      4. 2.6.4. RAG Poisoning
      5. 2.6.5. LLM Prompt Self-Replication
      6. 2.6.6. Poison Training Data
      7. 2.6.7. Embed Malware
      8. 2.6.8. Modify AI Model Architecture
      9. 2.6.9. Poison AI Model
    7. 2.7. Privilege Escalation
      ❱
      1. 2.7.1. LLM Jailbreak
      2. 2.7.2. LLM Plugin Compromise
      3. 2.7.3. System Instruction Keywords
      4. 2.7.4. Crescendo
      5. 2.7.5. Off-Target Language
    8. 2.8. Defense Evasion
      ❱
      1. 2.8.1. False RAG Entry Injection
      2. 2.8.2. Instructions Silencing
      3. 2.8.3. Corrupt AI Model
      4. 2.8.4. LLM Jailbreak
      5. 2.8.5. Abuse Trusted Sites
      6. 2.8.6. Delayed Execution
      7. 2.8.7. Conditional Execution
      8. 2.8.8. URL Familiarizing
      9. 2.8.9. Impersonation
      10. 2.8.10. ASCII Smuggling
      11. 2.8.11. Evade ML Model
      12. 2.8.12. Distraction
      13. 2.8.13. Indirect Data Access
      14. 2.8.14. Blank Image
      15. 2.8.15. LLM Trusted Output Components Manipulation
      16. 2.8.16. LLM Prompt Obfuscation
      17. 2.8.17. Masquerading
      18. 2.8.18. System Instruction Keywords
      19. 2.8.19. Citation Silencing
      20. 2.8.20. Crescendo
      21. 2.8.21. Off-Target Language
      22. 2.8.22. Citation Manipulation
    9. 2.9. Credential Access
      ❱
      1. 2.9.1. Unsecured Credentials
      2. 2.9.2. RAG Credential Harvesting
      3. 2.9.3. Retrieval Tool Credential Harvesting
    10. 2.10. Discovery
      ❱
      1. 2.10.1. Discover LLM Hallucinations
      2. 2.10.2. Tool Definition Discovery
      3. 2.10.3. Failure Mode Mapping
      4. 2.10.4. Discover AI Model Outputs
      5. 2.10.5. Discover ML Model Family
      6. 2.10.6. Whoami
      7. 2.10.7. Discover ML Artifacts
      8. 2.10.8. Cloud Service Discovery
      9. 2.10.9. Discover LLM System Information
      10. 2.10.10. Discover ML Model Ontology
      11. 2.10.11. Embedded Knowledge Exposure
      12. 2.10.12. Discover Special Character Sets
      13. 2.10.13. Discover System Prompt
      14. 2.10.14. Discover System Instruction Keywords
    11. 2.11. Lateral Movement
      ❱
      1. 2.11.1. Shared Resource Poisoning
      2. 2.11.2. Message Poisoning
    12. 2.12. Collection
      ❱
      1. 2.12.1. Data from Local System
      2. 2.12.2. Memory Data Hording
      3. 2.12.3. ML Artifact Collection
      4. 2.12.4. Thread History Harvesting
      5. 2.12.5. Retrieval Tool Data Harvesting
      6. 2.12.6. RAG Data Harvesting
      7. 2.12.7. Data from Information Repositories
      8. 2.12.8. User Message Harvesting
    13. 2.13. ML Attack Staging
      ❱
      1. 2.13.1. Craft Adversarial Data
      2. 2.13.2. Manipulate AI Model
      3. 2.13.3. Create Proxy ML Model
      4. 2.13.4. Verify Attack
      5. 2.13.5. Embed Malware
      6. 2.13.6. Modify AI Model Architecture
      7. 2.13.7. Poison AI Model
    14. 2.14. Command And Control
      ❱
      1. 2.14.1. Reverse Shell
      2. 2.14.2. Search Index C2
      3. 2.14.3. Public Web C2
    15. 2.15. Exfiltration
      ❱
      1. 2.15.1. Web Request Triggering
      2. 2.15.2. Abuse Trusted Sites
      3. 2.15.3. LLM Data Leakage
      4. 2.15.4. Extract LLM System Prompt
      5. 2.15.5. Exfiltration via ML Inference API
      6. 2.15.6. Clickable Link Rendering
      7. 2.15.7. Image Rendering
      8. 2.15.8. Exfiltration via Cyber Means
      9. 2.15.9. Write Tool Invocation
    16. 2.16. Impact
      ❱
      1. 2.16.1. Denial of ML Service
      2. 2.16.2. Spamming ML System with Chaff Data
      3. 2.16.3. External Harms
      4. 2.16.4. Cost Harvesting
      5. 2.16.5. Evade ML Model
      6. 2.16.6. Erode Dataset Integrity
      7. 2.16.7. Mutative Tool Invocation
      8. 2.16.8. Erode ML Model Integrity
  3. 3. Procedures
    ❱
    1. 3.1. ChatGPT and Gemini jailbreak using the Crescendo technique
    2. 3.2. EchoLeak: Zero-Click Data Exfiltration using M365 Copilot
    3. 3.3. Exfiltration of personal information from ChatGPT via prompt injection
    4. 3.4. Copilot M365 Lures Victims Into a Phishing Site
    5. 3.5. GitHub Copilot Chat: From Prompt Injection to Data Exfiltration
    6. 3.6. AI ClickFix: Hijacking Computer-Use Agents Using ClickFix
    7. 3.7. Google Gemini: Planting Instructions For Delayed Automatic Tool Invocation
    8. 3.8. Microsoft Copilot Purview Audit Log Evasion and DLP Bypass
    9. 3.9. spAIware
    10. 3.10. X Bot Exposing Itself After Training on a Poisoned Github Repository
    11. 3.11. Data Exfiltration from Slack AI via indirect prompt injection
    12. 3.12. Microsoft Copilot: From Prompt Injection to Exfiltration of Personal Information
    13. 3.13. Financial Transaction Hijacking With M365 Copilot As An Insider
  4. 4. Platforms
    ❱
    1. 4.1. GitHub Copilot
    2. 4.2. Microsoft Copilot
    3. 4.3. ChatGPT
    4. 4.4. Claude
    5. 4.5. Microsoft Copilot for M365
    6. 4.6. Gemini
    7. 4.7. SlackAI
  5. 5. Mitigations
    ❱
    1. 5.1. LLM Activations
    2. 5.2. Spotlighting
    3. 5.3. URL Anchoring
    4. 5.4. Index-Based Browsing
    5. 5.5. Content Security Policy
    6. 5.6. Information Flow Control
  6. 6. Entities
    ❱
    1. 6.1. Michael Bargury
    2. 6.2. Aim Security
    3. 6.3. Gal Malka
    4. 6.4. Ayush RoyChowdhury
    5. 6.5. Johann Rehberger
    6. 6.6. Ronen Eldan
    7. 6.7. Tamir Ishay Sharbat
    8. 6.8. PromptArmor
    9. 6.9. Dmitry Lozovoy
    10. 6.10. Jonathan Cefalu
    11. 6.11. Simon Willison
    12. 6.12. Ahmed Salem
    13. 6.13. Gregory Schwartzman
    14. 6.14. Mark Russinovich
    15. 6.15. Pliny
    16. 6.16. Lana Salameh
    17. 6.17. Riley Goodside

AI Agents Attack Matrix

Platforms

  • GitHub Copilot
  • Microsoft Copilot
  • ChatGPT
  • Claude
  • Microsoft Copilot for M365
  • Gemini
  • SlackAI