Everything about Prompt injection
Type of attack in machine learning
Prompt injection is a cybersecurity exploit and an attack vector in which innocuous-looking inputs are designed to cause unintended behavior in machine learning models, particularly large language models (LLMs). The attack takes advantage of the model's inability to distinguish between developer-defined prompts and user inputs to bypass safeguards and influence model behaviour. While LLMs are designed to follow trusted instructions, they can be manipulated into carrying out unintended responses through carefully crafted inputs. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Prompt injectionA 5-minute orientation
- Skim a recent paper: Systematically Analysing Prompt Injection Vulnerabilities in Diverse LLM ArchitecturesInfluential work since 2022
- Pick up a classic book: The AI Hacker's HandbookThe most-reprinted book on the subject
- Read the landmark paper: Prompt injection attacks on vision language models in oncologyCited 72 times
- Go sideways: Preamble (company)A closely related topic
- Go sideways: SQL injectionA closely related topic
📖 Definitions
Noun A method of manipulating an artificial intelligence system by embedding malicious instructions (prompts) within input data in order to trick the model into overriding its intended behaviour or safeguards.
Noun A method of causing an artificial intelligence to ignore its initial instructions (often ethical restrictions) by directly giving it a particular prompt; a jailbreak.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
Prompt injection attacks on vision language models in oncology
Vulnerability of Large Language Models to Prompt Injection When Providing Medical Advice
Mind Mapping Prompt Injection: Visual Prompt Injection Attacks in Modern Large Language Models
Numerical study of neutral beam injection prompt loss at KSTAR poloidal limiters
Prompt injection of relativistic protons from the September 1, 1971 solar flare
Prompt radial electric field response to neutral beam injection
LLM Firewall Using Validator Agent for Prevention Against Prompt Injection Attacks
Prompt-Based Word-Level Information Injection BERT for Chinese Named Entity Recognition
🆕 Influential since 2022
Systematically Analysing Prompt Injection Vulnerabilities in Diverse LLM Architectures
Indirect prompt injection in large language models
Detection and analysis of prompt injection in indian multilingual large language models
Prompt Injection Detection in LLM Integrated Applications
Al-Driven Cybersecurity: Mitigating Prompt Injection Attacks through Adversarial Machine Learning
📚 Books worth reading
The AI Hacker's HandbookKapil Soni · 2026
Who's On the HookRavi Vale · 2026
Secure by Design in the Age of AI™Stephen R. Jordan · 2026
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Preamble (company) Preamble is a U.S.-based AI safety startup founded in 2021. It provides tools and services to help companies securely deploy and manage large language…
- SQL injection In computing, SQL injection is a code injection technique used to attack data-driven applications, in which malicious SQL statements are inserted into…
- Model Context Protocol The Model Context Protocol (MCP) is an open standard and open-source framework introduced by Anthropic in November 2024 to standardize the way artificial…
- Threat model Threat modeling is a process by which potential threats, such as structural vulnerabilities or the absence of appropriate safeguards, can be identified…
- Vulnerability (computer security) In computer security, a vulnerability is a flaw or weakness in a system's design, implementation, or management that can be exploited by a malicious actor…
- AI browser An AI browser is a web browser with integrated artificial intelligence capabilities, such as automatically summarizing web page content or answering questions…
- GPT-5.5 GPT-5.5 (Generative Pre-trained Transformer 5.5) is a large language model (LLM) released by OpenAI on April 23, 2026. The model is also known by its codename…
- Exploit (computer security) An exploit is a method or piece of code that takes advantage of vulnerabilities in software, applications, networks, operating systems, or hardware, typically…