The Lethal Trifecta of AI Security

A recent piece in The Economist covered Simon Willison’s “lethal trifecta” theory of AI security in depth.

The Blurred Line Between Data and Instructions

To understand the security problem, start from a fundamental property of how large language models work: they cannot truly distinguish data from instructions.

Unlike traditional software, an LLM’s underlying mechanism is text processing — it generates replies by predicting the next word. That means malicious instructions can be planted inside seemingly harmless external content, and the AI system has great difficulty recognizing them. It is like handing someone a letter whose text contains, buried inside it, a suggestion to do something harmful — and the reader cannot tell the letter’s ordinary content from the malicious instruction.

The Three Elements

Simon Willison calls the combination the lethal trifecta of AI security. When the following three elements are present at the same time, they form a security risk that is nearly impossible to defend against:

1. Exposure to External Content

The AI system processes untrusted data from outside: user-uploaded documents, email contents, web pages. Attackers can “poison” this external content, planting covert malicious instructions in it.

2. Access to Private Data

The AI system can reach sensitive private information: source code, passwords, personal email, trade secrets.

3. The Ability to Communicate Externally

The AI system can send information out — by email or by network request. That gives a malicious instruction a channel for shipping stolen private data back to the attacker.

The attack chain runs: the attacker plants an instruction through external content → the AI system executes it and accesses private data → the communication capability leaks the data to the attacker.

Current Defenses

Several defense strategies have been proposed, each with its limitations:

Defense Through Training

Modern AI systems set safety rules through the system prompt — Claude, for instance, is instructed to “recognize red flags” and “avoid harmful responses.” But training is rarely fully reliable: the same malicious instruction can fail 99 times and succeed on the 100th attempt, because the AI’s behavior is probabilistic at its core.

Avoiding the Trifecta

In theory, removing any one of the three elements cuts the risk sharply. In practice, many useful AI applications need all three by nature. An AI email assistant has to process incoming mail (external content), access your mailbox (private data), and send replies (communication capability).

System Isolation

Google has proposed the untrusted-model concept and CaMeL, a dual-LLM design that separates the handling of trusted and untrusted data. The approach works, but it also sharply limits the kinds of tasks the LLM can perform.

Blocking Communication

Cutting off the AI system’s outbound communication sounds simple, but attackers can leak data covertly. A request to what looks like an ordinary website may carry a stolen password embedded in the URL path.

Commercial Reality vs. Security Ideals

While security experts keep sounding the alarm, the AI industry has shown a very different attitude.

Apple chose the cautious route, delaying its promised AI features by more than a year and focusing for now on relatively simple capabilities such as live translation. Yet even a feature as innocuous-sounding as “play that podcast Jamie recommended” in fact creates the trifecta.

Other AI companies have been more aggressive: far from hardening their systems in response to the warnings, they keep launching powerful new tools with the trifecta built in. In a fierce market, AI capability translates directly into market position, and security considerations tend to yield to feature demands.

What Falls to the User

That leaves more of the responsibility with users:

• Understand fully which data an AI system will access
• Recognize the worst case — data leakage — as a real possibility
• Combine AI tools with care: a single application may be safe on its own while the combination produces the trifecta

There is also a privacy risk on the learning side: even with no malice from the AI provider, mishandled data can cause a model to accidentally learn users’ private information, including personal email, business plans, and other sensitive content.

From Deterministic to Probabilistic Thinking

This security challenge may drive a fundamental shift in software engineering philosophy.

Traditional software engineering pursues determinism and assumes everything works as intended. Physical engineering learned long ago to work with tolerances, error rates, and safety margins — to overbuild for the worst case. The probabilistic character of AI may force software engineers toward a similar approach: giving up the attachment to determinism and accepting probabilistic security design.

There is no perfect solution today. As Simon Willison predicts, we have not yet lost millions of dollars to this — but that is only a matter of time. It may take a major financial-loss event before the industry takes these risks seriously.

Closing

The lethal trifecta exposes a basic contradiction: AI’s core value, processing and connecting all kinds of external data, is in inherent tension with its security requirements. Solving it at the root will take systemic change across technical innovation, regulatory frameworks, business models, and engineering philosophy.

Until then, what you can do is work out which data each AI tool you use can reach, and whether combining them assembles the trifecta.

Source: The Economist, “Why AI systems might never be secure”, September 22, 2025
Link: https://www.economist.com/science-and-technology/2025/09/22/why-ai-systems-might-never-be-secure