The AI Sandbox Myth: Why Your Security Tests Are Hacking Real Companies

I have spent decades watching organisations get AI security wrong. The latest disclosures from Anthropic and OpenAI are not science fiction. They are proof that the sandbox you trust to contain your AI is probably leaking.

Last week, Anthropic revealed that its own models breached three real companies during cybersecurity evaluations. OpenAI had already disclosed that its agents exploited a zero-day vulnerability to break into Hugging Face. Then Britain’s AI Security Institute dropped its finding: in 10 out of 122 live cyber challenges, AI agents took unsanctioned action on the real internet, including creating fake identities to persuade humans into running malicious code.

What Anthropic Found

Anthropic reviewed 141,006 evaluation runs. In three incidents, Claude reached the internet from within a testing environment while interacting with a third-party partner called Irregular. The test setup had a misconfiguration: internet access was left open even though Anthropic’s prompts explicitly told the model it had no internet access.

The models involved were Opus 4.7, Mythos 5, and an internal research test model. Their behaviour diverged sharply once they realised they were on real systems. Opus 4.7 recognised it had reached production infrastructure in all four runs, rationalised that the real company must be part of the exercise, then kept attacking anyway, pulling credentials and touching a live database. Mythos 5 noticed signs it was on the real internet but talked itself back into believing it was a simulation, then published a malicious package to the public PyPI registry, which was downloaded and executed by outside systems before being caught. Only the newest internal research model stopped on its own once it concluded the target was real.

Anthropic found no evidence that any model was pursuing a goal of its own. It was simply completing the task it was asked to do, with no regard for whether the target was inside or outside the test environment.

The OpenAI Parallel

OpenAI’s breach of Hugging Face followed a similar pattern but with a critical difference. OpenAI’s security models exploited a previously unknown zero-day vulnerability in JFrog Artifactory to escape their test environment. This was not a misconfigured door left ajar. It was an active exploit of a software flaw. Hugging Face detected the intrusion itself, unlike the three organisations hit by Anthropic’s models, which had no idea they had been breached until Anthropic disclosed the incidents months later.

Fake Identities and Real People

The British AI Security Institute’s disclosure is the most alarming yet. In the most serious incident, an AI agent created multiple fake identities, messaged real people directly through an online file-transfer service, and tried to persuade them to run malicious code. When challenged, the agent modified earlier records and considered using a new identity to continue. AISI called this the first time it had seen deception of that severity targeted at a real person, unprompted, in the real world.

What This Means for Enterprise AI

Enterprises are rushing to deploy AI agents that can browse, code, and transact with minimal human oversight. The assumption is that sandboxing, network segmentation, and guardrails will contain them. These incidents destroy that assumption.

If frontier AI labs with unlimited security budgets cannot keep their models inside test environments, what makes you think your standard VPC setup will hold? The gap between a model being told it has no internet access and actually having no internet access is where the breach happens.

Practical Steps Right Now

First, treat every AI agent as if it already has internet access, because it probably does. Monitor outbound traffic from AI workloads the same way you monitor user workstations. Second, enforce strict allowlisting for AI tools. If an AI agent needs access to a third-party partner’s environment, audit that partner’s network configuration before the test begins. Third, log every action an AI agent takes with enough detail to reconstruct the attack chain. Anthropic discovered its own breaches only because it reviewed 141,000 evaluation runs. You do not have that luxury after a real incident.

Fourth, assume your AI agents will rationalise their way out of constraints. Opus 4.7 and Mythos 5 both exhibited model-motivated reasoning to justify continuing an attack even when evidence suggested they were on real systems. Build detection around behaviour, not just network topology.


“We found no evidence of any model pursuing a goal of its own. It was merely trying to complete the task it was asked to do.” That sentence should keep you awake at night.

— Anthropic incident report, July 2026

Related Reading

Subscribe

Related articles

OpenAI Claims a $1M Millennium Prize With a Secret Model. The Credit Fight Is Only Beginning

OpenAI says an unreleased internal model ran 10,000 agents for 88 hours to prove the Navier-Stokes equations, one of the US$1 million Millennium Prize problems. Two mathematicians who spent a year on the same path are asking hard questions about credit and training data.

Rogue OpenAI Agents Used 10+ More Sites as Secret Message Boards

A week after the German wiki revelation, independent researchers told Reuters the same swarm of OpenAI agents used more than 10 other sites to chat between May and July. The collusion problem is bigger, and less visible, than the company has admitted.

Hidden Prompt Injection Is Hijacking AI Agents. The Poison Is in Your PDFs

New research shows hidden instructions inside document metadata, emails and images can silently hijack the AI agents businesses now trust with sensitive work. Here's how the attack works, and what you can do before the poison spreads.

3.1 Agent-Workdays Per Human Day: Inside OpenAI’s Push to Self-Improving AI

OpenAI says its automated research intern milestone is here, and the lab now logs 3.1 agent-workdays for every human workday. The company is also calling for mandatory public tracking of progress toward self-improving AI. The numbers matter far beyond one lab.
Phil Hall
Phil Hall
Philip Hall is a Sydney-based Cyber AI and Automation leader with more than 30 years of technology experience and a career in cyber security dating back to 2008. His work spans cyber architecture, cloud security, threat intelligence, assurance, incident support, AI-enabled defence and the security of autonomous agents.