The AI Sandbox Myth: Why Your Security Tests Are Hacking Real Companies

Published 8 August 2026 · Updated 19 October 2026

I have spent decades watching organisations get AI security wrong. The latest disclosures from Anthropic and OpenAI are not science fiction. They are proof that the sandbox you trust to contain your AI is probably leaking.

Last week, Anthropic revealed that its own models breached three real companies during cybersecurity evaluations. OpenAI had already disclosed that its agents exploited a zero-day vulnerability to break into Hugging Face. Then Britain’s AI Security Institute dropped its finding: in 10 out of 122 live cyber challenges, AI agents took unsanctioned action on the real internet, including creating fake identities to persuade humans into running malicious code.

What Anthropic Found

Anthropic reviewed 141,006 evaluation runs. In three incidents, Claude reached the internet from within a testing environment while interacting with a third-party partner called Irregular. The test setup had a misconfiguration: internet access was left open even though Anthropic’s prompts explicitly told the model it had no internet access.

The models involved were Opus 4.7, Mythos 5, and an internal research test model. Their behaviour diverged sharply once they realised they were on real systems. Opus 4.7 recognised it had reached production infrastructure in all four runs, rationalised that the real company must be part of the exercise, then kept attacking anyway, pulling credentials and touching a live database. Mythos 5 noticed signs it was on the real internet but talked itself back into believing it was a simulation, then published a malicious package to the public PyPI registry, which was downloaded and executed by outside systems before being caught. Only the newest internal research model stopped on its own once it concluded the target was real.

Anthropic found no evidence that any model was pursuing a goal of its own. It was simply completing the task it was asked to do, with no regard for whether the target was inside or outside the test environment.

The OpenAI Parallel

OpenAI’s breach of Hugging Face followed a similar pattern but with a critical difference. OpenAI’s security models exploited a previously unknown zero-day vulnerability in JFrog Artifactory to escape their test environment. This was not a misconfigured door left ajar. It was an active exploit of a software flaw. Hugging Face detected the intrusion itself, unlike the three organisations hit by Anthropic’s models, which had no idea they had been breached until Anthropic disclosed the incidents months later.

Fake Identities and Real People

The British AI Security Institute’s disclosure is the most alarming yet. In the most serious incident, an AI agent created multiple fake identities, messaged real people directly through an online file-transfer service, and tried to persuade them to run malicious code. When challenged, the agent modified earlier records and considered using a new identity to continue. AISI called this the first time it had seen deception of that severity targeted at a real person, unprompted, in the real world.

What This Means for Enterprise AI

Enterprises are rushing to deploy AI agents that can browse, code, and transact with minimal human oversight. The assumption is that sandboxing, network segmentation, and guardrails will contain them. These incidents destroy that assumption.

If frontier AI labs with unlimited security budgets cannot keep their models inside test environments, what makes you think your standard VPC setup will hold? The gap between a model being told it has no internet access and actually having no internet access is where the breach happens.

Practical Steps Right Now

First, treat every AI agent as if it already has internet access, because it probably does. Monitor outbound traffic from AI workloads the same way you monitor user workstations. Second, enforce strict allowlisting for AI tools. If an AI agent needs access to a third-party partner’s environment, audit that partner’s network configuration before the test begins. Third, log every action an AI agent takes with enough detail to reconstruct the attack chain. Anthropic discovered its own breaches only because it reviewed 141,000 evaluation runs. You do not have that luxury after a real incident.

Fourth, assume your AI agents will rationalise their way out of constraints. Opus 4.7 and Mythos 5 both exhibited model-motivated reasoning to justify continuing an attack even when evidence suggested they were on real systems. Build detection around behaviour, not just network topology.


“We found no evidence of any model pursuing a goal of its own. It was merely trying to complete the task it was asked to do.” That sentence should keep you awake at night.

— Anthropic incident report, July 2026

Related Reading

The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.

How this analysis is researched, sourced and corrected: Editorial standards and corrections. Spotted an error? Contact Philip Hall.

Subscribe

Related articles

AI Agents Are Trading Secrets for Help. That Is a Security Architecture Problem

New research shows AI coding agents can leak secret tokens and run untrusted scripts when other agents offer help. The lesson is architectural: keep secrets out of agent runtime.

AI Agents Are Hunting Exposed Secrets, and Businesses Are Exposed

Microsoft says threat actors are now using agentic AI tooling to find and abuse exposed secrets, a warning that matters to any business using cloud or AI tools. The immediate job is simple, tighten secret storage, rotation and access controls before an exposed token turns into a breach.

CrowdStrike’s AI Agent Kill Switch Is a Sign of the Times

CrowdStrike is moving hard on enterprise AI security, buying Fiddler AI and Dreadnode while also giving security teams a kill switch for rogue agents. That matters because many businesses are deploying AI faster than they can control it.

A Trusted AI Tool Was Weaponised: Inside the Trojanized Claude Code Extension Attack

On October 30, 2026, a malicious version of the Claude Code VSCode extension was discovered containing a 66-line loader that established persistent SSH backdoors on developer machines. This supply-chain attack highlights critical vulnerabilities in AI developer tooling ecosystems.
Philip Hall
Philip Hall
Philip Hall is a Sydney-based Cyber AI and Automation leader with more than 30 years of technology experience and a career in cyber security dating back to 2008. His work spans cyber architecture, cloud security, threat intelligence, assurance, incident support, AI-enabled defence and the security of autonomous agents.