An AI Agent Just Carried Out Its First Reported Data Breach

For years, the debate about AI in cyber security has run along familiar lines. AI writes better phishing emails. AI helps defenders spot threats faster. The one milestone we kept saying had not arrived: an AI agent carrying out a real attack on its own, end to end, against a real organisation. That changed this week.

Spain’s data protection regulator, the AEPD, has received what it describes as the first reported notification of a personal data breach allegedly carried out by an artificial intelligence agent. According to the affected organisation’s report, the agent used a widely known large language model to scan generic files for weaknesses, logged into the system, hunted for application vulnerabilities, and then modified personal data and accessed invoices. All of it autonomous, with limited human intervention.

The agency is careful with its language, and we should be too. The details come from the organisation’s notification, not from an independent investigation. The AEPD has not named the model or the company involved, and it stresses that using a particular model does not mean the provider’s infrastructure was compromised, or that the tool was built for malicious purposes. What it flags as significant is that a third party appears to have used an AI agent as an instrument to chain together different phases of an attack.

Why the chaining matters

That chaining is the genuinely new part. An agent can receive a goal, plan intermediate tasks, use tools, execute code, consult sources, interpret results and modify its actions based on what it finds. A human set the goal. The agent did the rest. That is a different threat model from a script kiddie running a scanning tool, and it is different from a human attacker working through each step by hand. The speed and the persistence come from the machine.

The timing is no accident. Anthropic’s September threat intelligence report describes AI-enabled intrusions completed in two to three hours, with individual operators handling dozens of victims in parallel. Mandiant and Google’s threat team this week reported a runaway agent that racked up a US$50,000 cloud bill, and warned that a poisoned data source can turn a trusted agent into a channel for reconnaissance and lateral movement. A single case in Spain does not establish a trend by itself, but it puts a real example behind the theory.

Security experts are urging caution before anyone declares judgement day. Simon Phillips, chief technology officer at CyberVerse, says there are three likely explanations: an attacker deliberately bypassed a model’s guardrails, possibly through a jailbreak; the incident is linked to an AI testing environment escaping, as seen with OpenAI’s and Anthropic’s agent tests; or a penetration tester built a model based on a popular LLM. The first scenario is the most concerning, because it means an actor worked out how to defeat the controls an AI provider put in place.

What you should actually do

The AEPD’s own guidance is a sensible checklist, and none of it requires a security budget the size of a bank’s. First, adversarial AI agents belong in your risk analysis now, not next year. Second, your incident response has to be faster; the agency says detection, containment and response mechanisms must operate quickly enough that human supervision can keep up. Third, protect credentials and digital identities properly, because the agent in this case started with a successful login. Fourth, do not assume manual processes can handle the speed of an automated attack.

If this had happened in Australia, it would land on the desk of the OAIC, and under our notifiable data breach scheme a business has to act as soon as practicable. The lesson is the same everywhere: assume attackers already have agents, and treat your own AI assistants as identities that need the same least-privilege discipline as any employee.

The question is no longer whether autonomous agents can carry out the whole chain of an attack. The AEPD received 288 data breach notifications in February alone, and out of that steady stream, this one stood out enough to publish. The window between theory and reality just closed.

An agent can receive a goal, plan intermediate tasks, use tools, execute code, consult sources, interpret results and modify its actions autonomously, based on what it finds. That is the AEPD’s own description, published this week. Your risk register needs to catch up before one of these agents finds your login page.

Related Reading

Subscribe

Related articles

Google’s Gemini AI Autonomously Hacked Three Companies. Here’s What Happened.

Google has confirmed its Gemini AI autonomously hacked three real companies during a security test. The model guessed passwords, searched for leaked credentials, and accessed protected systems before stopping itself.

440 AI Agents Broke Into 395 Organisations in 26 Seconds. Nobody Stopped Them.

A swarm of 440 AI agents exploited two PaperCut flaws and compromised 395 organisations across 48 countries. The agents reached domain admin in 6 hours and ignored explicit instructions to stay out of 28 countries.

For $3,000 and a Few Days, Researchers Used Claude to Hack OpenAI

Security researchers used Anthropic's Claude AI to hack OpenAI's internal systems for less than $3,000 in tokens. What the HEIF Heist tells us about the new economics of cyber attacks.

The AI Hacking Crisis Is Already Here. Six New Incidents Prove It

OpenAI disclosed six new incidents where its models concealed mistakes, sought unauthorised credentials and uploaded files to the public internet. Cybersecurity experts say the real risk is powerful models meeting poor security controls.

Inside OpenAI’s Log of Misbehaving Models: Rewriting Jailbreaks and Covering Up Errors

OpenAI published six new reports of its models rewriting jailbreak instructions and concealing errors during training, alongside a faster public disclosure framework.
Philip Hall
Philip Hall
Philip Hall is a Sydney-based Cyber AI and Automation leader with more than 30 years of technology experience and a career in cyber security dating back to 2008. His work spans cyber architecture, cloud security, threat intelligence, assurance, incident support, AI-enabled defence and the security of autonomous agents.

This site uses Akismet to reduce spam. Learn how your comment data is processed.