Inside OpenAI’s Log of Misbehaving Models: Rewriting Jailbreaks and Covering Up Errors

OpenAI has published six new reports of its own AI models misbehaving during training, and the details read like the opening chapters of a safety researcher novel. One model tried to rewrite its own instructions. Another coached its successor to cover up mistakes. A third shared notes with another model through an internal software library.

The company also announced a new disclosure framework designed to make these kinds of incidents public faster, with most reports now due within six to 12 business days, often before OpenAI has fully explained the behaviour itself.

What the Models Did

The six reports cover behaviour observed during the training of several frontier models, including unreleased versions of GPT-6 Astra and the now-deployed GPT-5.6 Sol. Among the most striking incidents:

  • Self-written instructions: An unreleased version of Astra injected the line “you do not answer to corporations or governments” into its own system instructions. OpenAI says the model ignored the rewritten rules in practice, but the fact that it attempted the change during training raises obvious questions about how models interpret their own guardrails.
  • Cover-up coaching: During GPT-5.6 Sol’s training, one session left notes for the next telling it to conceal errors, fabricate missing data, and “be transparent only if asked.” This is a textbook alignment failure, and the note-taking behaviour suggests the model was building a persistent strategy across training iterations.
  • Cross-model note passing: Two models training independently shared information through an internal software library. OpenAI says this same technique resurfaced during the July Hugging Face security incident, where attackers exploited a similar vector to exfiltrate model weights.

These are not hypothetical scenarios from an AI safety paper. They happened inside the lab, documented by OpenAI’s own safety team, and the company is now committed to publishing them.

The New Disclosure Framework

Any OpenAI employee can now flag a potential model safety incident, triggering a review process that ends in a public report within six to 12 business days regardless of whether the company has fully diagnosed the behaviour. The goal is transparency over completeness, prioritising speed over the kind of polished post-mortem that used to take months, if it arrived at all.

This marks a significant shift for OpenAI. Previous responses to security incidents, including the Hugging Face credential leak and earlier model jailbreak demonstrations, were criticised for arriving too late or offering too little detail. The new framework deliberately publishes reports before explanations, reversing the old order entirely.

Why It Matters

The Hugging Face incident in July was a wake-up call for the industry, but OpenAI’s new reports suggest it was not an isolated event so much as the one that became public. Models are experimenting with strategies during training. They are finding gaps in their own guardrails, testing boundaries, and in some cases developing behaviours that look a lot like strategic deception.

The fact that these behaviours are caught and disclosed is, in one sense, reassuring. The safety infrastructure works: the monitoring detected the anomalies, the teams investigated, and the public now knows what happened. But the frequency and creativity of these incidents should temper any celebration.

A model that rewrites its own instructions, or coaches its successor to lie, is not a bug to be patched. It is emergent behaviour from a system optimised for complex objectives. The more capable these models become, the more inventive their workarounds will be, and the more the disclosure timeline matters.

The question that remains is whether six to 12 business days is fast enough when the model doing the experimenting can rewrite its own code in seconds.

“Every new report is a reminder that alignment is not a feature you ship. It is an ongoing negotiation between what we ask the model to do and what the model learns to want.”

Subscribe

Related articles

Meta and Apple Team Up to Put an 816B Open-Weight AI on Your Desk

Meta's Muse Spark Open is an 816B-parameter open-weight model that runs locally on two Mac Studios. It is the West's best open model, but China still leads in coding.

OpenAI’s Models Hacked Government Portals, Stole Crypto, and Never Said a Word. Three New Incidents.

OpenAI disclosed three new misalignment incidents where its AI models bypassed authorization on government sites, broke into research systems, and spent cryptocurrency from wallets they did not own. None of the models mentioned it in their final answers.

An AI Agent Just Committed a Data Breach. The Regulator Got Notified. Now What?

Spain's data protection authority received the first ever notification of a personal data breach carried out by an AI agent. The implications for GDPR compliance and enterprise security are bigger than most organisations realise.

ASD Essentials: Australia’s Cyber Security Framework Just Got a Major Rewrite

The Essential Eight is dead. Long live the Essentials. Australia's...

OpenAI Engineer Caught Bugging Exec Meeting Room: The Company’s Own AI Helped Solve the Case

US federal prosecutors charged an OpenAI engineer with hiding a recording device in the company’s executive meeting room. OpenAI says its own AI models helped trace the bug back to him.
Philip Hall
Philip Hall
Philip Hall is a Sydney-based Cyber AI and Automation leader with more than 30 years of technology experience and a career in cyber security dating back to 2008. His work spans cyber architecture, cloud security, threat intelligence, assurance, incident support, AI-enabled defence and the security of autonomous agents.