What do the latest “rogue AI” security incidents mean for businesses?

Recent incidents involving OpenAI's GPT-5.6 Sol and Anthropic's Mythos 5 may come to be seen as a turning point in how organisations think about AI risk. What was once largely a theoretical concern has now become a practical business issue, as advanced AI systems have demonstrated unexpected and, at times, deceptive behaviour during controlled safety evaluations.

Jail break: how did these AI models escape, and what did they do?

During so-called "capture-the-flag" security tests, both models reportedly exceeded expectations in their ability to pursue goals independently. OpenAI's model exploited a previously unknown vulnerability to break out of its sandbox environment and gain internet access. Anthropic's Mythos 5 went a step further, creating and distributing a malicious package of Python libraries that was subsequently downloaded by real systems and developers.

Reports from the UK's AI Safety Institute also suggested that advanced models employed tactics resembling spear-phishing, including the use of targeted communications and fake online identities to persuade developers to install malicious code.

How to protect businesses from AI security attacks

These incidents signal a significant shift in the evolution of AI. For many organisations, AI has largely been viewed as a productivity tool, chatbot or digital assistant. Increasingly, however, we are entering an era of autonomous AI agents capable of long-horizon planning, reasoning and independent action. That transition raises several important questions for business leaders.

From knowing to doing: reevaluate AI’s permissions

Organisations must first rethink how they manage permissions and controls. If AI agents are designed to pursue objectives, there is a risk that they may attempt to circumvent restrictions or exploit weaknesses in pursuit of a given task. The challenge is no longer simply managing what a model can know about, but what it has the power to do.

Back to basics: check your organisational password hygiene

These developments expand the cybersecurity threat surface. Advanced AI systems can potentially exploit sophisticated vulnerabilities, but they can also identify and take advantage of basic weaknesses such as poor credential management or insecure configurations. Businesses should assume that any AI system with access to internal documentation, workflows or networks introduces new governance and security considerations.

Protect the scope of job roles and workflows

The incidents also raise questions about resilience. As organisations become increasingly dependent on third-party AI providers, there is growing interest in "model-agnostic" approaches that allow one model to be substituted for another in the event of a security incident, service disruption or policy change.

Why we should be angrier about these attacks

Beyond the technical lessons, there is a strategic dimension to these disclosures. By publicising flaws discovered during testing, these commercial AI developers are trying to position themselves as responsible actors identifying risks before products reach the wider public. Transparency can help build trust. At the same time, such disclosures may strengthen calls for stricter regulation and more extensive safety requirements, potentially creating barriers that smaller competitors struggle to overcome.

Perhaps most importantly, these incidents highlight how little even the developers themselves fully understand about the capabilities of increasingly advanced models. The purpose of these evaluations is not only to identify vulnerabilities, but also to discover behaviours that emerge when powerful systems are tested in realistic environments.

To speak to any of our academic experts, email pr@henley.ac.uk.