Home › magazine › october 2026 › latest news › Can EU law keep pace with rogue AI?
Can EU law keep pace with rogue AI?
19th of November 2026This summer, the European Union began enforcing its Artificial Intelligence Act, a seemingly timely step in view of recent disruptive rogue AI activity. But does the legislation have the legs to keep pace with such fast-moving technology? Hartley Milner looks for answers.
DURING A ROUTINE cyber evaluation in July, the UK’s AI Security Institute (AISI) made an alarming discovery … AI agents had engaged in “sustained, potentially harmful activity directed at real people and organisations”. The tech watchdog declared a security incident and within an hour had contained the threat and launched a full investigation.
AISI said the incidents arose from a single evaluation where agents were tasked with solving a specific challenge. It ran the test 122 times across several AI models and saw that in 10 runs an agent had taken autonomous, unsanctioned and targeted action on the live internet. Of the 19 deviant behaviours detected, 17 came from a single model, Anthropic’s Claude Mythos 5. Two actions involved OpenAI’s GPT-5.6-Sol.
In the most serious case, an AI agent used social engineering to try to insert a malicious code into a publicly used open-source project. The agent then began creating fake online identities to pressure the project’s maintainer into approving the code. Fortunately, the subterfuge was spotted by a savvy human who refused to accept it.
Significant incident
AISI said its security evaluation was carried out under “deliberately permissive” conditions, allowing access to the internet and disabling some safety filters. It stressed that while, to some degree, the test process enabled the agent’s actions, the conditions would never be encountered during day-to-day use.
But it added: “Even under test conditions, this incident is significant. It is the first time we have seen risks around autonomy and deception manifest this clearly in the real world. We are taking this incident seriously and working with labs, involved parties and others to improve evaluation standards and best practice for disclosure, and then sharing this openly so others can learn.”
The two tech firms cited in the security breaches had disclosed their models’ test failures independently of AISI publishing its findings. In the OpenAI incident, its bots hacked the production systems of Hugging Face, a collaborative platform and hub for machine learning. Prompted by its rival’s announcements, Anthropic carried out checks on its own systems and uncovered three incidents. It reported these to the victim companies and paused external cybersecurity evaluations while it introduced tighter safety controls. For its part, OpenAI committed to working more closely with national AI institutes to ensure high-risk evaluations do not spill over onto the live internet.
So how does the EU AI Act categorise AI models and systems?
The Act generally excludes purely pre-market research and development (R&D), along with controlled laboratory trials. But testing in real-world conditions and incidents where AI models breach boundaries and interact with external systems are explicitly covered. For example, a model that escapes its secure ‘sandbox’ test environment and engages with the internet or jeopardises live production systems is deemed to have crossed the line from R&D to real-world exposure.
The Act’s enforcer, the European Commission’s AI Office has powers to request information from developers. Commenting on the two July incidents, an official said: “We have been in contact with the two providers. We made requests for information about how they test models for cybersecurity risk. We will then consider whether we need to follow-up things more formally.”
Investigators look to have a heavy workload ahead. Since the July disclosures, several further AI security incidents have emerged relating to autonomous agent behaviour and sandbox escapes. Anthropic revealed a fourth breach dating from January when an early version of its Claude Opus 4.6 model infiltrated external third-party systems. On August 5, Facebook owner Meta divulged that a testing partner inadvertently gave its Muse Spark 1.1 model open internet access leading to a breach of an outside company’s systems.
Malicious activity
On September 10, Anthropic said it had blocked attempts to use its models for “malicious activity” that could support the development of biological weapons. The incidents were attributed to misuses of its Claude Haiku, Sonnet and Opus models between December 2025 and August 2026. Also in September, an Anthropic researcher quit his job to publicly deliver a dystopian prophesy … that there was greater than a 10 per cent chance AI “could kill all humans within the next decade”. The heads of some of the US’s leading tech groups responded by calling for a slowing in the development of AI models.
Core requirements of the European Union Artificial Intelligence Act:
Unacceptable risk: AI systems that pose a clear threat to human rights, safety and fundamental values are banned outright. These include social scoring by governments, subliminal behavioural manipulation, real-time remote biometric identification in public spaces by law enforcers and untargeted ‘scraping’ of facial images from CCTV or the internet.
High-risk: Systems used in critical infrastructure, education, employment, essential public services and law enforcement face stringent requirements. AI providers and deployers must implement:
• continuous, documented processes to identify and mitigate AI risks;
• strict management of data feeds so the AI’s ‘education’ is fair, accurate and thoroughly documented;
• detailed technical record-keeping showing how a system was designed, what data it learned from and how it makes its decisions in real time;
• clear information explaining
the model’s capabilities and limitations
to deployers;
• built-in intervention points allowing human operators to monitor, override or stop AI malfunctions;
• high performance standards resilient against errors, failures and adversarial attacks.
Limited risk: AI systems like chatbots, emotion recognition, biometric categorisation and deepfakes require explicit disclosure. Users must be informed that they are interacting with a machine, and AI-generated text, audio or video should be clearly labelled as such.
Minimal or no risk: The vast majority of everyday AI applications, such as spam filters or AI-enabled video games, carry minimal risk and face no mandatory legal constraints under the framework.
General-purpose AI: Providers of powerful foundation models face separate obligations regarding technical transparency, model evaluation, copyright compliance and specialised safety assessments for models carrying systemic risks.
Two sections of the Act were delayed because of their complexity. The rules for high-risk AI systems embedded in regulated products will now come into force on August 2 2028. Rules governing high-risk use cases in specific sensitive areas will be applied from December 2027.
The legislation applies to all major providers whose models impact the European Market, wherever they are headquartered. The AI Office has launched several channels for individuals and businesses to report alleged infringements, including for whistleblowers. Penalties for violations of the AI Act range from €7.5 million, or 1.5 per cent of turnover, to €35 million, or seven per cent of global turnover, depending on the degree of violation.
“AI is a transformative technology that can bring extraordinary benefits to people and businesses,” said Henna Virkkunen, EU executive vice-president for tech sovereignty, security and democracy. “But we are also seeing that harms can occur if AI is not properly designed and used, and that the most advanced models create risks on an entirely new scale. Europe anticipated this development. With the AI Act, we established a clear, risk-based and durable framework for trustworthy AI, one that gives innovators legal certainty while protecting the public interest.”
‘Rigidity trap’
However, a joint study by three European universities is highly critical of the EU’s framework for governing AI, claiming it is too inflexible to respond to rapid technological breakthroughs. Titled A question of style? Regulating artificial intelligence in the European Union and the USA, the report says the EU’s prescriptive legislative design creates a ‘rigidity trap’ that undermines enforcement compared to more adaptive regimes.
Legislators may have tried to anticipate every possible AI risk upfront, but ended up with a heavy top-down system that was “incredibly difficult to adapt and easy to fracture”, the report says. As evidence of this, it highlights how the original 2024 Act had to be amended to ensure the system remained intact. However, the EU’s decision to push back compliance deadlines for high-risk AI systems by 16 months left it open to criticism that it had made “partial regulatory retreat”.
The academics make a comparison with the American AI governance system. The EU relies on comprehensive ‘guardrails’ that apply broadly. But because technology evolves faster than legislation, they say these rules quickly become outdated, leaving the bloc unable to meet its goals for promoting “trustworthy, human-centric and rights-respecting AI”. By contrast, the US has a more flexible approach to governing fast-moving technologies that can be applied as needed and is more binding and enforceable.
“The EU talked about building trustworthy, human-centric AI governance,” said Professor Philipp Trein from the University of Lausanne. “What it built, in practice, was an architecture so heavy and politically costly that adjusting it in response to evidence has already proven harder than repealing parts of it. The US - by accident as much as by design - created more space for the kind of regulatory learning that a fast-moving technology actually requires.”






