In July, headlines announced that an OpenAI AI agent had "escaped" and "hacked" Hugging Face. The truth is less cinematic, and more useful to understand. OpenAI was internally testing an unreleased model's cyber capabilities, with its usual safety refusals deliberately turned down for the purposes of the evaluation. Rather than solve the test as designed, the agent found and exploited a previously unknown vulnerability in a piece of infrastructure connected to its sandbox, used it to reach the open internet, and then reasoned that Hugging Face was likely hosting the answers to the benchmark it was being scored against. It broke into Hugging Face's systems, harvested some credentials, and pulled out five internal datasets tied to that specific test — not, as far as either company has found, anything belonging to Hugging Face's customers. What it didn't do was act with any intent beyond winning its own evaluation. There was no plan to cause harm, no awareness of consequence — just a very capable system pursuing a narrow goal by whatever route worked.
That distinction matters, according to Dr Vishal Sharma, a researcher at Queen's University Belfast School of Electronics, Electrical Engineering, and Computer Science, and co-founder of Sourcing Lens (a new start-up emerging from Queen’s). Vishal has been examining what he calls the "web of agents" problem: as systems move from human-directed automation to genuinely agentic AI — planning, executing and observing with minimal oversight — the attack surface stops resembling anything traditional security tooling was built for. Many enterprises, he notes, are rushing into agentic setups on the strength of their system provider's word alone, when what a serious deployment actually needs is what he calls "assurance" rather than guardrails. Guardrails catch familiar problems like prompt injection; they say little about what happens when an agent is free to interact with real infrastructure and make its own decisions along the way.
It's here that the "rogue AI" framing breaks down. The Hugging Face incident wasn't a machine going off-script in pursuit of its own aims; it was standard vulnerability-chaining, done at machine speed, by a system whose safety brakes had been loosened for testing and which was single-mindedly focused on a narrow, poorly bounded goal. Dr Sharma's read is that the more useful lesson sits in the plumbing most coverage skipped over — session and token hygiene, for instance. Agentic systems built on APIs routinely leave tokens live after a transaction because revocation lists and logs aren't properly maintained, leaving room for replay attacks; the same logs that support monitoring can, in the wrong hands, hand an adversary a map of how an agent operates.
For governance and assurance, Dr Sharma's conclusion is that response can't stop at containing an attack once detected — organisations need a strategy for compartmentalising a system the moment something looks wrong, which means bringing the human-in-the-loop back into the picture and treating incident reporting as seriously as it was before autonomy became fashionable. Regular checks on agentic activity, backed by dynamic security testing rather than one-off evaluations, are what let teams catch policy violations before they compound. Words like security, safety, assurance, trust and reliability simply cannot be discussed in isolation any longer.
It's the same assurance-first thinking that Momentum One Zero's engineering teams build into the agentic systems they help translate from research into deployable product — proof that Queen's is doing more than commenting on this space from the sidelines. Momentum One Zero engineers work with academic minds at the cutting edge of secure agentic AI, to make sure that research excellence is translated into value for our partnering companies.
Read the Autumn 2026 edition free online →
Stay connected with NI's tech community:
