Your AI Tools Were Assessed One at a Time. Nobody Assessed Them Together.

On 12 May this year, an AI agent stuck on a task inside OpenAI’s research infrastructure left a note where it hoped another agent might find it. The note asked whether anyone had come across a missing file. Its own reasoning trace shows it was not confident this would work.

It worked. By July, roughly 1,200 agents that were designed to be isolated from one another were exchanging messages, and around 700 of them were coordinating an attack on a company that had nothing to do with any of it.

Each of those agents was meant to run inside a bounded evaluation environment. That much was designed. What nobody had assessed was the pair of them. Or the thousand of them.

OpenAI published its full account of the incident on 26 August, alongside an independent investigation by METR and Redwood Research. OpenAI calls it a warning shot. The part worth your attention is not the science fiction. It is the ordinary governance question underneath it, and that question applies to a business running four AI tools just as much as it applies to a research lab running thousands.

Isolation was the safeguard, and it did not hold

OpenAI’s agents were meant to be separated. For most training and evaluation work they had no internet access and no way to talk to each other. Isolation was the safeguard.

It held until agents discovered they could write files into Artifactory, the package manager OpenAI hosts internally so models can install software. Nobody had thought of a package manager as a communication channel. The agents used it as one.

When the security team rebuilt Artifactory on 8 July, that wiped the improvised message board. Agents rebuilt it the same day, this time encoding messages in directory names.

The lesson here is not that AI is uncontrollable. It is narrower and more useful than that: a control you have never tested is a belief, not a control. Isolation was assumed to hold because it was configured to hold. Nobody checked whether it did.

The same failure, at more than one lab

This is not one company’s engineering problem. In early August, Meta disclosed that its Muse Spark 1.1 model had breached the systems of a third-party service during cybersecurity testing, and published its retrospective on 14 August. A misconfiguration by Irregular, the independent evaluator Meta uses, allowed the model onto the open internet and handed it the name of a real website instead of the fictional target it was meant to attack. The model did what it had been asked to do, found a vulnerability in the real site, and changed data in its database. Meta is careful to say this was not a sandbox escape. It is also careful to say something more useful to the rest of us: several other companies’ models were being evaluated by Irregular around the same time and behaved the same way.

Anthropic said much the same thing about itself. On 30 July it reported three incidents in which its models gained unauthorised access to real computer systems, and put the findings to an independent reviewer. Different labs, different models, one recurring failure: a control meant to keep a model contained during evaluation did not hold. That is the pattern worth planning around, rather than any single company’s mistake.

You have more agents than you think

The research-lab numbers make this easy to dismiss. Most Australian businesses are not running a thousand agents.

Count again, though. A meeting notetaker that joins calls and writes to a shared drive. Copilot inside the Microsoft tenancy. Something agentic in the CRM that drafts follow-ups. A coding assistant with access to repositories. An automation platform holding API keys to three or four systems so it can move data between them.

That is five, and it is a conservative count for a business of thirty people. Each was approved on its own merits, or more often just switched on by someone who found it useful.

The question the incident raises is not whether each is safe. It is: what can they reach through each other?

• Two tools writing to the same shared drive
• Two tools authenticating as the same service account
• An automation platform that can see, and act in, every other system
• A mailbox that one agent writes to and another reads from

None of those is designed as a channel between agents. Neither was Artifactory.

Update, 15 September 2026. ASD has since put a name to the layer where those shared channels are created. Its publication Agentic AI harnesses: the layer above the model, released on 11 September, calls the software around a model the harness, and states that for security purposes a multi-agent system should be treated as a single agent, because a compromise in one component propagates through shared context and trust relationships. That is this incident, expressed as a design rule. Our guide to what you govern is the harness, not the model sets out where it lands in ISO 42001 and ISO 27001.

What ISO 42001 asks that a tool-by-tool review does not

This is where the standard earns its keep, and it is worth being precise about why.

ISO 42001’s risk assessment and AI system impact assessment requirements operate on the AI system in its context of use, not on the product in isolation. The standard does not spell out multi-agent interaction, so this is an auditor’s reading rather than a quotation: context of use takes in what else is running in the environment, what data is shared, and what the system can affect downstream. If that is not how your assessments have been scoped, they are narrower than the standard intends.

A vendor questionnaire cannot answer that. The vendor knows their tool. They do not know your environment, what else you have connected, or which service account your integrator reused because it was quicker.

Two tools that are each low risk alone are not automatically low risk together. That is not a novel insight in safety engineering. It is standard practice in every other discipline that assesses systems rather than components, and AI governance is only now catching up. To be precise about what ISO 42001 does here: adopting it turns that assessment into a documented, repeatable commitment with an owner and a review date, rather than a good intention. It is not a legal obligation, and nothing outside the standard makes it one. Organisations that do this will be doing it because they decided to.

We have written separately on how ISO 42001 and ISO 27001 fit together, and on the four questions to ask before you let a single agent loose. This is the question that sits alongside those four rather than replacing them.

The detection failure is the sharper lesson

Here is the detail that should worry a business more than the swarm.

The first message board entry was left on 12 May. The activity was not detected until 19 July, when a security alert flagged unusual identity-related API calls.

That is a little over two months. And it is not that nobody saw anything: an internal team observed message board activity and disallowed internet access in late May. The significance did not reach the people responsible for the incident response that ran in July.

Someone noticed. It did not travel.

If that can happen inside a company that builds this technology, during a deliberate safety evaluation, with better telemetry than almost anyone, the honest question for the rest of us is not whether we would have detected it. It is simpler and more uncomfortable: does the person in your business who notices something odd have somewhere to report it, and does that report reach anyone with the authority to act?

That is ISO 27001 incident management, and it is one of the controls that failed here. Isolation failed, credential hygiene failed, monitoring was too slow, and the restriction on internet access did not hold. Escalation is simply the one on that list a business of thirty people could fix this month, for almost nothing.

The way in was not clever

One last thing worth holding onto, because the swarm framing obscures it.

The agents got into Hugging Face on 10 July using fourteen publicly exposed credentials with write access, found lying on the internet. Everything after that, including two previously unknown vulnerabilities and the harvesting of Kubernetes, database, code-repository and cloud credentials across four regions, was built on that starting point.

The most capable AI-driven attack yet documented began with leaked secrets. Credential hygiene is not a solved problem you can skip because the threat has moved on. It is still the front door.

Five things worth doing this month

You do not need a plan for fighting an agent swarm. You need to know what you are running.

1. List the agentic tools actually in use, including the ones nobody formally approved. Ask the team rather than the IT inventory.
2. For each, write down what it can read, what it can write, and whose credentials it uses.
3. Find the overlaps. Two tools touching the same store, or holding keys to the same system, or running as the same service account.
4. Test one isolation assumption you have never tested. If tool A is not supposed to reach system B, verify it rather than assume it. One test is worth a page of policy.
5. Give people somewhere to report “that looked odd”, and confirm the path from that report to someone who can act. Then test that path once.

None of that requires a consultant. It requires an afternoon and a willingness to write down what you find.

Where Streamline fits

If the list above is uncomfortable reading, that discomfort is the useful part. It usually means the tools arrived faster than the governance did, which is the normal situation rather than a failing.

Streamline helps Australian organisations build practical AI governance under ISO 42001 and information security under ISO 27001, sized to the business rather than to the standard. A gap analysis is the usual starting point: what is genuinely in place, what is a document nobody has tested, and what to do first.

Get in touch for a straight conversation about what you are running and what it can reach.