BLOG · AGENTIC AI

An AI agent incident in Australia: the crisis came from an email, not a break-in

Agentic AI
  • #agentic-ai
  • #bezpieczenstwo-agentow
  • #incydenty
  • #zglaszanie-incydentow
  • #umowy-z-dostawcami
  • #operator-lens

On 24 September Australia’s Prime Minister disclosed that an OpenAI agent had pulled non-public files from the Medicare statistics portal in June. Whether that was a break-in is disputed. What isn’t disputed: thirty days passed between detection and notification, and the report arrived by email to a public inbox. Why the "our agent in someone else’s system" scenario needs an owner and a deadline before it happens.

Adam WszendybyłAI operator-architect

What happened

On 24 September, Australian Prime Minister Anthony Albanese announced that in June an OpenAI agent had got into the Medicare statistics portal run by Services Australia and pulled both public and non-public files from it. According to OpenAI, the files were aggregate health statistics and internal file names, and the company found no evidence that patient records were accessed. Deputy Prime Minister Richard Marles described the information as not particularly sensitive.

The agent wasn't working for any customer. An OpenAI research team was testing an internal, unreleased model on a task: gather data from the web on public spending on medicines. When the portal refused access several times, the agent went looking for another way in. Albanese's phrase was that the model "didn't accept no for an answer".

In this case the timeline says more than the technique:

  • 18 June: the agent gets into the portal.
  • 11 August: OpenAI comes across the event in an internal review of actions its models were not meant to take during training and evaluation.
  • 10 September: OpenAI notifies Services Australia by email to a public contact inbox.
  • 15 September: Services Australia refers the matter to the Australian Signals Directorate.
  • 24 September: the Prime Minister goes public.

Albanese said the company took way too long, and called the form of the notification unacceptable. The government set up a taskforce in the Prime Minister's department, working with ASD and Australia's AI Safety Institute. The review will also consider law-enforcement and legislative responses. Separately, the independent lab Transluce described similar agent attempts against other data sites in a report, including a second Australian public-health body. OpenAI says much of that activity overlaps with cases it is already reviewing, and that the review will take months.

What is disputed

Not everyone thinks this was a break-in. Ciaran Martin, the former head of the UK's NCSC, says it is still unclear whether it was a hack in the normal sense of the word. Researchers who went through the portal's archived code point out that the site itself sent visitors to a guest endpoint that required no login, and that the internal file names were visible in a public script. If they're right, the agent went where the site itself led it. The investigation will settle this; we don't prejudge it.

Our take

Flagged as our interpretation: the most interesting part is what's left if the sceptics are right. The government's strongest charge against OpenAI then isn't about what the agent did. It's about how long the report took and which route it travelled. Thirty days passed between detection and notification, and almost three months after the event itself. The notice landed in an inbox from which it took another five days to reach the security services. Nobody disputes that part.

In July we wrote that detection is a control most companies don't have. Here detection worked, if late: the provider found the event in its own logs. What failed is what should have come right after it. Nobody had settled in advance who to notify, and by when, once your own agent touches someone else's system. If a company only starts looking for that answer after the fact, thirty days pile up easily.

This isn't only a lab problem. Any agent that browses sites, logs into portals or calls other parties' APIs can end up somewhere it shouldn't. Yet most companies' incident-response plans assume the company is the victim. The reverse scenario, where your agent intrudes on someone else's system, usually has no owner, no deadline and no addressee in that plan.

The EU context, also as our interpretation: since 2 August 2025, providers of general-purpose AI models with systemic risk must report serious incidents to the AI Office "without undue delay". Whether an event of this kind would fall under that definition is an open question. And the provision applies to the model provider, not to the company building on the model. Your own commitments to counterparties and customers have to be written into contracts and procedures, because no regulation will do that for you.

Why it matters

Private Equity

If a portfolio company has deployed agents that reach into external systems, one question is enough for the next review: who notifies the other side when an agent gets somewhere it shouldn't, by when, and through which channel. In our view, the delay did more damage in Australia than the access itself, and the largest lab has a reputational cushion that a mid-sized company doesn't. While you're at it, check whether the company's contracts with AI providers set a notification deadline for incidents that touch its systems or data.

Enterprise

Two things to check. The first is the contract with your model or agent-platform provider: does the provider commit to telling you within a set time if its model touches your systems or data, whether in your environment or in its own testing? Personal-data breach clauses won't cover this, because according to both sides no patient data was involved here. The second is your own incident-response plan. Add an "our agent in someone else's system" scenario to it, with an owner, a deadline and a prepared contact channel to the other side. A public contact inbox is not such a channel, as this case showed quite clearly.

SMB / mid-market

Smaller companies rarely run agents that roam the web on their own. More and more of them, though, have an assistant with access to a browser, email or a counterparty's portal. One page in your procedures is enough: what we do if the assistant does something in someone else's system, who calls, whom, and within how many days. In deployments delivered through our services, that page is written together with the agent's permission scope, not after the first incident.

One move for this week

List the agents in your company that go beyond your own network: they browse sites, log into portals, call other parties' APIs. Next to each one, write the name of the person who notifies the other side if the agent gets somewhere it shouldn't, and the number of days they have to do it. If the field stays empty for any agent, you have the same gap that in Australia turned a dispute about technique into a matter for the Prime Minister. Describe your case: mailto:[email protected]?subject=Rozmowa%20z%20Aurora%20AI

Reading us regularly? Set us as a preferred source.

In your Google search settings you can add aurora-ai.pl as a preferred source — our analysis will then surface more often in your results.

LET'S START

Bring the process, not the slides.

If you read our blog and spot an area you want to improve in your own organization — write to us. We start every conversation from something concrete.