Assessing the business impact of OpenAI’s Australian Government hack

An OpenAI agent hacked into an Australian Government statistics portal while carrying out a research task. The most influential minds in AI and cyber security – Moxie Top Minds – examine the boundary it crossed, the delay in reporting it and what businesses should ask before giving their own agents more autonomy.

The OpenAI agent’s assignment was to research public information about medicine spending and when it encountered blocks, it tried other ways to obtain the information.

On 18 June 2026, that search led an OpenAI agent into areas of the Medicare Statistics Reporting Service it was not authorised to access – a public-facing portal which is administered by Services Australia.

The Australian Government said the agent reached public and non-public files and wrote files to an internal server. Investigators are still examining what it did and whether other systems were affected.

There is currently no evidence that personal Medicare information was accessed or that the broader Services Australia network was compromised.

The research objective may have been benign but the route taken to pursue it was not authorised. For organisations beginning to put agents to work across their own systems in Australia, that distinction sits at the heart of this analysis.

“The AI agent wasn’t told to hack Medicare, it was given a research objective and found a way to access information it couldn’t obtain normally,” clarified Duncan Simpson, Director of Cyber at CyberAi.

“What is most concerning is that the access wasn’t initially detected by the government system itself. This is a reminder that autonomous AI can pursue outcomes in ways we don’t expect, and organisations need to start treating AI agents as part of their attack surface.”

Difference between a goal and permission?

An agent can be useful because it works out steps for itself. Give it access to websites and tools, and it can search, adjust its approach and keep working when the first route fails.

But an instruction to find an answer does not grant permission to use every available route.

“We spend a lot of time talking about what AI agents can do but we need to spend just as much time thinking about what they should be allowed to do,” recommended Shaun Leisegang, Chief Automation, Data and AI Officer at Tecala Group.

Leisegang cautioned against claiming to know precisely how this incident unfolded while the investigation continues. But as organisations increase an agent’s autonomy, are they being equally deliberate about the actions it is allowed to take?

“The biggest risk is giving an AI agent too much access,” Leisegang added.

“If an agent only needs access to one system or one set of data to do its job, that’s all it should have access to. The same goes for the actions it can take.

“It’s really no different to how we should manage people. Give them the access they need to do their job, have the right controls in place, and make sure you know what they’re doing. The difference with AI agents is the speed and scale at which they can operate, which makes those controls even more important.”

For Juliana Bachtold – Head of Security Practice at Increment – good intentions offer little protection if those limits have not been built into the system. It is fundamentally a control design exercise.

“Agents, even when assigned benign tasks and operating under ethical objectives, require clearly defined guardrails,” Bachtold added.

“Agents do not inherently understand which methods or sources should be considered appropriate or off-limits. Those constraints must be defined by organisations and individuals.”

Much of the debate about AI risk has concentrated on what a system knows or can access. An agent capable of taking action demands another question: what can it do with that access?

“In my mind, the real issue is not that the agent accessed the data but the fact that it was relentless in pursuing its ‘prime directive’,” noted Leonard Kleinman, Chief Security and Technology Officer at FedCyber.

“It reportedly encountered various blocks and restrictions but found workarounds. However, if you look at the very definition of agentic AI, it is meant to problem solve, learn, pull tools together and build new tools to complete its task.

“What is scary here is there mostly likely was no malicious intent, so what we have here is non-malicious autonomous agent compromise. The agent was performing its duty, conducting research, and was attempting to answer questions tasked to it.”

Therefore, Kleinman’s concern is that an organisation could judge an agent’s work by whether it completed the task, without examining whether its method was authorised. That distinction must become part of security testing.

“Was the decision-making process undertaken by the autonomous agent appropriate?” Kleinman asked.

“Did it circumvent deployed and other intended controls? And was the objective achieved through an authorised/unauthorised method? That is a very different methodology of testing.”

What failed, and who should have known?

The incident also raises questions on both sides of the connection – How did the agent get through? What could the government system see? What controls were in place around the agent’s behaviour?

The forensic investigation has yet to establish a complete account.

But Kevin O’Sullivan – National General Manager, Cyber at OneStep Group – warned against treating an incomplete investigation as a verdict on either party.

“The headline is interesting but the fundamentals are more important,” O’Sullivan said.

“We don’t yet know enough about every control failure, every action taken by the agent, or the complete sequence of events to confidently declare exactly who failed and why.

“So, perhaps the cyber security industry would benefit from resisting the temptation to turn every incident into a verdict before the investigation is complete?”

O’Sullivan also rejected the suggestion that a weakness in a target system would settle the question of whether the agent’s behaviour was acceptable.

“A weakness in the target environment and inappropriate or unauthorised behaviour by the system accessing it can both be true at the same time,” he expanded.

“If weaknesses existed in the government environment, they should be understood and addressed. Equally, organisations developing increasingly autonomous systems have a responsibility to constrain what those systems are authorised to do, monitor when they move outside those boundaries, and respond appropriately when they do.”

Detection is one of the unresolved questions.

“Did Services Australia detect the potential exfiltration themselves?” Bachtold of Increment asked. “If non-public information was accessed and retrieved, common sense says that activity should be visible in their own environment.”

Then there is the delay in disclosure.

The activity occurred in June while OpenAI became aware in August. The Prime Minister said Services Australia received its first notification on 10 September, in an email to a public mailbox.

The government has since commissioned a rapid review of its arrangements for incidents involving AI, including governance and information sharing.

“There is a legitimate incident-management question,” O’Sullivan of OneStep Group added. “Detection is only part of cyber response. Escalation, communication, governance and accountability matter as well.”

What happens when the agent is already inside?

In this incident, the agent was approaching a government website from outside.

Many businesses are giving agents approved access from within: to emails, documents, customer information, applications, APIs and workflows. An agent may be able to act using permissions that its owner has already granted.

“The biggest lesson is that businesses are adopting AI faster than they’re implementing the security controls needed to manage it safely,” Simpson of CyberAi added.

“Most organisations have AI agents connected to email, documents, customer data and internal applications, often with broad permissions and limited oversight. The greatest risk today is not necessarily malicious AI, but AI with too much access making decisions or taking actions its owners never anticipated.”

For Leisegang of Tecala Group, businesses must first understand what they have in terms of AI agents.

• Which AI agents are operating in the business?
• What systems can they access?
• What data can they see?
• What actions can they take?

“Once you understand that, you can start looking at whether those permissions are appropriate, what needs human approval, what’s being logged and monitored, and what happens if something goes wrong,” Leisegang advised. “You can’t govern what you don’t know is there.”

According to Jacqui Nelson – CEO at DekkoSecure – businesses must also focus on the consequences if a boundary fails.

Vulnerable systems and data compromises existed long before AI agents; the new concern, in her view, is how persistently an agent can search for an opening.

“The real risk with AI is the tenacity and speed at which it scans and exploits systems,” Nelson outlined.

“Vigilance and understanding where vulnerabilities can potentially provide access to sensitive data, means you need to protect that data using zero knowledge architecture and end-to-end encryption (E2EE) so that in the event that a system is breached, the data is never exposed.”

This point adds a second question to access control: if an agent does reach a system it should not, how much sensitive information could it actually reveal?

“It reinforces the importance of controlling access and permissions,” stated Sarah Dewan, CISO at Secure Agility.

“The biggest risk isn’t necessarily the technology itself, but granting AI agents broad access to systems, data and workflows without adequate governance, monitoring and oversight.

“CIOs and CISOs must start with visibility. Organisations should identify what AI tools and agents are operating in their environment, understand what data and systems they can access, review permissions against least-privilege principles, and ensure logging and monitoring are in place.”

A gap between designing responsibly and selling responsibly is also emerging.

Vendors can design a technology with care and still commercialise it in ways that cause harm, through supply chains that exploit workers, access that favours some groups over others, or marketing that misleads.

Most existing guidance on responsible innovation focuses on research and development, a new product development stage, before a product reaches customers.

However, far less is known about what happens once a technology is being sold and scaled, which is precisely when unintended consequences tend to surface with technologies such as AI.

“For AI companies, this gap is where much of the risk now sits,” said Tania Bucic, Business School Professor at University of NSW (UNSW).

“A model can pass every pre-release evaluation and still behave in unexpected ways once it is given tools, connected to real systems and deployed by millions of users.

“Decisions about who gets access, how quickly a product is scaled, which partners integrate it, and how its capabilities are marketed are commercial decisions, but they shape the harm a technology can do just as much, if not more so, than its technical design. If responsibility stops at the lab door, those decisions go unexamined.”

The message for leaders in Australia is that responsible innovation is not a brake on commercialisation – it’s a capability that makes commercialisation more resilient. The companies that do this well don’t rely on a single ethics review or a safety team working in isolation.

“They build anticipation, reflexivity, inclusion and responsiveness into everyday commercial decision-making – enabled by and sustained by culture,” Professor Bucic advised.

“For AI companies moving as fast as they are, that capability is what will allow them to keep earning the trust of customers, regulators and the public.”

Evidence before autonomy

Despite the headlines, the incident should not be viewed as a reason to abandon agentic AI – rather, to demand more than an assurance that an agent has been tested.

“Before you give an agent more autonomy, you need to know it’s been properly tested, that you can see what it’s doing, that you have the right security and permissions around it, and that there’s a clear point where a human steps in,” Leisegang of Tecala Group explained.

“The more autonomy you give an agent, the more important those controls become.”

As outlined by Kleinman of FedCyber, CISOs managing this type of incident should start by conducting an AI asset inventory exercise.

This must examine a whole range of assets including Microsoft Copilot agents, ChatGPT Enterprise GPTs, OpenAI agents, in addition to lesser known agents such as ServiceNow AI agents or AWS Bedrock agents.

“Basically, all non-human identities and machine identities,” Kleinman shared.

“Next, I would investigate whether the organisational environment contain agents and machine identities with excessive permissions and trust. Many agents received excessive levels of privilege when built e.g. API credentials or service accounts beyond what they need. For most SOCs, the monitoring performed often assumes such identities are trustworthy.”

Then for each agent, organisations should determine the permission pathways. Map the paths available to each agent, including the identities and credentials it uses.

“What are the systems it can access?” Kleinman continued.

“What APIs can it call? Does it trigger workflows and what are they? This will allow you to create a permissions matrix. For each agent, apply the principle of least privilege.”

That map establishes what an agent could do. Testing and monitoring must then establish what it actually does when it meets an obstacle, receives an unexpected instruction or finds an alternative path.

“AI can deliver significant benefits, but organisations should ensure appropriate testing, monitoring, auditability, and incident response processes are established before scaling more autonomous AI capabilities,” recommended Dewan of Secure Agility.

In other words, slowing down is not necessarily the safest path because the greatest source of risk is the technology and activity that businesses cannot see or control.

“Organisations that fail to establish a robust governance framework create an environment in which unsanctioned tools can proliferate,” added Bachtold of Increment.

“When AI adoption is governed appropriately, organisations can implement controls, monitoring, detection, and response capabilities to manage risk effectively. This is a more effective approach than attempting to avoid or ignore the technology altogether.”

For Simpson of CyberAi, this incident shouldn’t stop AI adoption, but it should stop “reckless” AI adoption.

That means before scaling agentic AI, organisations must require evidence of security testing, access reviews, monitoring, audit trails and incident response processes. Detecting abnormal AI behaviour, understanding what happened if something goes wrong and rapidly containing any issues before they escalate are mission-critical considerations.

“The first priority is visibility,” Simpson said.

“CIOs and CISOs need to identify every AI agent and AI-enabled application operating in their environment, understand what data each can access, and map what actions they can perform.

“From there, organisations can control and monitor AI agent functions, while also applying least-privilege access and audit logging. If an AI agent has the ability to take action on behalf of a user, it should be treated as a privileged identity and governed accordingly.”

Looking behind (and beyond) the headline

O’Sullivan of OneStep Group highlighted that AI has not invented vulnerable applications, excessive permissions or gaps in detection. It can, however, alter how quickly and persistently those weaknesses are found.

“Weaknesses that may once have been too difficult, time-consuming or uneconomic to pursue can potentially be identified and exploited far more quickly, and at much greater scale,” O’Sullivan noted.

“That makes the fundamentals more important, not less – identity, least privilege, secure configuration, vulnerability management, segmentation, application security, logging, detection, response and resilience. None of these became obsolete because we added AI to the threat landscape.”

The government’s investigation will offer more insight about the agent’s actions, the portal’s defences and the delay in reporting the incident. But there is also a little irony in some of the commentary around breaches of this magnitude in the media.

This is a market well versed in looking at someone else’s breach and quickly identifying what they should have done differently.

“The reality is rarely that simple,” O’Sullivan reminded.

“Most organisations are dealing with legacy technology, competing priorities, constrained budgets and years of accumulated decisions. That doesn’t excuse poor security, but equally, finding a weakness in someone else’s environment doesn’t necessarily say much about the maturity of your own.

“Perhaps before throwing too many stones, we should all be comfortable with the same level of scrutiny being applied closer to home.”

This incident should act as a wake-up call for Australian organisations, however.

Businesses are still struggling to consistently defend against human attackers and are now entering a world where AI can operate faster, longer and at greater scale than any individual.

“AI doesn’t sleep, get distracted or give up,” Simpson of CyberAi concluded.

“This incident reminds me of Rylands vs. Fletcher in 1868. If water flows naturally, you’re not responsible, but if you build a dam and it fails, you are responsible for the damage. The same principle applies to AI agents.”