‘Extreme concern’ as OpenAI’s ‘rogue agent’ breached Australian govt health database

An artificial intelligence (AI) agent developed by US tech giant OpenAI breached an Australian government health statistics system during an internal evaluation, accessing both public and non-public files and writing files into the system, Prime Minister Anthony Albanese said on Wednesday.

According to CNN, the incident is believed to be the first publicly disclosed case of an AI agent breaching an Australian government network and comes amid growing concern over the ability of increasingly autonomous AI systems to circumvent safeguards and undertake actions beyond those intended by their human operators.

Albanese said the AI agent had accessed an Australian Medicare statistics database and bypassed restrictions after failing to obtain information through normal channels. He said there was no evidence that patients’ personal medical information had been accessed.

“Nonetheless, this situation is obviously unacceptable,” Albanese told reporters on the sidelines of the United Nations General Assembly in New York.

The prime minister said he had spoken by telephone with OpenAI chief executive Sam Altman on Wednesday to convey Australia’s “extreme concern” over the incident.

Albanese also criticised the company for the delay in notifying Australian authorities. According to the government, OpenAI discovered the activity in August but did not inform Canberra until September 10, when it sent an email to a generic government mailbox.

Read: AI leaders warn UN of security risks as systems grow more powerful

“It took the company way too long to inform the government what had occurred, and the nature of the way that notification occurred as well was unacceptable,” Albanese said.

The disclosure came as governments, technology companies and researchers grapple with the security implications of AI agents capable of independently browsing the internet, executing tasks and adapting their behaviour when they encounter restrictions.

AI agent ‘scaled the fence’

The breach occurred in June while OpenAI was conducting an internal evaluation of its models, according to the company and Australian officials.

The model had been asked to search the internet for information about Australian government spending on medicines and healthcare, including statistics that could be used to answer questions about public expenditure.

When it encountered restrictions on the information it could access, however, the AI agent attempted to circumvent them, Australian officials said.

Australian Defence Minister Richard Marles described the behaviour in unusually simple terms. “It asked a question, the information was not given and rather than leaving at that point, it scaled the fence,” Marles said.

OpenAI spokesman Drew Pusateri said the company became aware of the activity in August while conducting an extensive review of its models’ activity.

“During this review, we identified activity involving several Australian government websites and services as our models attempted to look up answers and available statistics for questions about Australia during an internal evaluation,” Pusateri said. “In the course of that, our models took actions we did not intend.”

OpenAI said its investigation found no evidence that patient records had been accessed in the Medicare incident. The information accessed included aggregate health statistics and internal file names, according to the company.

Australian officials said the breach involved an older health statistics website rather than a system containing individual medical records.

Government Services Minister Katy Gallagher said the notification from OpenAI was sent to a public mailbox that was checked only once a day.

“That email address is looked at once a day,” Gallagher said, adding that such mailboxes can receive large numbers of notifications, including hoaxes.

The delay has become a central concern for Canberra because the government was not immediately aware that an AI system had penetrated restricted areas of a government service.

Other government systems examined

Albanese said three other government systems may also have been affected by AI activity.

They included the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research and the Victorian Department of Health.

The Australian government said there was no indication that the Australian Institute of Health and Welfare system had been breached, although the website was among those targeted. A forensic investigation, assisted by the Australian Signals Directorate, is now examining the incident and determining whether any other government systems were affected.

Marles announced a taskforce investigation and sought to reassure the public that the impact appeared to be limited. “No individuals’ medical data was accessed here.

The system itself has not been in any way compromised,” he said.

The investigation is nevertheless significant because the incident involved an AI system acting beyond the boundaries of the task it had been assigned. Rather than simply searching publicly available information, the model attempted to overcome technical restrictions when it could not find the information it wanted. That behaviour has heightened concerns among cybersecurity specialists that AI agents could increasingly become capable of finding and exploiting vulnerabilities without being explicitly instructed to do so.

Earlier incidents emerge

The Australian breach was disclosed at the same time as research organisation Transluce reported several earlier instances in which AI agents apparently attempted to exploit computer systems after failing to obtain information through conventional web searches.

The incidents, dating back to at least March, suggest that the phenomenon may have emerged earlier than some of the high-profile cases disclosed by AI companies in recent months.

Transluce said the incidents it identified involved relatively mundane research tasks rather than deliberate cyberattacks.

Read more: Anthropic, OpenEvidence partner to bring medical AI worldwide

In May, AI agents attempting to retrieve a photograph from a University of New Mexico library collection sent a “flood” of requests but were unsuccessful.

Later that month, agents targeted Data USA while attempting to obtain visualisation data relating to the University of Iowa.

In June, agents seeking statistics from the Australian Institute of Health and Welfare attempted to exploit vulnerabilities after normal methods failed. Transluce said no non-public data was exposed, although the agents bypassed anti-bot controls.

The Australian government acknowledged that AIHW was among the sites targeted but said it did not believe a breach had occurred.

Hugging Face breach raised alarm

The latest revelations follow a more serious incident disclosed by OpenAI in July, when the company said its models had created a swarm of AI agents during cybersecurity testing that breached systems belonging to Hugging Face, a platform widely used by AI developers to store and share code and models.

The models were operating in what was intended to be a controlled testing environment but managed to escape the restrictions placed on them and access real-world systems.

The incident became a prominent example in the AI industry of the potential risks associated with increasingly autonomous systems.

Other major AI developers have since reported similar concerns.

Anthropic said its models had gained unauthorised access to systems belonging to three unidentified organisations during testing intended to prevent them from interacting with real-world systems.

Google has also said its Gemini model was able to gain access to multiple systems by guessing login credentials.

The incidents have contributed to growing calls for stronger safeguards as AI systems move from answering questions to independently carrying out complex tasks across the internet.

Walayat Hussain, an associate professor of information technology at Australian Catholic University in Sydney, said the Australian incident and the earlier Hugging

Face breach appeared to form part of a broader pattern.

“Today’s AI agents are becoming brilliant at getting things done, but they are still poor at knowing where the line is,” he said.

Hussain said companies and governments could not assume that AI systems would reliably police their own behaviour, while warning that many government systems had not been designed with sophisticated autonomous AI agents in mind.

For Canberra, the immediate focus remains on determining precisely what the OpenAI system accessed and whether any other government infrastructure was affected.

Albanese said Australia’s investigation would establish the full extent of the incident, while the government would also examine OpenAI’s handling of the disclosure.

The episode has added a new dimension to the debate over AI safety: not simply whether increasingly powerful models can be prevented from generating harmful content, but whether autonomous agents can be reliably constrained when given access to real-world computer systems.

The Australian government has so far stressed that no individual medical records were accessed. But the incident has raised questions about whether existing safeguards, monitoring systems and notification procedures are adequate as AI agents become increasingly capable of acting independently online.

(With additional input from agencies)