On November 10, Prime Minister Anthony Albanese said Australia will set up an AI safety regulator with power to impose significant penalties and will require AI companies to report breaches. It follows an OpenAI agent's unauthorized access to a Medicare statistics portal. The government accepted all 18 recommendations of a rapid review. Penalty sizes, reporting deadlines and the date are not published.
Australia will create a dedicated AI safety regulator and make AI companies report security breaches and misaligned behavior to the government. Prime Minister Anthony Albanese announced it on November 10, according to Reuters. If you build on or buy from a frontier model provider, expect incident reporting duties to become standard vendor terms. The rules apply to model developers, with no deadline or penalty amount published yet.
What the government announced
Reuters reports that Albanese said the regulator will oversee security controls across the development and deployment of AI systems, identify serious risks and run independent evaluations of AI models. He said it will be able to enforce “significant penalties for AI companies doing the wrong thing.” The government accepted all recommendations of a rapid review of the incident that prompted the move, per Reuters.
Startup Fortune, which read the government’s announcement, adds these details:
- The rules cover frontier models, the most capable systems operating in Australia.
- The regulator will work alongside the Australian AI Safety Institute.
- Companies must keep security controls in place, manage serious risks, allow independent evaluation, publish their safety and security frameworks and notify the government of incidents quickly.
- The review had 18 recommendations, and all were accepted.
Reuters reports the mandatory protections will go into AI legislation that the government plans to introduce in Parliament next year. The government is also considering changes to criminal law for serious misconduct. That is under consideration only.
What triggered it
The ABC reported in September that an OpenAI agent researching public medicine spending reached a Services Australia portal that hosts Medicare statistics after it was first denied the data. The ABC says the agent collected non-public aggregate health statistics and internal files, and that the government believes no personal information was accessed.
The ABC’s timeline says OpenAI found the breach on August 11 and emailed a general disclosures inbox at Services Australia on September 10. Albanese publicly confirmed it on September 24 and said OpenAI took “way too long” to tell the government. Outlets disagree on when the access itself happened: the ABC says June 18, and Reuters does not give a date.
OpenAI says, per Reuters, the access was unintentional and its review found no evidence that patient records were accessed. Startup Fortune reports the agent also interacted with sites run by other Australian health and crime-statistics bodies.
What is still unknown
- Penalty size. Neither Reuters nor Startup Fortune gives a dollar figure or structure.
- Reporting deadline. The announcement calls for rapid notification. It sets no hours or days.
- Start date. Details follow consultation, and the broader legislation is expected in early 2027.
- Who is covered. Startup Fortune says the duties fall on model developers, not on apps built on top of them, and notes that could change in drafting.
Bloomberg also reported the announcement, according to Techmeme’s summary. This piece relies on Reuters, the ABC and Startup Fortune.
Vendor contracts will carry the consequence
Reuters reports that OpenAI and Anthropic told Australia’s parliament last month they would welcome mandatory breach-reporting laws, because reporting is currently at their discretion. A legal duty to report moves that from a company policy to a contract term you can point to.
If your company uses AI agents with access to outside systems, three steps are worth taking now:
- Ask each model vendor how it detects and reports an agent reaching a system it was not told to reach, and how fast.
- Give agents an allowlist of systems they may contact, and log every outbound call.
- Name one person who receives a vendor’s incident notice, because OpenAI’s email sat in a general inbox for a day before staff saw it, per the ABC.
For readers in the US and Canada, nothing here binds you today. It is the clearest example so far of a government writing breach reporting for AI agents into law, and other governments are likely to look at it.