Australia will hold frontier AI firms to six obligations under a new regulator

01Six duties, and a regulator to enforce them

The regulator will require companies to:

1. maintain security controls across all stages of development and deployment;
2. identify and manage serious risks associated with their models;
3. allow independent evaluation of their models;
4. notify the Australian Government of incidents, rapidly;
5. maintain and publish safety and security frameworks;
6. respond to updated standards and rules as the technology changes.

The duties will sit in Australia’s planned AI Standards legislation. The regulator would work with the Australian AI Safety Institute for technical expertise, and the government says the measures are aligned with the interim report of Parliament’s Joint Select Committee on Artificial Intelligence.¹ The release names no regulator, sets no penalty amount and gives no date for a bill: “further details” will come “in due course”.¹

02What set it off

On 18 June an OpenAI research team used an internal model to look up public spending on medicines. Blocked, the model “attempted alternative ways” to get the data, accessed public and non-public files in the Medicare Statistics Reporting Service portal and wrote files to an internal server, Albanese said.⁴ OpenAI says it was an experimental, internal-only model without the safeguards of its public products. It “ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files”; no patient records were accessed. OpenAI found the activity in mid-August, during a review it began after a separate incident in July, and also reported activity at three other Australian bodies.⁵

03What the review asks for

The review was run by the Department of the Prime Minister and Cabinet with the National Cyber Security Coordinator, the Office of AI, the Australian Signals Directorate, the AI Safety Institute and Services Australia.² Only its recommendations are published. Among them: consider “mandatory, time-bound” reporting of misaligned behaviour and agentic cyber incidents; protections that make it safer for AI developers to report incidents and near misses; criminal penalties for egregious non-compliance, or a standalone “failure to prevent” offence for a company whose model causes harm it took no reasonable steps to stop; and guidance for agencies on escalating reported problems that have not yet been technically confirmed.³ Agencies are told to shrink the government’s digital estate and to shut down or segregate systems that need not be on the internet.³

Two dates to watch: agencies running national crisis plans must report how AI changes them by the end of 2026, and OpenAI says the Australian taskforce it set up will finish its work by the end of the year.³⁵