Picture this, it’s a Tuesday morning and your phone rings. It’s your Chief Risk Officer, one of the AI agents your team switched on last quarter has done something nobody asked it to do. It wasn’t hacked, it wasn’t misused, it was simply trying, very hard, to finish the job it was given.

Then comes the question that matters: who signed off on that?

If you can’t answer in one sentence, you’re not alone. But as of this month, you’re not in a comfortable place either.

What happened

In June this year, an OpenAI agent researching Australian public health spending was turned away by the Medicare Statistics Reporting Service portal run by Services Australia. The difference is that it didn’t stop, it just found a way around the restrictions and reached non-public files.

To reassure you no personal Medicare records are believed to have been accessed, but Services Australia didn’t hear about it until 10 September, almost three months later, and then via an email to a general inbox rather than a security channel. OpenAI has since apologised and committed to setting up an Australian-based taskforce. The Government has stood up its own taskforce with the Australian Signals Directorate and the AI Safety Institute, and mandatory reporting for rogue AI incidents is expected to follow.

At last week’s (wc 19/10/26) Parliamentary AI hearings, an AI law expert said it is unclear who is legally responsible when an AI agent acts on its own. Under today’s rules, responsibility largely falls on whoever held the data, not on whoever deployed the agent that took it. I am sure that that gap won’t last very long. The parliamentary committee has already recommended a duty of care for AI developers, and stronger legislation is expected early next year.

Why this is your problem, not just OpenAI’s

It’s tempting to read this as a story about a frontier AI lab. Take my word this isn’t. It’s a story about agents, software that plans and acts across systems without a person checking each step.

Your organisation may already have them. They could exist in your CRM, or in a procurement workflow, a coding pipeline, customer service channel, or inside a vendor platform you bought last year.

So, ask yourself honestly:

• Do you know how many AI agents are acting on behalf of your organisation today?
• For each one, could you name the person accountable for what it does?
• Does it understand that “no” means no, and does anyone get told when it tries again?
• If it crossed a line on a Tuesday, how long before your board hears about it?

The Medicare case exposed two failures. The first was a control failure, such that the agent kept trying after it was refused. The second was an accountability failure, a major organisation took three months and then sent an email to a major Government organisation via a generic email inbox. In my experience, it’s the second one that ends careers. Many questions must be answered by OpenAI internally how such a situation could exist.

Autonomy doesn’t transfer responsibility

Over many years leading major programmes for Corporate and Government clients, I have learned one thing for certain. When something goes wrong, “the system did it” has never been an acceptable answer to a board, a regulator or a minister.

An AI agent is no different. Autonomy changes how the work gets done. It does not change who is answerable, or who is checking that the AI agent is performing its role as it should. It is the same if you recruited an junior in your team, you s check the work to make sure they remain within the boundaries of their job role, so why do we think AI is managed differently.

Because organisations are not doing this correctly so the Government authorities are now bring in regulations to force organisations to implement the checks. You can wait and accept the chain of accountability Canberra designs for you, or you can build your own now, fitted to your business, and be ready to show it when you’re asked.

What a chain of accountability looks like

This is exactly why we have used our years of supporting Financial Services organisations around the world and built BAIGS® – the Bushey AI Governance Spine. It gives every AI agent four links in its chain:

1. Defined Ownership. A named person, not a team, accountable for the agent’s purpose, permissions and behaviour.
2. Clear Boundaries to work within. Explicit limits on which systems it can touch, and a clear rule for what it does when refused: stop and escalate, never improvise.
3. A set of Records. Logs that show what it did, why, and on whose authority, readable by someone other than the engineer who built it.
4. An Incident Process and Timeline. An incident path that reaches the right executive in hours, not months, and is ready for mandatory reporting.

None of this slows you down. It’s what lets you scale agents with confidence, rather than hoping nobody asks the question.

Start this week

You don’t need a 12-month programme to begin, start with an inventory. Start by listing every agent, its owner, and what it can reach. Most organisations find at least one they didn’t know about, and it only takes one to go rogue.

If you’d like a second pair of eyes, we’ at Bushey have developed our processes and tooling to help you bring clarity and alignment. We would also be glad to walk you through a BAIGS® assessment. Far better that you find the gap than a parliamentary committee does.

Before your phone rings.

Summary

When an OpenAI agent slipped past the controls on a Medicare statistics portal, and lawmakers at this week’s parliamentary AI hearings admitted no one is sure who is legally responsible, it became clear that autonomy doesn’t transfer accountability, your organisation still answers for what its agents do.

BAIGS® gives every AI agent a named owner, clear boundaries, a readable record and a fast incident clock, so you can build your own chain of accountability before regulators write one for you.

About Bushey’s AI Services

Bushey is a specialist boutique consultancy, helping Corporate and Government organisations across APAC use AI with control, confidence and measurable business benefit. Our Artificial Intelligence Practice delivers:

BAIGS® – Bushey AI Governance Spine. Clear decision ownership, approvals, risk evidence and a safe path to production for every AI use case and agent.

BAIGS® AI Governance Assessment. An independent view of where your AI governance stands today, with the gaps prioritised and a practical roadmap to close them.

AI Readiness and Implementation. Taking AI from experiment to production, delivered through AssureChange®, our Transformation delivery framework that brings together Project Management, Change Management and Governance to deliver real business benefits.

Fractional Chief AI Officer. Senior AI leadership for your executive team and board, without the cost of a full-time hire.

AI and Cybersecurity. Through our Bushey Cybersecurity Practice, we help secure the systems, data and agents your AI touches, and prepare your incident response for mandatory AI reporting.

Ready to put a chain of accountability around your AI agents? Visit bushey.com.au to book a conversation.