Who governs the agents?

For two years the whole industry was busy teaching AI to talk — to draft, summarise, answer. That was the easy part, and it’s mostly solved. The part that actually keeps me up is quieter: we’re now handing these systems the ability to do things. Not just describe a refund — issue it. Not just suggest a record change — make it. The moment an AI can act, the interesting question stops being “is the answer good?” and becomes “who is accountable for what it just did?”

That is the whole of AI governance, stripped of jargon. It’s not a compliance checkbox or a committee. It’s the unglamorous engineering of making an autonomous system’s actions legible, bounded, and answerable to a human.

The speech doesn’t hold

Most of what passes for AI safety today is a speech — a paragraph in a system prompt asking the model, politely, to behave. It works right up until it doesn’t: a clever input, an edge case, a goal pursued a little too literally. You cannot build the safety of a system that takes real actions out of a request it’s free to reinterpret. The constraints have to live somewhere the model can’t argue with.

I think of it the way we think about any other piece of infrastructure that can hurt someone. A building isn’t kept safe by a sign asking people not to fall. It’s kept safe by the structure — the railing, the load rating, the inspection. Governance is the structural engineering of AI: the railings you can’t talk your way past.

What governing well looks like

Underneath the noise, the pieces are surprisingly consistent — and they line up neatly with the public frameworks now converging, from the NIST AI Risk Management Framework to the OECD principles to the way the EU’s AI Act is phasing in:

• Identity — every action is tied to who (or what) authorised it, all the way up the chain.
• Permission — the system can do only what it was explicitly granted, and nothing wider, by construction.
• Oversight — a person can see what’s about to happen, understands it, and can stop it, in time to matter.
• Record — an audit trail that can’t be quietly edited after the fact, so a decision can actually be inspected.

None of those are exotic. They’re the same ideas we already trust in banking, aviation, and medicine — applied to a new kind of actor. The work isn’t inventing new ethics; it’s making old, well-earned discipline hold for software that moves at machine speed.

Governance isn’t a brake on AI. It’s the thing that lets you take your foot off the brake at all.

Why it’s worth the patience

It’s tempting to read all this as friction — as the cautious people slowing down the exciting people. I’ve come to believe the opposite. The organisations that will move fastest with AI are the ones that trust their own systems, because they can see what they’re doing. Governance is what earns that trust. Get it right and you don’t adopt AI more slowly; you adopt it without flinching.

That’s the work I keep returning to. Not because the actions AI takes frighten me, but because I’d like us to deserve the confidence we’re about to place in it.