9Governance (assurance and feedback)

Governance is the domain that makes the other four trustworthy over time. It provides the transparency, accountability and feedback that keep AI aligned to the agency’s intent, and it is the domain that closes the assurance loop, feeding what it learns back into strategy. The central shift this framework asks of an agency is to stop thinking of governance as a document produced after a system is built, and to start treating it as a substrate built into the system itself. This section sets out the runtime controls and layers of that substrate, the assurance lifecycle that operationalises it, and the obligations of transparency and contestability that government carries above all other sectors.

9.1Governance as substrate, not overlay

Traditional governance is retrospective and external: it reviews, audits and reports after the fact. For AI acting at machine speed, that is too late. The necessary reframing is that governance must become the living condition of the system: policies expressed as executable data rather than PDFs, constraints enforced at the moment of action rather than reviewed afterward, and escalation built into the structure rather than left to chance. This does not replace the agency’s committees, audits and reports; it makes them effective, by ensuring that what they govern is observable and that the rules they set are actually enforced in runtime.

9.2Five runtime controls for governing AI

Five structural controls let an agency govern AI in real time rather than after the event. They are the load-bearing elements of the governance substrate, and each translates into deliverables an agency can build and audit.

9.3The five layers of governance

Where the controls are the structural supports, five layers of governance describe where governance must reach, from the foundational to the evolutionary. The framework uses them as a completeness check: an agency should be able to point to governance at each layer, not just the convenient ones.

9.4Policy as code, and a lifecycle for policy

The mechanism that makes embedded governance real is policy-as-code: rules, thresholds, permissions and escalation paths expressed as a versioned data bundle that the runtime enforces, rather than prose a developer interprets. Like any code, policy needs a lifecycle, draft, shadow, canary, active, deprecated, retired, and tests, so that a change to the rules is promoted on evidence, not declared. For an agency, the payoff is consistency and contestability: the same rule is applied to every case, and the agency can show exactly which rule, in which version, applied to any particular decision. This is the architectural answer to the Robodebt lesson.

9.5The assurance lifecycle

Governance becomes operational through a single assurance lifecycle that every AI use case travels, from intake to retirement, with go/no-go gates between stages. The lifecycle is the place where this framework’s models meet the Commonwealth’s instruments: it is structured on the AI technical standard’s three phases, it carries the assurance framework’s risk-based approach, it embeds the AI6 practices, and it produces the policy’s mandatory artefacts. Section 10 details it; the figure here shows the whole.

Discover

1. 1Intake and triage
2. 2Impact assessment
3. 3Risk tiering
4. 4Design assuranceG1
5. 5Build, test and simulateG2

Operate

1. 6Deploy (gated)G3G4G5
2. 7Operate and monitor

Retire

1. 8Retire

G1 design complete, G2 simulation passed, G3 governance validated, G4 shadow proven, G5 organisationally ready. AI6 practices run throughout.

9.6Transparency, contestability and human review

Government carries obligations of transparency and contestability that most sectors do not. People affected by a government decision are entitled to know that AI was involved, to understand the basis of the decision, and to seek review. Three concrete commitments follow, and the framework treats them as mandatory for any AI that contributes to a decision about a person. First, transparency: the agency publishes and maintains transparency statements, now aggregated by the DTA in a central public list that makes absence as visible as presence, and discloses automated decision-making in its privacy policy ahead of the December 2026 obligation. Second, contestability: every such decision has a clear, accessible path to human review, and the provenance to support it. Third, human accountability: a named official remains accountable, and the system is designed so a human can understand, override and, where required, decide. These are the Robodebt lessons rendered as design and governance commitments.

9.7An ethical floor for AI

As a final backstop, Renly recommends the agency adopt a short ethical floor for AI: a set of conditions that hold for any system regardless of its use, even when a business case is pressing. We propose seven: do no harm; accept no unverifiable input as a basis for action; keep the system aligned to its declared purpose; bound its autonomy explicitly; embed governance in the runtime; keep accountability with a named human; and keep oversight adaptive as the system changes. These are consistent with the eight AI Ethics Principles and give engineers a short, memorable charter that sits beneath every use case.

9.8Governance outputs