Product

Product overview Agents Automations Apps Code (Beta) Integrations Governed Runtime Security

Where Renly runs

Windows app Android app Inside other AI apps

Solutions

Finance Sales Operations People and HR Marketing Enterprise

Resources

Pricing Docs Academy Blog Compare Support

AI governance and capability

AI governance framework AI governance assessment AI capability maturity AI maturity assessment
Book a demo Sign in

AI Capability Maturity Assessment · Part II. The eight domains

Domain 3 · Ways of working layerRisk, assurance and transparency

22 minute read · Version 1.0.0 · five criteria

In this chapter

Covers how AI risk is assessed and managed for each use case: impact assessments, tiering, the lifecycle and its gates, transparency and contestability for the people affected, and incident handling.

  1. What this domain covers
  2. The domain at each level
  3. Risks when it is weak
  4. The five criteria
  5. 3.1 Impact assessments
  6. 3.2 Risk tiering in practice
  7. 3.3 Lifecycle and gates
  8. 3.4 Transparency and contestability
  9. 3.5 Incident and near-miss management
  10. In Governing Intelligence

What this domain covers

Covers how AI risk is assessed and managed for each use case: impact assessments, tiering, the lifecycle and its gates, transparency and contestability for the people affected, and incident handling.

This is where obligations are met or missed. Impact assessments, risk tiers and gates decide whether a harmful use is caught before deployment. Transparency and human review paths decide whether people affected by AI-assisted decisions can understand and challenge them, which privacy law now requires for many automated decisions. Incident processes decide whether the organisation learns from near misses or repeats them at scale.

Initiatives for this domain are run through the Governance and assurance playbook.

The domain at each level

A summary of what the domain looks like at each level. Each criterion below has its own descriptors and pass tests: score the criteria, not this summary. The domain score is calculated from them (see domain aggregation).

LevelWhat it looks like
1InitialNo AI impact assessments, tiers or AI-specific gates; the risks are unknown. People affected are not told AI is involved and have no specific path to challenge a decision. AI incidents are not recognised as such.
2RepeatableSome prominent use cases have been assessed, using varying templates and often after deployment. Tiering is informal and inconsistent. Disclosure is partial. AI incidents go through general IT or privacy processes, and near misses are not recorded.
3DefinedA standard impact assessment is completed before deployment for every in-scope use case. Tiers are applied by written criteria. A lifecycle with go or no-go gates scaled by tier is followed and recorded. A transparency statement is current, decisions about people are inventoried with a human review path, and an AI incident runbook with halt authority has been rehearsed.
4ManagedGates are enforced so nothing reaches production without a recorded decision. Assessments are refreshed on change and tiering consistency is tested. Review requests and incidents are tracked and analysed, with corrective actions closed. Audit evidence can be produced on demand.
5OptimisingAssurance is continuous and draws on generated evidence such as test, shadow and monitoring results. Lessons from assessments and incidents improve controls across the portfolio. Transparency measures are reported and improving, and affected people are consulted on high-risk designs.

Risks when it is weak

  • Harmful or unlawful AI-assisted decisions about people that cannot be explained or reversed
  • Breach of privacy, administrative law or sector obligations, including automated-decision disclosure
  • Repeated incidents because near misses were never recorded
  • Assessments written after the fact to justify decisions already made
  • Inability to evidence controls when audited

The five criteria

No.CriterionImportanceScope
3.1Impact assessmentsFoundationEnterprise arrangement, applied locally
3.2Risk tiering in practiceCoreEnterprise arrangement, applied locally
3.3Lifecycle and gatesCoreEnterprise arrangement, applied locally
3.4Transparency and contestabilityCoreEnterprise arrangement, applied locally
3.5Incident and near-miss managementFoundationEnterprise arrangement, applied locally

Criterion 3.1

Impact assessments

Every in-scope AI use case has a completed impact assessment before deployment, covering effects on people, privacy, fairness, security and lawfulness, refreshed on material change and at a frequency set by its tier.

Why it matters

The impact assessment is where harm is anticipated rather than discovered. Done before deployment, it changes designs; done afterwards, it only justifies them.

The five levels

LevelWhat it looks likeTo score this level
1InitialNo AI impact assessments are done. Privacy impact assessments, where they exist, do not consider AI-specific harms.The starting point: no test.
2RepeatableSome use cases have assessments, usually the largest projects. Templates vary, and assessments are completed after deployment or never refreshed.At least one completed assessment that considers AI-specific harms.
3DefinedA standard impact assessment method is applied to every in-scope use case before deployment, new use cases are screened at intake, and completed assessments are linked from the register.At least 90 percent of in-scope register entries link to an assessment completed before deployment, and intake records show screening of new use cases.
4ManagedAssessments are refreshed on material change and on a schedule set by tier. Mitigations are tracked to completion, and the second line samples assessment quality.Refresh records for use cases that changed materially, a mitigation tracker with closures, and a second-line quality sample in the last 12 months.
5OptimisingAssessment findings are aggregated across the portfolio to improve design patterns and controls. Affected people or their representatives are consulted for high-tier uses, and their input changes designs.A portfolio analysis of assessment findings that led to a changed pattern or control, and records of consultation with affected people for a high-tier use.

Evidence to request

Documents

  • Impact assessment method and template
  • Refresh schedule by tier

Records of it operating

  • Completed assessments linked from the register
  • Intake screening records
  • Mitigation tracker
  • Second-line quality review

In the evidence request: ER-3-01.

Interview probes

  • Use-case owner or business leadWhat changed in your design because of the impact assessment?
  • Legal, privacy and records leadHow do AI impact assessments and privacy impact assessments fit together?
  • Risk, compliance and internal auditHow do you know an assessment is refreshed when a model or its data changes?

Red flags

  • Assessments completed by the vendor selling the system
  • Every assessment concludes that the risk is low
  • Assessments dated after go-live

Standards crosswalk

ISO/IEC 42001
Clause 6.1.4Clause 8.4Annex A.5
EU AI Act
Article 27
NIST AI RMF
MAP 5

Government and organisation editions

Australian Government edition

Under Policy v2.0, agencies assess every AI use case against the policy’s in-scope criteria during design, and must begin AI use case impact assessments (using the DTA’s tool or an equivalent internal process) within 12 months of the policy taking effect, that is by 15 December 2026. Existing use cases not yet assessed must be brought into line by 30 April 2027. High-risk use cases must be reviewed at least every 12 months, with the review reported to the governing board or senior executive. Agencies must also meet the Australian Government Agencies Privacy Code, which requires privacy impact assessments for high privacy risk projects.

Dated obligations

  • 15 December 2026In-scope assessment of new use cases during design, and impact assessments begun (Policy v2.0)Met at level 3 (Defined) or higher
  • 30 April 2027Existing use cases assessed and brought into line (Policy v2.0)Met at level 3 (Defined) or higher

References

Organisation edition

ISO/IEC 42001 requires AI system impact assessment, and ISO/IEC 42005 gives guidance on doing it. The OAIC recommends a privacy impact assessment before adopting AI products that handle personal information. In the EU, certain deployers of high-risk systems must complete a fundamental rights impact assessment.

References

In Governing Intelligence

In the AI governance assessment: Impact assessments are current for every in-scope use case.

Initiatives that lift it

  • I-301 Adopt an impact assessment method and screen at intake (level 1 to 2)
  • I-302 Complete impact assessments for high-tier and existing use cases (level 2 to 3)
  • I-306 Refresh assessments and tiers on change, and calibrate tiering (level 3 to 4)

Rate 3.1 in the assessmentBack to top

Criterion 3.2

Risk tiering in practice

Risk tiers are applied to every use case at intake using the written criteria, the result is consistent between assessors, and high-risk uses are identified early and governed accordingly.

Why it matters

Tiering is what makes governance proportionate: light for assistive tools, full for decisions about people. If tiers are applied inconsistently, high-risk uses slip through with light controls and low-risk uses are buried in process.

The five levels

LevelWhat it looks likeTo score this level
1InitialUse cases are not tiered. All AI is treated alike, either all lightly or all as projects needing full review.The starting point: no test.
2RepeatableTiers are assigned by the proposer’s own judgment without reference to written criteria, and similar use cases receive different tiers.Some use cases carry a tier label.
3DefinedTiers are assigned at intake by someone other than the proposer, using the written criteria, with the rationale recorded. High-tier uses are flagged on the register and receive the full lifecycle.A sample of five use cases shows tiers assigned by an independent role against the written criteria, with a rationale.
4ManagedTiering consistency is tested (two assessors tier a sample independently and disagreements are reviewed), use cases are re-tiered on material change, and the tier distribution is reported.A calibration exercise record from the last 12 months and evidence of re-tiering after a material change.
5OptimisingTier criteria are refined from incident and assessment data, and automated signals (data classes, autonomy settings, affected population) suggest tiers and flag drift between declared and actual behaviour.A recorded revision of tier criteria driven by incident or assessment data, and tooling that suggests or checks tiers.

Evidence to request

Documents

  • Tier criteria (from 1.2)

Records of it operating

  • Tiering records with rationale and assessor
  • Calibration exercise
  • Re-tiering events
  • Tier distribution report

In systems

  • Tooling that derives or checks tiers from system attributes (optional)

In the evidence request: ER-1-03, ER-3-02.

Interview probes

  • Risk, compliance and internal auditShow me two similar use cases. Do their tiers match, and why?
  • Use-case owner or business leadWho assigned your tier, and did you agree with it?
  • Engineering or architecture leadWhen the system’s autonomy or data changed, did its tier change?

Red flags

  • Proposers tier their own use cases
  • No high-tier uses have been identified even though AI contributes to decisions about people
  • Tiers are never revisited after intake

Standards crosswalk

ISO/IEC 42001
Clause 6.1.2Clause 8.2
EU AI Act
Article 6Annex III
NIST AI RMF
MAP 1MAP 4

Government and organisation editions

Australian Government edition

High-risk use cases under Policy v2.0 must be governed through a board or senior executive, reported to the DTA once deployment is decided and reviewed at least every 12 months. Consistent tiering is how an agency knows which uses those are.

References

Organisation edition

ISO/IEC 23894 and ISO/IEC 42001 call for consistent, repeatable AI risk assessment. Organisations operating in the EU must classify systems against the EU AI Act’s prohibited and high-risk categories.

References

In Governing Intelligence

In the AI governance assessment: A written risk-appetite statement names our risk tiers and who decides.

Initiatives that lift it

  • I-102 Agree the AI risk appetite and tier criteria (level 1 to 2)
  • I-305 Tier every use case independently against the criteria (level 1 to 3)
  • I-306 Refresh assessments and tiers on change, and calibrate tiering (level 3 to 4)

Rate 3.2 in the assessmentBack to top

Criterion 3.3

Lifecycle and gates

An assurance lifecycle from intake to retirement is adopted, with go or no-go gates proportionate to tier, and gate decisions are made by the right authority on evidence and recorded.

Why it matters

Gates are where governance bites. Without them, AI moves from idea to production on project momentum, and the questions that matter (does it work, is it safe, can we stop it) are asked after it is live.

The five levels

LevelWhat it looks likeTo score this level
1InitialThere is no AI lifecycle or gate. AI moves from idea to production on project momentum, with nobody deciding it is ready.The starting point: no test.
2RepeatableProject management gates exist but do not test AI-specific matters such as evaluation results, oversight design or fallback. AI-specific reviews are optional.Project gates exist that AI projects pass through.
3DefinedAn AI lifecycle standard defines stages and gates (for example design, testing or simulation, governance validation, shadow and readiness) with pass conditions scaled by tier. Every in-scope use case introduced in the last 12 months passed through it with recorded decisions.An endorsed lifecycle standard, and gate decision records for every in-scope use case introduced in the last 12 months.
4ManagedNo AI enters production without a recorded gate decision, enforced through change and deployment controls. Conditions attached at gates are tracked, and gate cycle time is measured and managed.Change or deployment controls that block release without a gate decision, at least one ‘no’ or conditional decision, and measured gate cycle times.
5OptimisingGates draw on generated evidence (test results, shadow comparisons, monitoring) rather than written submissions. The lifecycle includes retirement, and retired systems have documented decommissioning and records disposition.Gate decisions that reference generated evidence, and at least one documented retirement with records disposition.

Evidence to request

Documents

  • AI lifecycle standard with gates and pass conditions by tier

Records of it operating

  • Gate decision records
  • Conditions tracker
  • Retirement records

In systems

  • Change or deployment control configuration that checks for a gate decision

In the evidence request: ER-3-03.

Interview probes

  • Engineering or architecture leadWhat stops an AI component reaching production without a gate decision?
  • Use-case owner or business leadWhich gates did your use case pass, and what evidence did you provide at each?
  • Risk, compliance and internal auditShow me a gate decision that was ‘no’ or ‘not yet’.

Staff survey items: S05 If my team wanted to use AI for a new task, I know the process we would follow.

Red flags

  • Gate decisions recorded after go-live
  • The delivery team approves its own gates
  • No retirement stage, so nothing is ever decommissioned

Standards crosswalk

ISO/IEC 42001
Annex A.6
EU AI Act
Article 9Article 17
NIST AI RMF
MANAGE 1

Government and organisation editions

Australian Government edition

The DTA’s AI technical standard structures an AI system’s life into Discover, Operate and Retire; Policy v2.0 strongly recommends it rather than mandating it. Policy v2.0 requires a process for adopting use cases by 15 December 2026, and the ANAO’s criteria test controlled deployment.

Dated obligations

  • 15 December 2026A process for adopting AI use cases (Policy v2.0 responsible AI practices)Met at level 3 (Defined) or higher

References

Organisation edition

ISO/IEC 42001 Annex A sets controls across the AI system life cycle. Providers of high-risk systems in the EU must run a risk management system and a quality management system across the life cycle.

References

In Governing Intelligence

Initiatives that lift it

  • I-303 Adopt the assurance lifecycle and readiness gates (level 1 to 3)
  • I-304 Enforce gates in change and deployment controls (level 3 to 4)
  • I-710 Govern one priority use case end to end as the reference case (level 2 to 3)

Rate 3.3 in the assessmentBack to top

Criterion 3.4

Transparency and contestability

People affected by AI know when it is used, can understand the basis of AI-assisted decisions about them, and can seek human review. Public statements and automated-decision disclosures are current.

Why it matters

Transparency and contestability are what make AI-assisted decisions legitimate. The Robodebt Royal Commission showed the cost of automated decisions people could not understand or challenge. Privacy law now requires many organisations to explain automated decisions in their privacy policies.

The five levels

LevelWhat it looks likeTo score this level
1InitialAI use is not disclosed to customers, citizens or staff, and there is no path to contest an AI-assisted decision.The starting point: no test.
2RepeatableAI use is disclosed in some places (for example a chatbot banner), but there is no organisation-wide statement, no inventory of AI-assisted decisions about people, and contestability relies on the general complaints process.At least one notice tells people that AI is involved in a service.
3DefinedA published statement describes how AI is used and is reviewed at least yearly. Decisions about people that AI contributes to are inventoried; affected people are told and have a documented path to human review; and the privacy policy explains automated decisions where required.A published statement reviewed in the last 12 months, an inventory of AI-assisted decisions about people, a documented human review path for each decision type, and a privacy policy that covers automated decisions where the law requires it.
4ManagedDisclosures and review paths are tested (for example through complaint sampling or mystery shopping). Requests for human review and their outcomes are tracked and reported, and the statement is updated whenever the register changes materially.A review-request log with outcomes, and evidence that a register change triggered an update to the statement.
5OptimisingExplanations are generated from decision records at the point of decision. Review outcomes feed model and process changes, and transparency measures (requests, reversals, time to review) are reported to the board or published.Point-of-decision explanations generated from records, and a review outcome that changed a model or process.

Evidence to request

Documents

  • Published transparency statement and review history
  • Privacy policy section on automated decisions
  • Notices shown to affected people
  • Human review procedure

Records of it operating

  • Inventory of AI-assisted decisions about people
  • Review-request log and outcomes
  • Complaint or mystery-shopper sampling results

In systems

  • Explanation generation from decision records (optional)

In the evidence request: ER-3-04.

Interview probes

  • Legal, privacy and records leadWhich decisions about people does AI contribute to, and how would someone affected find out?
  • Use-case owner or business leadIf a customer or citizen asked why a decision went the way it did, what would you show them?
  • Frontline staffWhat do you say when someone asks whether AI was involved in a decision about them?

Staff survey items: S19 I know when I must tell a customer or member of the public that AI was involved in their matter.

Red flags

  • The statement lists tools rather than uses
  • No inventory of automated decisions with the 10 December 2026 privacy obligation approaching
  • Contestability means ‘contact us’ with no human review step

Standards crosswalk

ISO/IEC 42001
Annex A.8
EU AI Act
Article 50Article 86
NIST AI RMF
GOVERN 4MEASURE 2

Government and organisation editions

Australian Government edition

Policy v2.0 requires a published AI transparency statement, reviewed at least yearly, with the DTA told of every change. From 10 December 2026, APP entities, including agencies, must explain in their privacy policies the automated decisions, using personal information, that significantly affect people’s rights or interests. The Automated Decision Making Better Practice Guide sets out lawful authority, transparency, reasons and review paths. Policy v2.0 also requires staff and public pathways to raise AI safety concerns by 15 December 2026.

Dated obligations

  • 10 December 2026Automated decisions using personal information explained in the privacy policy (Privacy Act)Met at level 3 (Defined) or higher

References

Organisation edition

From 10 December 2026, APP entities must explain in their privacy policies the automated decisions, using personal information, that significantly affect people’s rights or interests. Organisations operating in the EU must meet the EU AI Act’s transparency obligations for systems that interact with people or generate synthetic content.

Dated obligations

  • 10 December 2026Automated decisions using personal information explained in the privacy policy (Privacy Act, APP entities)Met at level 3 (Defined) or higherApplies to APP entities under the Privacy Act
  • 2 August 2026EU AI Act transparency obligations for AI that interacts with people or generates content (Article 50)Met at level 3 (Defined) or higherApplies where the EU AI Act reaches you

References

In Governing Intelligence

Initiatives that lift it

  • I-307 Inventory decisions about people and explain them in the privacy policy (level 1 to 2)
  • I-308 Publish the transparency statement and design human review paths (level 2 to 3)
  • I-309 Test disclosures and track review requests (level 3 to 4)

Rate 3.4 in the assessmentBack to top

Criterion 3.5

Incident and near-miss management

AI incidents and near misses are defined, reported, triaged by severity and handled through a runbook that includes the authority to halt a system, notifications where required, and review for lessons.

Why it matters

AI fails differently from ordinary software: a model can act confidently on bad data at scale. An organisation that cannot recognise an AI incident, halt the system and learn from near misses will meet the same failure repeatedly.

The five levels

LevelWhat it looks likeTo score this level
1InitialThere is no definition of an AI incident. Problems are handled as IT tickets or complaints and are not recognised as AI-related.The starting point: no test.
2RepeatableAI incidents go through the general IT or privacy incident process, which does not capture AI-specific causes such as model behaviour, data drift or prompt injection. Near misses are not recorded.AI-related problems can be found in an incident or ticket system.
3DefinedAn AI incident definition, severity scale and runbook exist, including who can halt a system and the notification map (privacy regulator, cyber authorities, other regulators). A register records incidents and near misses, and the runbook has been rehearsed at least once.A definition, severity scale and runbook naming halt authority and notification routes; an incident and near-miss register; and a rehearsal record from the last 12 months.
4ManagedIncidents and near misses are reported at a healthy rate and reviewed monthly for patterns. Post-incident reviews identify the control that failed and track corrective actions to closure, and time to contain is measured.Minutes of monthly pattern reviews, a post-incident review with corrective actions closed, and measured time to contain.
5OptimisingLessons change controls across the portfolio, not just the affected system. Monitoring raises incidents automatically, and lessons are shared externally where appropriate.A control change applied across several systems because of an incident lesson, and incidents raised automatically by monitoring.

Evidence to request

Documents

  • AI incident definition and severity scale
  • Incident runbook with halt authority and notification map

Records of it operating

  • Incident and near-miss register
  • Rehearsal record
  • Post-incident reviews
  • Corrective action tracker

In systems

  • Monitoring rules that raise incidents automatically

In the evidence request: ER-3-05.

Interview probes

  • CISO or security leadWalk me through the first hour if an AI system produced harmful outputs at scale.
  • Frontline staffIf an AI tool gave you a wrong or unsafe answer at work, what would you do?
  • Risk, compliance and internal auditHow many AI near misses were recorded last quarter, and what changed because of them?

Staff survey items: S15 If an AI tool produced something wrong, biased or unsafe, I know how to report it.; S17 When problems with AI are reported here, something changes as a result.

Red flags

  • Zero near misses recorded, which usually means nobody is looking or nobody feels safe saying so
  • The runbook has never been rehearsed
  • Nobody is named as authorised to halt a system

Standards crosswalk

ISO/IEC 42001
Clause 10.2
EU AI Act
Article 73
NIST AI RMF
MANAGE 4

Government and organisation editions

Australian Government edition

By 15 December 2026, Policy v2.0 requires a pathway for staff to report AI safety concerns, including AI incidents, and clear processes to address AI incidents aligned with the agency’s ICT incident management, with remediation overseen by an appropriate governance body or senior executive. Notification duties continue to apply: the OAIC and affected individuals under the Notifiable Data Breaches scheme, ASD for cyber incidents, and the DTA where the policy requires it (for example when a use case is re-assessed as high-risk).

Dated obligations

  • 15 December 2026AI incident processes overseen by a governance body (Policy v2.0)Met at level 3 (Defined) or higher

References

Organisation edition

The NIST AI RMF expects incidents and errors to be communicated and tracked after deployment, and ISO/IEC 42001 requires corrective action. Providers of high-risk systems in the EU must report serious incidents. APRA-regulated entities have incident notification duties under CPS 230.

References

In Governing Intelligence

In the AI governance assessment: Incidents and near misses are reported, reviewed and fed back into policy.

Initiatives that lift it

  • I-310 Adopt the AI incident runbook and register, and rehearse it (level 1 to 3)
  • I-311 Review near misses monthly and close corrective actions (level 3 to 4)

Rate 3.5 in the assessmentBack to top

In Governing Intelligence

The framework chapters that describe the practice this domain measures, and the governance assessment questions that test it at a higher level.

In the AI governance assessment: Impact assessments are current for every in-scope use case. A written risk-appetite statement names our risk tiers and who decides. Incidents and near misses are reported, reviewed and fed back into policy.

This assessment is not legal advice and is not a certification. Obligations and dates were checked against official sources on 15 October 2026; confirm them against the source instruments, which can change.