Scoring method

Principles

1. Score on evidence, not assertion. A claim that nobody can back with an artefact, a record or observed practice is capped at Level 2.
2. Assess the organisation, not the AI team. The unit of assessment is the enterprise (or the chosen function): executives, business areas, technology, security, privacy, data, people functions and the AI team.
3. Every criterion has a crisp rating rule: its level is the highest level whose pass test is met. Descriptors explain what a level looks like; pass tests decide it.
4. Validate before you write. Scores are played back to the people who gave the evidence, and corrections arrive as new evidence, not negotiation.

Scoring a criterion

A criterion’s level is the highest level whose pass test is met, provided the pass tests of every lower level are also met. Level 1 has no pass test. Use the level descriptors to recognise practice and the pass tests to decide the score.

Evidence confidence and caps

Every score carries a confidence. Confidence is not decoration: it caps the score.

The caps

From criteria to a domain score

A domain’s score is the mean of its scored criteria, capped at the lowest scored criterion plus 1.0, rounded to one decimal place.

domainScore = round1( min( mean(criterionScores), min(criterionScores) + 1.0 ) )

Round half away from zero to one decimal place (2.25 becomes 2.3). Compute from unrounded criterion scores.

• Criteria marked ‘not known’ are excluded.
• If one or two criteria in a domain are not known, the domain score is computed from the rest and flagged ‘partial coverage’.
• If three or more of the five are not known, the domain score is not reported and the domain is shown as ‘insufficient evidence’.

Worked examples

Why a capped mean

• A plain mean rewards strength in some criteria and hides a missing foundation. A domain with four criteria at Level 4 and no register at all would score 3.4, which no auditor would accept.
• A plain minimum (weakest link) is too harsh and unstable: one criterion decides everything, real progress elsewhere earns no credit, and assessors are tempted to score generously to avoid a collapse.
• A median of five ignores the two weakest criteria entirely, so a domain at 1, 1, 4, 4, 4 would score 4.
• The capped mean gives credit for progress while preventing any domain from being reported more than one level above its weakest criterion. It can be stated in one sentence to a board.

Every domain result shows the domain score, the uncapped mean and, where the cap applied, the criterion that caused it.

The overall profile

Overall score

The mean of the reported domain scores, rounded to one decimal place. Not reported if fewer than six domains have scores.

Bands

The overall band, and each domain’s band, is the level whose number is the whole part of the score (2.0 to 2.9 is ‘Repeatable’). A score whose decimal part is 0.7 or more is described as ‘approaching’ the next level.Rounding 2.5 up to ‘Defined’ would describe an organisation that has not met the Level 3 pass tests as if it had. The whole-part rule keeps the label honest; the ‘approaching’ phrase gives credit for progress.

Strongest and weakest

Name the two highest and two lowest domains. Ties are broken by the higher (or lower) uncapped mean, then by domain number.

Foundation flag

If any foundation-importance criterion scores below 2.0, the report’s headline must say that foundations are not in place, whatever the overall score.

Layer scores

Each layer (foundations, ways of working, practice) is reported as the mean of its domain scores.

Responsible AI overlay

Each responsible AI dimension is scored from the criteria mapped to it, using the same capped-mean rule as domains. The overlay adds no questions: it re-reads the 40 criteria through a responsible AI lens and crosswalks them to Australia’s AI Ethics Principles, ISO/IEC 42001 and the EU AI Act.

Shape

Importance weights

A criterion’s importance weight is its base weight multiplied by every applicable context multiplier, capped at 4.5.

Context multipliers

The weight is capped at 4.5.

Setting targets

Targets are set per domain by leaders in the target-state workshop, then translated into per-criterion targets that can be overridden.

Balanced: where ai contributes to decisions about people or ai acts with supervised autonomy, the target for 3. Risk, assurance and transparency, 5. Platform, architecture and security and 6. Design and engineering practice rises to 3.5.

Time frames: 12 months or 24 months.

1. A target is never below the current score.
2. Default pace: a domain target may sit at most 1.0 above the current domain score per 12 months of time frame. Up to 1.5 per 12 months is allowed when dedicated resourcing is confirmed, and is labelled ‘stretch’. Anything larger needs a written reason recorded in the workshop outputs.
3. Layer rule: no domain in the ways-of-working or practice layers may have a target more than 1.0 above the lower of the D1 and D2 targets.
4. Criterion targets default to the whole-level part of the domain target. Where the domain target’s decimal part is 0.5 or more, foundation and core criteria take the next level up and enabling criteria stay at the whole level, because pass tests are whole levels. Leaders may lower a criterion target (not every criterion needs Level 4) or raise it, within the pace rule for that criterion (at most two levels above current per 12 months).
5. Obligation floor: where an edition obligation applies to a criterion, its target is at least the obligation’s minimum level, due by the obligation date, whatever the ambition preset.
6. Function scope: targets for enterprise-level criteria are inherited from the enterprise and shown as dependencies, not set by the function.

Gaps

Criterion gap

criterionGap = max(0, criterionTarget minus criterionScore). For a ‘not known’ criterion, the gap is computed against an assumed score of 1.0 and flagged ‘verify before acting’.

Domain gap

domainGap = max(0, domainTarget minus domainScore).

Overall gap

The mean of domain gaps, one decimal place.

What the results show

• Heat map: the 40 criteria coloured by current level, each cell marked with its target level, with gaps of two or more levels outlined.
• Biggest gaps chart: the ten criteria with the highest priority score (below), showing current and target.
• Recommendations per gap: for each criterion with a gap, the initiatives selected to close it (see roadmapLogic).

Prioritising the gaps

priorityScore = criterionGap x importanceWeight x feasibility

How readily the gap can be closed now, judged from the initiative that would close its next step.

An assessor or team may override feasibility by one step with a recorded reason (for example funding already approved, or a key person unavailable).

Order

1. Class A, must-do: the criterion is below the minimum level of an applicable edition obligation due within 12 months (or already overdue).
2. Class B, foundation: the criterion has foundation importance.
3. Class C, value: everything else.
4. Within a class, sort by priority score (highest first), then by lower effort, then by criterion number.