Running an assessment

An assessor-led assessment takes about six weeks from kickoff to read-out. The model’s toolkit supplies every instrument: the evidence request, the interview guides, the staff survey and five workshops. The steps below follow the order the toolkit sets.

Timeline

1. Week 1

   W1 Kickoff and ambition sounding (120 minutes)

   Confirmed scope and edition, Evidence request issued with owners and due dates and Interview schedule

2. Week 1

   Evidence request issued

   Issue in week 1. Where an item does not exist, say so; its absence is recorded as evidence and is not chased. Provide documents in their current version; provide records for the last 12 months unless stated.

3. Weeks 2 and 3

   Interviews by role

   Interviews run 30 to 60 minutes with two assessors: one leads, one codes each substantive answer to a criterion and an evidence confidence as it is given. Interviews test and extend what the documents show; they do not replace them. Run them in weeks 2 and 3, after the evidence request has been issued.

4. Weeks 2 and 3

   Staff survey open

   Open to all staff in scope for 7 to 10 days. Anonymous; no free-text question asks for names. Use a survey tool the organisation already approves.

5. Week 3

   W2 Evidence review and lifecycle walk-through (150 minutes)

   Observations coded to criteria in D2, D3, D4, D6 and D7, List of unregistered AI found and Follow-up evidence actions

6. Week 4

   W3 Scoring moderation (180 minutes)

   Agreed score, confidence, evidence IDs and reason for all 40 criteria, Moderation log and Domain and overall results computed

7. Week 5, after playback of validated scores

   W4 Target state and roadmap planning (210 minutes)

   Domain targets with ambition and time frame, Criterion target overrides with reasons and Prioritised initiative list with horizon and owner

8. Week 6

   W5 Read-out and decisions (90 minutes)

   Recorded decisions, Endorsed or amended targets and Funded first horizon

Roles

Who takes part, and how. Interview guides exist for the roles marked interviewed; workshop attendance and evidence-request ownership come from the toolkit.

Scope and edition

An assessment covers either the whole enterprise or one business function. The criteria are the same in both; what changes is the evidence sampled and who owns the resulting initiatives.

• Whole enterprise: All business units, all AI however acquired. The default, and required for the government edition’s obligations view.
• One business function: A named function such as finance, marketing, customer service, human resources or a program area. Use when a function is moving faster than the enterprise, or to pilot the assessment before an enterprise-wide run.

In a function-scoped assessment, sample only the function’s use cases, staff and records for ‘both’ and ‘function’ criteria, and state the scope on every generated document.

Choose the edition at kickoff (workshop W1). The government edition carries the Commonwealth obligations and their dates; see Editions and obligations.

Gathering evidence

Evidence comes from four sources, and the strongest scores rest on more than one:

• Documents and records from the evidence request (41 items, each with an owner and a format).
• Interviews with 9 roles, coded to criteria as they happen: Code each answer as criterion number plus confidence: ‘evidenced’ only when the evidence is seen or provided afterwards; otherwise ‘asserted’. Log the follow-up evidence request with an owner and date.
• The staff survey (25 items), which can corroborate, flag or cap a score but never raise it.
• Observation in the workshops, especially the lifecycle walk-through of a real use case (W2).

Validation and moderation

1. Two assessors score each criterion independently before discussing it (assessor mode).
2. Differences of more than one level are resolved by seeking more evidence, never by averaging. Differences of 0.5 to 1.0 are resolved in moderation with the reason recorded.
3. Every score of Level 3 or above cites at least one evidence ID, and every pass test claimed names the evidence that meets it.
4. Survey flags are reviewed criterion by criterion in moderation, and the outcome is recorded.
5. Domain scores are played back to the people who provided the evidence before any report is drafted. Corrections arrive as new evidence logged in the evidence register, not by email or negotiation.
6. A quality reviewer who did not score re-checks a sample of 20 percent of criteria (at least eight, covering every domain) against their pass tests. If more than two sampled scores change, the whole domain is re-moderated.
7. Assessors do not score areas they manage or have advised on in the last 12 months.
8. Any score that changes after playback records the new evidence and who supplied it.

Moderation

Turn two independent scores and the survey into one agreed, evidenced score per criterion.

1. Show both assessors’ scores, confidence and evidence IDs side by side; start with the largest differences.
2. For each criterion, read the pass tests aloud from the lowest unmet level and check the evidence against them.
3. Apply caps (confidence, currency, survey, function scope) and record which applied.
4. Record the agreed score, the confidence, the evidence IDs and a one-sentence reason.
5. List open evidence actions with owners and dates; criteria with open actions are scored on the evidence held now.

Calibration examples

Worked cases that show how the pass tests and caps decide a score. Review two together before moderation.

Running it as a self-assessment

One respondent chooses, for each criterion, the level descriptor that best matches the organisation and states their confidence. No assessor sees evidence.

Confidence is asked in plain words: We could show the evidence today, We believe so, but have not checked or Not sure.

Every output is labelled ‘Self-assessed and not validated’. Generated text uses ‘you reported’ rather than ‘the assessment found’.

The free assessment applies every rule on this site, from the caps to the roadmap, and produces the same reports as an assessor-led engagement, labelled as self-assessed.