Judging AI output

The problem with a fluent wrong answer

In module one you learned that a model predicts plausible text. A hallucination is what happens when that plausible text is false: a citation to a paper that does not exist, a statistic that was never measured, a quote nobody said, a confident summary of a document that says the opposite. It arrives in the same polished, self-assured voice as everything correct, which is exactly why it slips through.

The instinct to fight is the one that says "it sounds right, so it is right." Fluency is not accuracy. A model can be eloquent and completely mistaken in the same sentence.

When do you actually need to verify?

You do not have to fact-check every sentence, that would make AI useless. The trick is to spend your attention where it matters. Two questions decide it: how much would a mistake cost, and does the output make specific, checkable claims.

How to check, quickly

Verifying does not mean a research project. A few fast moves catch almost everything:

• Ask for the source, then actually open it. If a model cites something, follow the link. Hallucinated sources fall apart the moment you click.
• Check the specifics. Numbers, names, dates, quotes, legal or medical claims, and citations are the highest-risk items. Confirm those first.
• Cross-check a different way. Confirm a key fact against a system or document you trust, not by asking the same AI again.
• Read the document yourself when AI summarises something important. A summary can quietly drop the one caveat that changes the meaning.

Watch for bias, especially about people

A model learns from vast amounts of human text, and it absorbs the patterns in that text, including the unfair ones. That can show up as skewed assumptions about roles, groups, or names. It is subtle and it is not always obvious.

The practical rule: be extra careful when AI touches decisions about people, such as hiring, performance, lending, or anything that affects someone's opportunities. Use AI to draft and inform, keep a human making the actual judgement, and be ready to explain the reasons in your own words.

Stay the human in the loop, not the rubber stamp

Here is the part that matters most for your own accountability. When you send an AI-drafted email, publish an AI-written summary, or paste an AI answer into a report, you are the author. Not the model. The responsibility, and the credit, are yours.