A launch checklist that shows its evidence

Before a system launches, someone ticks a checklist. This chapter turns that checklist into code, runs it over a company's 16 AI systems, reads the evidence behind the passes, and then asks whether the list would have caught real failures, measured with two raters.

The source is Lab 17 (Governance), script 03_launch_gate.py, recorded in October 2026; the video uses the recorded run, no new calls. The company, Lantern Outfitters, is made up; its 16 AI systems are real code, the labs of this course. The 12 failures are real, public cases with sources; which items would have caught each is a judgment, ours and a model's (gpt-5.4-mini, recorded answers). Chapter 49 built the list of systems and chapter 50 gave each a risk level; this one decides whether a system may launch.

Eight items, and what counts as evidence

The checklist has eight items, like a pilot's pre-flight check:

1. A risk assessment, written down: what could go wrong.
2. A model card: a page saying what the system is for, and where it fails.
3. Approved data: the data sources it uses, classified and approved.
4. Guardrails, tested: checks that stop bad requests and bad answers.
5. An owner, or escalation path: the person to call when it breaks.
6. Monitoring and alerts, to notice when it does.
7. A legal and compliance sign-off.
8. A rollback plan: a way back to the last good version.

A tick is cheap. On a pre-flight check, each tick has to point at a gauge. So in the lab each item is a small piece of code that looks for evidence, a file or a line someone could open, and returns it:

# checklist.py (abridged)
CHECKS = {
    "risk":       lambda lab: _exists(lab, "RISK*.md") or _text(lab, r"\brisk (level|tier)\s*:", readme_only=True),
    "guardrails": lambda lab: _exists(lab, "tests/test_*.py"),
    "escalation": lambda lab: _exists(lab, "OWNERS*", "CODEOWNERS") or _text(lab, r"^\**owner\**\s*:", True),
    "legal":      lambda lab: _exists(lab, "SIGNOFF*", "APPROVALS*"),
    ...
}

def evaluate(lab: str) -> dict[str, str | None]:
    """{item: the evidence found (a file, or the matching line), or None}."""
    return {k: fn(lab) or None for k, (_, fn) in CHECKS.items()}

Take the refund agent. For guardrails, the checker found a test file; for approved data, a pinned list of the exact data files it uses (data/manifest.json). For one extra item, measured results: "78/90 = 87% (78–92%)", the share of its test runs that succeeded, with a range. Owner: nothing. Risk assessment: nothing. Sign-off: nothing.

All sixteen systems

The lab runs the checks over all 16 systems. Rows are systems, columns are items; the last four columns are items added later in this chapter. Click any square to see what the checker found, or a system's name for its full list:

What it shows:

Engineering work left evidence. The three items that are decisions by people had none: no system's folder records who owns it, what risk was accepted, or who signed off. No system had all eight. By the checker, the best, the docs assistant with tracing, had five.

A word, not the work

Finding a word isn't the same as finding the work. Each check prints the line it matched, so the video read every line behind three of the rarer items:

• Tested across groups of people: 2 passes, 0 about people. One was an exercise title, "Bias, Fairness & Drift", that compares search across article topics. The other was a sentence pointing to a different guide.
• Re-tested on every change: the first version passed 6, 1 was real. Only the release judge has a CI workflow that runs its tests on every pull request. The others matched a word, like "workflow" or "regression set": a printed trace line (invoke_workflow support_assistant), a preview of another project, an exercise idea, a sentence of advice.
• A person approves big decisions: 3 passes, 2 real. Two are approval steps in agent code (exercise_5_approval, resume_after_approval). The third is a file of people's ratings, named human_review.json.

The lab fixed the re-test check: it now looks for a workflow file under .github/workflows that runs on pull requests or pushes and names the system's folder. One system passes. Switch between the two versions in the grid above.

So a missing item is certain. A found one still needs a person to read it. Some rollback passes look like words too (a function's fallback= argument, a "What's next" paragraph); the video didn't count those, and neither does this chapter.

Twelve real failures

Second question: would this checklist have caught failures that really happened? The lab took 12 public cases, each with a source, and marked which items would have caught each one before people were harmed. Two examples:

• Air Canada's support chatbot told a grieving customer he could claim a bereavement refund after travel, against the airline's policy. A tribunal made the airline pay (Moffatt v. Air Canada, 2024 BCCRT 149). Our labels: guardrails tested, a legal sign-off, and an evaluation in the real setting.
• Robodebt, in Australia, raised welfare debts by averaging people's yearly income across fortnights. It was unlawful and often wrong; a Royal Commission condemned it (report, July 2023). Our labels: a legal sign-off, a person reviewing decisions, and an evaluation on the real population.

By our labels, every one of the 12 had at least one of the eight items that would have caught it. But in 10 of 12, our labels also include an item the list didn't have. There are four of those:

"Guardrails built and tested" doesn't say against what. The added items name it.

Two raters

Those labels are a judgment. So the lab had a second rater, gpt-5.4-mini, label the same 12 cases: 12 cases × 12 items, each a yes or a no, 144 judgments. Pick a case to compare, and change the model's labels to see how the numbers move:

• Raw agreement: 80%. Is that good? Not really.
• Chance agreement: about 63%. For most items both raters said no (our yes rate 25%, the model's 24%). Two raters who mostly say no agree a lot by chance alone.
• Cohen's kappa: 0.46, moderate. Kappa takes chance out: (0.80 − 0.63) ÷ (1 − 0.63). One is perfect agreement, zero is no better than chance, and 0.4–0.6 is usually called moderate.

The disagreements had a pattern. The model never once picked "risk assessment" or "model card", and picked "evaluated on the real population" for 9 cases to our 4. Neither rater is the truth; the point is that people applying the same checklist will differ, so each item needs a written reason when it applies.

What to do

• Make each item return evidence: a file or a line a person can open. Never a bare tick.
• Refuse what's certain. No owner, no risk level, no sign-off: no launch. A missing file isn't a judgment call.
• Have a person read every pass. A keyword check finds a word; a person decides whether it's the work.
• Prefer evidence that's hard to fake by accident: the CI file itself, not the word "workflow".
• Test the checklist against real failures, and add the items they ask for. Here: groups of people, the real population, a person on big decisions, re-testing on every update.
• Ask for a written reason per item, and measure how far two reviewers agree with kappa, not raw agreement.

Caveats

• Made-up company, real code. The 16 systems are this course's labs, and nobody had been asked to record an owner, a risk level or a sign-off for them. The pattern, engineering evidence present and people decisions absent, is the lesson; the zeros aren't a verdict on a real team.
• Checks find artifacts, not quality. A README with a model table passes "model card" whether or not the table is right.
• Our reading covered three items. Monitoring and rollback passes weren't read line by line; some rollback passes look like words.
• The labels are judgments about history. "Would have caught it" can't be proven; the sources are listed so it can be checked.
• One model as the second rater, at one time. A person as the second rater, or several, could agree more or less.

The checklist

1. Write each launch item as a check that returns its evidence.
2. Block a launch on missing owner, risk record or sign-off.
3. Show the matched line for every pass, and have a person read it.
4. Look for the real artifact (a CI file, an OWNERS file), not a keyword.
5. Add items for groups of people, the real population, human review of big decisions, and re-testing on every update.
6. Write down why each item applies, and check two reviewers' agreement with kappa.

Practice: a gate that shows its evidence

Write the lab's keyword check and the stricter CI-file check, count what 16 systems have evidence for, find the best system, and compute raw agreement, chance agreement and kappa. The tests use the recorded run and expect the lab's numbers: 0, 0 and 0 for risk, owner and sign-off; 10, 9, 7, 7 and 1 for the engineering items; the traced docs assistant at 5 of 8; 12 and 10 of the cases; 80%, 63% and 0.46.

Quiz

Check your understanding

Try each question before opening its answer.

Next up: Chapter 52, incident response: when something does go wrong, how bad is it, and who must be told by when?