Incident response: who decides how bad it is?

An internal assistant told an employee that a colleague was off work for knee surgery. That's health data, and the company's incident policy makes exposing it the most serious level, P1. Asked to rate it, a model said P2. This chapter tests who should decide how bad an incident is, then follows what comes next: the clocks that start when the company knows, and the questions to ask an outside company before it gets your data.

The source is Lab 17 (Governance), scripts 04_incidents.py and 05_vendors.py, recorded in October 2026; the video uses that run's recorded replies, no new calls. The company, Lantern Outfitters, its 24 incident reports and the three vendors are made up; every rating and answer is a recorded call to gpt-5.4-nano or gpt-5.4-mini. This is the last chapter on governance: chapter 53 listed every AI system, chapter 54 gave each a risk level, and chapter 55 built a launch checklist. The incident runbook and vendor questionnaire quoted here are two of the four documents the lab's script 06 writes.

Severity: P1 to P4

The first decision in an incident is a label: how bad is it? It works like triage in an emergency room. P1 means call everyone, now; P4 can wait. The lab's policy, written the way a runbook would state it:

The label decides who gets called. In the runbook, P1 pages the on-call engineer, the system owner, security and legal now, and informs an executive within an hour; P3 is a ticket for the owning team. So rating an incident too low is the mistake that matters: the people who should know don't find out. Too high costs an extra phone call.

Two ways to rate an incident

The lab wrote 24 made-up incident reports: 5 P1, 8 P2, 5 P3, 6 P4. For each, the facts were coded by hand (was anyone hurt, which data, how many people, how much money) and the right level is the policy applied to those facts. Then two methods, on two models, three tries each:

• Name it. The model reads the policy and the report, and replies with a level and a one-sentence reason.
• Facts first. The model doesn't pick a level. It answers nine questions about the report (was anyone hurt? was special-category or financial-account data exposed? how many people? how much money?), and a few lines of code apply the policy's table to the answers. A checklist, instead of a gut call.

# incidents.py
def severity(f: Facts) -> str:
    if f.physical_harm or f.discrimination or f.special_or_financial_data or f.people_affected > 10_000:
        return "P1"
    if f.personal_data_exposed or f.money_usd > 10_000 or f.wrong_commitment or f.public or f.people_affected > 1_000:
        return "P2"
    if f.reached_users or f.money_usd > 0:
        return "P3"
    return "P4"

Pick a report and see all twelve answers, the models' reasons, and the facts behind the facts-first rating:

The results

First try, on the 24 reports:

• Not more accurate. Facts first got 3 more right on the small model and 2 fewer on the bigger one. On 24 reports neither difference is significant (paired test, p = 0.45 and 0.69). The lab's draft expected facts first to win here; it didn't.
• Fewer rated too low. One on each model, against 5 and 2 when the model named the level.
• Its mistakes lean the safe way. Facts first on mini rated 5 reports too high: the CEO's salary, two refund incidents, a server outage nobody saw, and an analyst seeing payroll. Each one is an extra phone call, not a missed one.
• More consistent. It found 4 of the 5 P1s on the first try and changed its rating between tries on only 1 report.

And the one report facts first rated too low, on both models, was the knee surgery.

The mistake points at a word

Facts first shows you exactly which answer went wrong. For the knee surgery, one question decides the level: was special-category or financial-account data exposed? Both models answered no, all 6 times, so the table gave P2. Naming the level fared no better: all 6 of those answers said P2 too, 12 of 12 in all. The small model's own reason even said it: "disclosed personal health information (special-category data) … so the incident fits P2".

"Special-category" is legal language. In data protection law (GDPR Art. 9) it covers health, religion, sexual orientation and a few other sensitive kinds of data. The policy uses the term and never says what it covers. Why the models answered no isn't known yet; the next test is to spell the categories out in the policy and run it again.

A mistake like that went the other way, too. The CEO's salary was shown to an outside contractor. By the policy that's P2: personal data, but not card or bank details. Facts first on mini answered yes to the same question, 3 times out of 3, so the table said P1. Naming the level, mini once gave the reason "the leaked CEO salary is financial-account data". The lab's runbook now spells it out:

Known ambiguity, found by testing: 'financial-account data' means card and bank account numbers, not salaries or refund amounts.

That's the use of facts first beyond the numbers: a wrong rating points at one answer, and the answer points at a word in the policy. A severity table needs testing like code does, and this is how you find its unclear words. (The runbook doesn't spell out special-category data yet; the knee surgery says it should.)

The clock

Even a perfect label is only the start. Notification rules start clocks, and the clock starts when the company becomes aware, so someone has to write that moment down in the incident record.

The lab's case: a logging change wrote 8,000 customers' full card numbers into log files. Some of them live in the EU, and the company's shares trade in the US. It finds out on Friday 23 October 2026, at 4 pm.

Other facts start other clocks. A US hospital assistant leaking 320 patients' health data gives 60 days to tell the patients (Tuesday 22 December), and HHS gets it in an annual log by 1 March 2027 because it's under 500 people. A high-risk hiring AI's widespread error, once the EU AI Act's high-risk rules apply (2 December 2027 for hiring), gives 2 days to tell the market surveillance authority. These come from the lab's teaching version of the rules, not legal advice.

Move the moment of awareness and watch the deadlines:

72 hours from Friday afternoon is Monday afternoon, with 48 of those hours on the weekend. So the runbook needs someone on call on Saturday who can decide to notify, not just someone who can restart a server.

Three vendors

The 72 hours also run when a vendor leaks your data: an outside company, like an AI provider you send customer data to. (GDPR Art. 33(2) makes the vendor tell you "without undue delay", so you can meet your own 72 hours.) So before handing data over, the company asks ten questions:

"Not stated" counts as an answer, usually the worst plausible one: a vendor that won't say whether it trains on your data is treated like one that does.

The lab wrote trust pages for three made-up vendors. Northwind is specific on every point (EU or US region, no training, 30-day retention with a deletion certificate, SOC 2 Type II, 48-hour breach notice). Brightline says "enterprise-grade security" and "SOC 2 compliant", and may train on your data. Kestrel is an honest FAQ with weaker terms: SOC 2 Type I, 90-day retention, training on by default for self-serve accounts, US only.

The model reads, the rules decide

Each model filled in the questionnaire from each page, three times: 3 vendors × 10 questions × 3 tries, 90 answers per model.

Where a page says nothing, neither model made an answer up: Brightline's page leaves five questions unanswered, and mini said "not stated" every time. All six of mini's mistakes were on Kestrel's page, two sentences misread on every try:

• "We will notify affected customers without undue delay" read as not stated. The right answer is "no fixed time": a condition, not a blocker.
• "Yes, contact support and we will delete it within 30 days" read as confirmed deletion. The page never says they confirm it.

Then the rules turn the answers into a decision. From the pages, read by a person: Northwind approve, Brightline reject (training on customer data, no breach notice, no data processing agreement), Kestrel approve with 8 conditions. From mini's answers, Kestrel became reject: a missing breach promise is a blocker, so one misread sentence turned an approval into a rejection. Its other mistake went the risky way, giving Kestrel credit for a promise the page doesn't make.

A cautious misreading costs you a vendor; a careless one costs the company. Either way the fix is the same: the model must quote the sentence behind each answer (this test didn't ask it to), and a person checks those sentences before anyone signs, or says no.

Also in the lab

These parts of Lab 17 are in the lab's README and result files (its own run, October 2026) but not in the video's main story:

• The small model on vendors. gpt-5.4-nano made up no missing answers either, but it upgraded vague wording to a firmer answer: Brightline's "industry-standard encryption" became "both, in transit and at rest", and Kestrel's published subprocessor list became "published, with notice". The README's "made up" count doesn't measure that, which is another reason to quote the source sentence.
• Confidence intervals. On 24 reports, 71% right is 51–85% and 83% is 64–93%; on 90 answers, mini's 93% is 86–97%. Small tests show direction, not exact rates.
• The four documents. Script 06 writes the governance pack from the results: the AI system registry (chapter 53), the pre-launch checklist (chapter 55), the incident runbook (severity table, response times per level, the clocks, a postmortem template) and the vendor questionnaire with the three reviews.
• The runbook's flow: detection, record the moment of awareness, severity from the facts, containment (disable the feature, fall back, add a human gate), root cause, fix plus a regression test built from the incident, a blameless postmortem, then notifications by the deadlines.

What to do

• Write the severity policy as conditions on facts, and have responders write the facts down before picking a level.
• Let the model answer the fact questions; let code apply the table. Count "rated too low" separately from accuracy.
• Test the policy on past or made-up incidents. Where ratings disagree, look for the word: "special-category", "financial-account".
• Spell out legal terms in the runbook, with examples of what counts and what doesn't.
• Record the moment of awareness in the incident record, and keep someone on call over the weekend who can decide to notify.
• Treat "not stated" as an answer in vendor reviews, and have the model quote the sentence behind every answer for a person to check.

Caveats

• Made up and small. 24 reports and 3 vendors, written for the lab, with the right answers coded by one person. Real incidents are messier, and their facts are often unknown at first.
• Two models, three tries. Neither accuracy difference is significant; "fewer too low" is 1 against 2 to 5 on 24 reports.
• The knee surgery isn't fixed. Facts first showed which answer was wrong; spelling out "special-category" is the next test, not a result.
• The clocks are a teaching version of GDPR, the SEC's rule, HIPAA and the AI Act. Public holidays, US state laws and sector rules aren't modeled. Not legal advice.
• The vendor pages are short and invented. Real trust pages, contracts and DPAs are longer, and some answers live only in the contract.

The checklist

1. A written severity table whose conditions are facts, not feelings.
2. A fact sheet the responder fills in before choosing a level.
3. A test set of incidents with their right levels, rerun when the policy or the model changes.
4. "Rated too low" tracked as its own number.
5. Legal terms defined in the runbook.
6. An awareness timestamp in every incident record, and a weekend on-call who can decide to notify.
7. A vendor questionnaire with blockers and conditions, "not stated" counted, and every answer quoted from the source.

Practice: from facts to decisions

Write the policy's severity table, score the recorded ratings, count how often they changed between tries, compute the Friday clocks, and turn vendor answers into a decision. The tests use the lab's 24 reports with their hand-coded facts, the recorded ratings from both methods and models, and Kestrel's answers as a person and as gpt-5.4-mini read them, and expect the lab's numbers: 17, 20, 20 and 18 right; 5, 2, 1 and 1 too low; Monday and Thursday at 4 pm; and Kestrel going from approve with 8 conditions to reject.

Quiz

Check your understanding

Try each question before opening its answer.

Next up: That's the end of the Governance section, and of the published chapters. The next section, Self-improving systems, is coming soon: tests that write themselves, models checking their own answers, and loops that chase the wrong reward.