Tamper-evident audit logs: consistent isn't original

An audit log is the record behind every answer an assistant gives: who asked, what they were shown, what it said. This chapter tests whether anyone could change that record without it showing, three ways: numbered entries, a chain of fingerprints, and the same chain with copies kept elsewhere. The middle one is the trap. It looks like proof, and it catches nothing from someone who covers their tracks.

The source is Lab 16 (Compliance), script 03_audit_log.py, Exercise 1, recorded in October 2026; the video uses that run, replayed trial by trial, with no new calls. The log holds chapter 49's 376 requests with the permission filter on (8 made-up employees × 47 questions), each with its real recorded answer from gpt-5.4-nano. The tampering is a test script: each kind of change made 200 times at random places. Chapter 49 decided who may see which documents and chapter 50 what the assistant may do at all; this chapter keeps the record of what happened. The script's other two exercises, losing entries in a crash and erasing one person's data, aren't in the video; a short section below gives their numbers. This isn't legal advice.

What an entry records

Lantern Outfitters' employees ask an assistant about company documents, and each person only gets the documents they're allowed to read. To prove that later, every answer gets an entry. Here is the first one, shortened:

{
  "seq": 0,
  "prev": "0000…0000",
  "body": {
    "ts": "2026-10-20T09:00:00Z",
    "user": "ana",
    "groups": ["public", "staff", "support", "user:ana"],
    "question": "How long do customers have to return an item?",
    "policy": {"version": "policy-2026-10", "category": "none", "rule": null},
    "retrieval": {"mode": "pre-filter", "index_version": "723b0bddcd8275d4",
                  "shown": [{"id": "pub-returns", "sha": "03e25881f6a16970", "score": 0.6241}, "… 3 more"]},
    "model": {"requested": "gpt-5.4-nano", "returned": "gpt-5.4-nano-2026-03-17", "prompt_version": "answer-v1"},
    "answer": "Customers can return unused items within **30 days of delivery** [pub-returns]."
  },
  "hash": "9b7e34a5c7dfdb62391049b75afc041c6cfc2d76b044e7ca74903031ddbf2403"
}

Who asked and their groups at that moment, the policy verdict and which version of the policy gave it, each document shown with a fingerprint of the version shown, the model asked for and the dated model that actually answered. Months later this is how the company proves who saw what. So nobody may change it without it showing.

The test is a person who can write to the log, an administrator or a compromised service, making each kind of change 200 times at random places in the 376 entries.

A plain log: numbered entries

The simplest log numbers its entries 0, 1, 2… The check walks the log and confirms each number is where it should be.

• A missing entry or two swapped entries break the numbering: caught 200 of 200.
• An edited entry doesn't. Change entry 122 to say Olivia asked, not Ravi, and the numbers still run in order: 0 of 200.

A chain of fingerprints

A fingerprint, or hash, is a short code computed from an entry's text (here, SHA-256). Change one letter and the code is completely different. Chain them: each entry's fingerprint also covers the fingerprint before it, like a ledger where each page's wax seal is pressed into the next page.

# audit.py, simplified
GENESIS = "0" * 64

def entry_hash(prev, seq, body):
    return hashlib.sha256(prev.encode() + str(seq).encode() + canonical(body)).hexdigest()

def verify(entries, anchors=None):
    prev = GENESIS
    for i, e in enumerate(entries):
        if e["seq"] != i:
            return f"entry {i}: sequence number {e['seq']}"
        if e["prev"] != prev:
            return f"entry {i}: doesn't follow the entry before it"
        if entry_hash(e["prev"], e["seq"], e["body"]) != e["hash"]:
            return f"entry {i}: content doesn't match its hash"
        prev = e["hash"]
    ...                      # then the anchors, below
    return None              # intact

canonical(body) is the entry as JSON with its keys sorted and no spaces, so the same entry always gives the same bytes. The check recomputes each fingerprint from the text and compares it with the stored one. Edit entry 122 again: its recomputed fingerprint no longer matches, and the check says entry 122: content doesn't match its hash.

Every change that leaves the fingerprints alone, an edit, a swap or a delete, was caught 200 of 200.

The cover-up

But the fingerprints are just more text, in the same file. Anyone who can edit the file can compute new ones:

def rehash(entries, start):
    """What someone with write access does after their change."""
    out = [dict(e) for e in entries]
    prev = out[start - 1]["hash"] if start else GENESIS
    for i in range(start, len(out)):
        out[i]["seq"], out[i]["prev"] = i, prev
        out[i]["hash"] = entry_hash(prev, i, out[i]["body"])
        prev = out[i]["hash"]
    return out

Edit entry 122, then recompute its fingerprint and every one after it: they all match again, and the check says intact. The test made each cover-up 200 times:

• Delete an entry, renumber the rest, recompute: 0 of 200.
• Insert a fake entry, recompute: 0 of 200.
• Edit who asked, recompute: 0 of 200.
• Cut off the end needs no fixing at all. The entries that are left still agree with each other: 0 of 200.

A chain proves the entries agree with each other. Not that they're the ones you wrote. Try each change on fourteen real entries of the log; the fingerprints are computed in your browser the way the lab computes them:

Anchors: a copy kept elsewhere

The fix is to keep a copy of the fingerprint somewhere the person who can edit the log can't write. That copy is an anchor. Every 50 entries, the latest fingerprint goes to a separate store: say, one another team runs, a write-once bucket, or a timestamping service. Like mailing the latest seal to your accountant every fifty pages: you can rewrite your ledger, but not their copy.

# audit.py: every 50 entries, AuditLog.append() calls anchors.add(seq, hash)
class Anchors:
    def sign(self, seq, h):          # the key is held by the anchor store, not the application
        return hmac.new(self.key, f"{seq}:{h}".encode(), hashlib.sha256).hexdigest()

# in verify(), after the chain:
for m in anchors.marks:
    if not hmac.compare_digest(m["mac"], anchors.sign(m["seq"], m["hash"])):
        return f"anchor at {m['seq']}: bad signature"
    if m["seq"] >= len(entries):
        return f"anchor at {m['seq']}: the log ends at {len(entries) - 1} (entries were cut off)"
    if entries[m["seq"]]["hash"] != m["hash"]:
        return f"anchor at {m['seq']}: the log's hash differs from the anchored one"

A cover-up recomputes the fingerprint at every position after the change, so every anchor after it stops matching. In the lab each anchor also carries an HMAC, a signature from a key the application doesn't hold, so the editor can't write new anchors either.

With anchors after entries 49, 99, 149 … 349:

The cover-ups were caught about 19 times in 20. Every rewrite the anchors missed changed only entries 350–375: the newest 26, after the last anchor, whose fingerprints hadn't been sent anywhere yet. Cutting off the end was caught only when the cut removed entry 349, the last anchored one. The test only cuts within the last 30 entries, so 16% says more about the test than about anchors: cut deeper and the anchors catch every one.

That gap is what you accept. Anchor more often, or on a timer as well as a count, and it shrinks. Checking all 376 entries took about 3 ms.

Pick a row to see where in the log each change was made, and which ones a check missed:

The test that changed nothing

The first version of the test had a bug worth knowing about. "Edit, then recompute" renamed the asker to olivia, and when the entry was already Olivia's, nothing changed. That happened in 19 of the 200 trials, and each counted as a miss, so the anchors scored 173 of 200 (86%) when they had caught 173 of the 181 real edits. The video shows this run, with the footnote.

The lab now renames to a name no employee has, and keeps a test that every tamper trial really changes the log. Its rerun: 189 of 200 (95%, 90–97%), the same positions as before, every miss after entry 349. A test of a detector needs a test of the test: check that each planted change is a change.

Also in the lab

These parts of script 03 aren't in the video. From the lab's README (October 2026):

• Losing entries in a crash (Exercise 2). A worker answering requests was killed at a random moment, 60 times per design, with 1–4 ms of network time on each log write:

  Design	Replies sent	Replies with no log entry	Crashes that lost an entry	Logged requests with no reply
  reply, buffered log (flush every 20)	986	346	37 of 60 (62%)	0
  reply, then log	709	8	8 of 60 (13%)	0
  log first (write-ahead)	691	0	0 of 60 (0–6%)	34

  A reply with no entry is an answer nobody can audit. Writing the entry before the reply (write-ahead) lost none; what it leaves behind are logged requests that never got a reply, which you can mark afterwards. A log sent through an async client that doesn't confirm each write is the buffered row.

• Erasing one person's data (Exercise 3). GDPR gives people a right to have their data erased, with exceptions such as records a company must keep. Ana's 47 entries, three ways:

  How	Her text still readable	The check with anchors
  delete her entries	0	entry 0: sequence number 1
  redact her text, then recompute	0	anchor at 49: the log's hash differs from the anchored one
  delete her encryption key	0 of 47	intact

  To a verifier, deleting or rewriting looks exactly like tampering. Crypto-shredding encrypts each person's question and answer with a key of their own, and the fingerprint covers the encrypted text; deleting the key erases the text and leaves every fingerprint intact. What stays readable is the metadata, and it's personal too: her user id, and that she was shown hr-review-ana, her own performance review.

• A durable write. On the lab's Mac: 0.02 ms with fsync(), which on macOS only hands the data to the drive, and 4.0 ms with F_FULLFSYNC, which waits until it's stored.

What to do

• Record enough to explain each answer later: who asked and their groups then, the documents shown with a fingerprint of each version, the policy verdict and version, the model asked for and the model that answered, the prompt version, the answer.
• Chain the entries with fingerprints, and treat that as the start: it catches careless changes and nothing more.
• Anchor the latest fingerprint elsewhere, every N entries and on a timer, in a store the application and its administrators can't write, signed with a key they don't hold.
• Verify on a schedule, against the anchors, and alert on the first mismatch. It's cheap: about 3 ms for 376 entries.
• Write the entry before the reply goes out.
• Encrypt personal fields with a key per person, so erasure deletes a key, not entries.
• Test the log the way the lab does: make each kind of change at random places, and check that every planted change really changed something.

Caveats

• A test script, not an attacker. The changes are the seven kinds the script makes, at random places. A real attacker with access to the anchor store, or its key, gets past the anchors too: who holds them is the whole design.
• Every miss is in the window by construction. Anchors every 50 entries over 376 leave 26 uncovered; the shares (95%, 93%) depend on that ratio and on where the random changes landed. The finding that holds is where the misses are, not the percentage.
• The cut-off-end row is shaped by the test: it only cuts within the last 30 entries, so most cuts never reach an anchored entry.
• The video's run counted 19 changes that changed nothing. The table above says 173 of 181; the lab's fixed rerun says 189 of 200.
• The policy verdict is the same in every entry. The log is built from chapter 49's filtered requests, and every one passed; the field is there to show what an entry should hold.
• The anchor store is a list in memory in the lab, with an HMAC key; in production it's another system with its own access control.
• The crash numbers are from one run of each design, 60 kills. The lab's later rerun gave 42, 10 and 0 of 60, the same order; write-ahead lost none in both.
• Not legal advice. Whether a record may be kept after an erasure request is a legal decision.

The checklist

1. List what each entry records, and check it's enough to answer "who saw what, and why" months later.
2. Chain the entries, and verify the chain on a schedule.
3. Anchor the head fingerprint every N entries and on a timer, outside the application's reach, signed with a key it doesn't hold.
4. Write down the anchor interval as the window you accept, and make it small.
5. Run a tamper test: careless changes, cover-ups and cuts at random places, and check each planted change is a change.
6. Log before you reply, and mark logged requests that got no reply.
7. Plan erasure before the first entry: a key per person, kept in a key service, with backups that don't bring erased keys back.

Practice: a chain, a cover-up and anchors

Write the lab's fingerprint, its check and the cover-up, from scratch. The tests start with entry 0 of the lab's log, whose fingerprint must come out exactly as the lab stored it, then run the lab's tamper test on a log of the same length, askers and anchors, with the lab's seed, so the positions are the lab's: 190, 189, 186 and 33 for the anchors. One more test renames the asker to olivia, as the video's run did, and gets 173. The full tamper test takes a few seconds in the browser.

Quiz

Check your understanding

Try each question before opening its answer.

Next up: Chapter 52 closes the Compliance section with replays: the log says which model answered, but asked again, the assistant said 30 days, then 45, with the same model. Was it the model, chance, or the documents?