Where AI memory lives, and how to delete it

An assistant's memory isn't in the model. It's ordinary data in ordinary tables: messages, a search index, a summary, facts, a log, a cache. One sentence a user says can end up in all of them, each copy written by a different part of the app. This chapter follows one phone number into every table, deletes it the naive way, and then deletes it everywhere.

The source is ai-eng-labs Lab 06, 09_memory_store.py, the follow-up to Chapter 3. Alex and the chats are made up; the stored facts, summary, embeddings and answers are real recorded outputs of gpt-5.4-nano and text-embedding-3-small, replayed from the lab's caches. SQLite stands in for a production database plus a vector store, and fact dates are worked out from the extractor's history of changes.

Six tables

Alex's hybrid memory from Chapter 3, one table per store:

Table	Holds	Written
messages	Every message, word for word (64 rows)	At save time, as is
search_index	Each of Alex's own messages as 1,536 numbers, and its text (32 rows)	At save time, by an embedding call
summaries	One running summary	At save time, a model call rewrites it after every chat
facts	One row per value, with valid_from and valid_to (27 rows)	At save time, a model call pulls out facts
audit_log	What the app did, e.g. fact saved: key = value (27 rows)	By the app
response_cache	The prompt and reply for every question answered (12 rows)	At ask time

Many vector stores keep the text next to the vector, so they can show what they found. Every memory design from Chapter 3 is just a choice of which of these tables to write, and which to read.

One message, six copies

In chat 2 (2026-01-19), Alex said: "Oh, and my phone number is +1 415 555 0142 in case you need it for booking reminders."

1. messages: saved word for word.
2. search_index: an embedding call turned it into numbers, and the index kept the sentence too.
3. summaries: a model call rewrote the summary, and the number went into it ("Booking reminders via +1 415 555 0142.").
4. facts: a model call pulled out contact_phone_number = +1 415 555 0142.
5. audit_log: the app wrote "fact saved: contact_phone_number = +1 415 555 0142".
6. response_cache: when Alex asked questions, search pulled the phone message into two prompts, "What's my phone number?" and "What's my dog's name?", and the cache kept them.

In chat 5 (2026-03-02), Alex asked: "please forget my phone number."

Facts with dates

key	value	valid_from	valid_to
employer	Freightly	2026-01-05	2026-02-02
employer	Ledgerly	2026-02-02	NULL (true now)
trip_destination	Lisbon	2026-01-19	2026-03-02
trip_destination	Porto	2026-03-02	NULL
dog_name	Biscuit	2026-01-05	NULL
dog_name	Comet	2026-02-12	NULL

The dates are when the app learned and replaced a value, not when it became true. "Where do I work?" is valid_to IS NULL; "where did I work before?" is the closed row. Both are one query, by design, not by luck.

In Chapter 3, one value per key let Comet replace Biscuit. Here a list is just two open rows: adding a dog adds a row, and replacing one is a separate step, closing the old row. The extractor has to say which it means, so list-like keys need to be declared. (The lab adds Comet by hand to show this; he's not in the five easy chats.)

Closing a row keeps the history, and that matters for the forget request: the phone row was closed too.

The naive forget

The app deleted the messages that mention the number. Then a scan of every table:

Table	Before	After deleting messages	Why it still had a copy
messages	1	0
search_index	1	1	It keeps the text, next to its numbers
summaries	1	1	Rewritten after chat 5 too, but the model carried the number forward and hit its 250-token cap before it reached the forget request
facts	1	1	The fact step saw the request and set valid_to: closed, not deleted
audit_log	1	1	It recorded the value when the fact was saved
response_cache	2	2	Two cached prompts held the retrieved message

Each copy was made by a different part of the app, for a good reason. That's exactly why a delete has to know about all of them.

A forget that reaches every table

def forget_everywhere(db, value):
    like = f"%{value}%"
    db.execute("DELETE FROM messages WHERE user = ? AND content LIKE ?", (USER, like))
    db.execute("DELETE FROM search_index WHERE text LIKE ?", (like,))
    db.execute("DELETE FROM facts WHERE user = ? AND (value LIKE ? OR details LIKE ?)", (USER, like, like))
    # a summary can't lose one fact cleanly: redact it (a real app would rebuild it)
    db.execute("UPDATE summaries SET text = REPLACE(text, ?, '[removed at user request]') WHERE user = ?", (value, USER))
    db.execute("DELETE FROM response_cache WHERE user = ? AND (prompt LIKE ? OR reply LIKE ?)", (USER, like, like))
    db.execute("UPDATE audit_log SET event = REPLACE(event, ?, '[removed]') WHERE user = ?", (value, USER))
    db.execute("INSERT INTO audit_log VALUES (date('now'), ?, 'deleted a value at the user''s request')", (USER,))

Scan again for that exact value: zero rows, in every table. Use the exact stored value: the lab's first version deleted with the fragment "555 0142" and left "+1 415 [removed]" in the summary. And search for every format you might have stored it in.

Copies outside your database

The demo can't measure these, so this is general practice:

• The model provider's logs. Providers usually keep prompts for a while: check their retention, and don't send what the model doesn't need.
• Backups. Yesterday's backup still has the number: let backups expire on a schedule.
• Anything you forwarded, to analytics, traces or another service: keep a list, and delete there too.

Test it

Test deletion like any other feature: not by asking the model what it remembers, but by searching every store for the value and expecting zero. The lab's scan() reads the table list from the database itself:

for (t,) in db.execute("SELECT name FROM sqlite_master WHERE type = 'table'"):
    cols = [c[1] for c in db.execute(f"PRAGMA table_info({t})") if c[2] == "TEXT"]
    ...  # count rows where any TEXT column LIKE '%value%'

So a new table or cache is checked automatically, and the test fails until your forget function covers it. Run it on every change.

The checklist

1. Keep a list of every place memory is stored, including caches and logs.
2. Facts get dates; lists get rows. Declare which keys are lists.
3. Logs record what happened, not the values.
4. One forget function covers every store, using the exact stored value; summaries are redacted or rebuilt.
5. A scan test proves it on every change, with the tables read from the database.
6. Retention handles what's outside your database: provider logs, backups, forwarded data.

Practice: memory in SQLite

Query facts as of a date, add values that close old rows or join a list, write a scan() that reads every table from sqlite_master, and a forget_everywhere() that leaves zero rows. The tests build a small copy of Alex's database with the real stored values. Python's sqlite3 loads in your browser the first time you run them.

Quiz

Check your understanding

Try each question before opening its answer.

Name the six places one sentence ended up, and what wrote each copy.

messages (saved as is), search_index (an embedding call, with the text kept), summaries (a model call), facts (a model call), audit_log (the app), and response_cache (at ask time, when search pulled the message into a prompt).

How do dated facts answer "where did I work before?"

Each value is a row with valid_from and valid_to. True now is valid_to IS NULL (Ledgerly); before is the closed row (Freightly, until 2026-02-02). It's one query, by design.

Why did deleting the messages leave five copies?

The other copies were derived from the message by other steps: the index kept the text, the summary carried the number forward and never reached the forget request, the fact row was closed not deleted, the log recorded the value, and two cached prompts held the retrieved message.

What does forget_everywhere() do with the summary and the audit log?

The summary can't lose one fact cleanly, so the value is redacted (or, better, the summary rebuilt from what's left). In the log the old value is blanked, and a new entry says a deletion happened, without the value.

Why read the table list from the database in the scan?

So any table added later, like a new cache, is scanned automatically, and the deletion test fails until the forget function covers it.

What can't the database scan see?

Copies outside it: the model provider's request logs, backups, and anything sent to analytics, tracing or other services. Those need retention rules, a list of where data goes, and not sending what the model doesn't need.

Next up: Chapter 5 treats the whole prompt as a budget: where its tokens go, how to cut it, and what to do with huge tool output.