Intermediate7 min read

Long-term Memory

Long-term memory is a store outside the model — you write a distilled fact in one session and retrieve it in another, because the model itself forgets the moment the window is gone.

Explain like I'm new to AI

Close the chat, open a new one. Short-term memory is gone. If the assistant still knows you're vegetarian, that fact lived in a database, not in the neural net.

Write once, retrieve later:

Mon: writeStoreThu: retrieveUse

Monday: Ada: "I'm vegetarian." → system writes a long-term fact.

Long-term memory is a write in one session and a retrieve in another — the model weights never stored the fact.

That is the same shape as RAG (embed, store, search) but the corpus is this user's or this agent's extracted memories, not your public docs. RAG answers "what's in the handbook?"; long-term memory answers "what did we already decide about Ada?"

Mental model

A medical chart. Each visit, the doctor doesn't replay every prior appointment from RAM. They pull the chart (allergies, meds) and add a new note. The brain of the doctor on Tuesday is not a copy of Monday's — the chart is.

How it works

Decide to write — not every sentence. Extract durable facts ("timezone = IST") or important events.
Store with metadata: user_id, key, timestamp, source, trust.
Retrieve at the start of a turn or when a tool says recall(query) — typically embedding search plus filters.
Pack a handful of hits into the working set (context engineering).

MemGPT (now Letta) popularized the OS analogy: the context window is RAM; long-term stores are disk; the agent gets tools to page memory in and out.

Real-world example

ChatGPT-style "memory," a CRM assistant that remembers account quirks, a coding agent that remembers "this repo uses pnpm." All of them are writes to a store plus retrieval next session — not fine-tuning.

Technical explanation

Don't confuse:

Fine-tuning — changes weights; slow, not per-user, bad for facts that change.
Session history — dies with the thread (unless you export it).
Long-term memory — per-entity records with retrieval. Production stacks in 2026 are often hybrid: vectors for fuzzy recall, a graph or KV for exact keys ("diet"), timestamps for freshness.

Write policy is the hard part. Save too much → retrieval noise. Save too little → the product feels amnesiac. Extract with an LLM into a schema, then dedupe / conflict-check before commit (later lesson).

Common mistakes

Common mistake

Embedding raw chat logs as "memory." You built a messy RAG corpus of transcripts, not facts. Retrieve noise, contradict yourself, and never expire anything.

One global store with no user_id (privacy and quality both die).
Using long-term memory as a substitute for the current task state (todos don't belong in "Ada is vegetarian").

When to use it

Preferences, durable facts, and cross-session continuity for a product people return to.

When NOT to use it

Secrets that shouldn't persist; one-off tasks; facts that already live in a system of record you can query with a tool (fetch the live order instead of memorizing it).

Alternatives

RAG over official docs; tools to the source of truth; fine-tuning for style/skill, not for Ada's diet.

Quick quiz

Related concepts

Agent Memory & State — Agents use short-term memory (the context window), long-term memory (facts stored outside and retrieved), and state (task progress) to stay coherent over many steps.
What is RAG? — RAG retrieves relevant external information and gives it to a language model as context before it answers.