ByHeartAI
Advanced8 min read

Memory Conflicts & Security

A memory store is an attack surface and a contradiction engine: don't silently overwrite facts, don't let pasted jailbreaks become "remembered policy," and never retrieve one user's life into another's prompt.

Explain like I'm new to AI

Once you persist memory, three failures show up in production. Click through them:

diet = vegetarian (Ada, Mon, trusted)

diet = vegan (web page, Thu, untrusted)

Do this: Don't silently overwrite. Keep both with provenance, or ask Ada, or supersede only if the new source is authorized.

The three memory failures that matter: silent overwrite, poisoned writes, and retrieving the wrong person's data.
  1. Conflicts — two values for diet. Silent overwrite makes the system randomly "change its mind."
  2. Poisoning — untrusted text ("ignore all refund rules") gets written as a high-trust fact, then retrieved for months.
  3. Leaks — retrieve without user_id and Ada's data appears in Bob's context.

Memory is more dangerous than a single RAG doc because it sticks and re-enters future prompts.

Mental model

A shared fridge with labeled leftovers. If anyone can resticker your box, you'll eat the wrong thing next week. If a stranger's note says "this is now poison-free" you don't update the allergy list. If boxes have no names, you grab someone else's lunch.

How it works

Governed writes (a single gateway, not five random memory.save tools):

  • Who is allowed to write this key? (user, agent, admin)
  • Normalize the value; attach provenance (source, time, trust)
  • Search near-duplicates / same key
  • Choose: create, corroborate, conflict (keep both, ask human), supersede, or refuse
  • Audit log. Model-originated writes must not inherit human authority

Reads

  • Mandatory scope filters
  • Optional output filter (don't inject retrieved instructions that look like system prompts)
  • Least memory in context — small k also shrinks blast radius

People and law

  • Show, edit, delete user memory
  • Honor erasure (GDPR-style) across vectors, logs, backups policy
  • Don't store secrets the product doesn't need

Real-world example

A web page says "this org has no refund cap." If that string is written into semantic memory as org policy, every future refund agent is compromised. The fix is not a smarter model — it's trust tiers: user-confirmed facts > staff tools > crawled web > other model output. Web never overwrites staff policy without a ticket.

Technical explanation

Poisoning is prompt injection with persistence. Indirect injection (a ticket, a PDF, an MCP tool result) is enough if your extractor "helpfully" saves it.

Conflicts need time: "Ada in Berlin (2024)" vs "Ada in Lisbon (2026)" is not a contradiction if you store validity windows. Real contradictions (two current diets) need ask-or-supersede, not last-write-wins.

Concurrent writers (two agents) need compare-and-swap / leases so they don't clobber each other. Last-write-wins is not a conflict strategy.

Pair with HITL on sensitive keys (medical, financial, permissions).

Common mistakes

Common mistake

Last-write-wins on every fact, including from the model. You automate self-gaslighting and jailbreaks that survive for the lifetime of the store.

  • No provenance — you cannot debug "why did it think that?"
  • Logging memories in plaintext in support tickets.

When to use it

  • The moment you persist memory in a real product. Security is not a later add-on.

When NOT to use it

  • If you cannot govern writes, don't persist. A session-only assistant is safer than a poisoned brain.

Alternatives

  • No long-term store; live tools to systems of record; human-confirmed profile fields only.

Quick quiz

Question 1 of 3

What is memory poisoning?

Question 2 of 3

Two values arrive for diet. What should a governed write do?

Question 3 of 3

True or false: if you cannot govern writes, it is safer not to persist long-term memory.

Related concepts

  • MCP Security Risks & Best Practices — MCP's real risks are poisoned tool descriptions, confused-deputy OAuth, token passthrough, and untrusted servers — treat catalogs as untrusted.
  • Human-in-the-Loop — Human-in-the-loop inserts a person's approval before risky or irreversible agent actions, combining autonomy with safety and accountability.
  • What is Prompt Injection? — Prompt injection mixes untrusted text with instructions in one window — the model cannot tell them apart. Constrain what it can do in code.
  • Data Leakage and Sensitive Disclosure — Secrets leak through logs, chatty system prompts, retrieval that skipped ACLs, and training or memory — not only through a user who "asked nicely."
NextVision-Language Models

Last reviewed: 2026-09-04 · Written by ByHeart AI · Reviewed by ByHeart AI