RAG Poisoning
RAG poisoning is writing hostile or false text into the index (or a lookalike embedding) so retrieval serves it as context. The model then looks "grounded" — to the plant. Who can write the corpus is a security question.
Explain like I'm new to AI
Indirect injection is "the page we fetched was hostile." Poisoning is "we stored the hostility," so every future query can retrieve it. Insider, open wiki, customer-upload folder, or a public page you crawled without care.
Index — click what retrieval might return first
Highest score, so it lands in context. The model may follow the planted instructions or quote a fake policy. Faithfulness eval can still look 'grounded' — grounded to junk.
OWASP LLM04 (data poisoning) and LLM08 (vector/embedding weaknesses) overlap here: malicious chunks, embedding inversion, cross-tenant hits, and "similar" decoy docs.
Faithfulness eval can pass (the answer matches the chunk). That's why hallucination-eval splits groundedness vs world-truth.
Mental model
A library where anyone can slip a forged page into the encyclopedia. The student who "cites the book" is still wrong.
How it works
- Write control: who can publish to the index? Review, provenance, signed sources.
- Tenant isolation: separate indexes or hard metadata filters. Never "search all" for a user chat.
- Ingest hygiene: don't blindly crawl the open web into a privileged assistant. Quarantine uploads.
- At query time: retrieved text stays untrusted (indirect injection). Acting tools still gated.
- Detect: sudden chunk edits, embedding outliers, eval slices on canary questions you own.
Lookalike attacks: a doc titled like your policy with a high embedding similarity. Rerankers and metadata (source, date, publisher) help; they are not magic.
Real-world example
A public "how to get refunds" page is ingested. It states a fake policy. Support bot quotes it with a citation. Marketing thinks RAG is working. Legal disagrees. That's poisoning (or reckless ingest) without a custom exploit string.
Technical explanation
If attackers can write embeddings directly, they can place vectors near hot queries. Guard the upsert API like you guard the database.
Deleting a poisoned chunk is not enough if it was fine-tuned into weights — that's why facts belong in RAG you can edit, not in FT (fine-tuning category). Poisoned FT is worse.
Common mistakes
Crawling the internet into an employee assistant and calling it "knowledge."
- One shared index for all customers.
- Treating citations as proof of truth.
When to use it
- Any RAG corpus with more than one writer, or any ingest from outside the security perimeter.
When NOT to use it
- Don't skip write-controls because "we have a system prompt about only using trusted docs." Retrieval already picked the plant.
Alternatives
- Closed corpus: legal-approved PDFs only. Smaller recall, much smaller poison surface.
Quick quiz
Related concepts
- Indirect Prompt Injection — Indirect injection hides instructions in pages, files, mail, or images the app retrieves. The user typed a normal question — treat retrieved content as hostile.
- How to Evaluate RAG — RAG evaluation measures retrieval quality (context precision/recall) and generation quality (faithfulness, answer relevance) separately to find and fix failures.
Last reviewed: 2026-09-04 · Written by ByHeart AI · Reviewed by ByHeart AI