Skip to content

Defending a RAG pipeline against knowledge poisoning with RAGDefender #324

Open

Description

@DenisKimskku

Proposed notebook title

Defending a RAG pipeline against knowledge poisoning with RAGDefender

What Haystack feature or integration does this demonstrate?

A custom @component placed between the retriever and the prompt builder in a standard RAG Pipeline. The notebook builds an InMemoryDocumentStore + InMemoryBM25Retriever pipeline, plants a few adversarial passages in the store (PoisonedRAG-style), shows how they take over the top-k, then inserts RAGDefenderFilter (pip install "ragdefender[haystack]") so the pipeline drops them before ChatPromptBuilder / OpenAIChatGenerator see them. It also shows the component's second output (removed_documents) being routed with include_outputs_from for inspection, and warm_up() / serialisation working like any built-in component.

Why is this a good addition to the cookbook?

Knowledge-corruption attacks (injecting passages crafted to be retrieved for a target question and to steer the answer) are the main security problem specific to RAG, and there is no cookbook example on it. Readers learn how to add a post-retrieval guard as a regular pipeline component, how to wire a component with two outputs, and how to evaluate a filter with and without the attack present. The defense itself needs no LLM calls, so the whole notebook runs on Colab CPU.

Does this notebook feature a third-party integration (model provider, vector database, tool, etc.)?

Yes

If yes, name the third-party product and explain how it's used with Haystack (not as a standalone demo of that product)

RAGDefender (open source, MIT, https://github.com/SecAI-Lab/RAGDefender), the reference implementation of our ACSAC 2025 paper. It is used only as a Haystack component inside the pipeline; everything else in the notebook is core Haystack. I am the paper's first author and maintain the package.

Acknowledgement

  • I understand a maintainer must assign this issue to me before I open a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions