原帖引用 · 原文
New Google paper on RAG poisoning.
Shows that poisoning a RAG system mostly depends on copying the query, since adding an attack payload barely changed which documents got retrieved.
fake documents only need to repeat a user’s question to be picked up by AI systems that search before answering, and adding hidden attack instructions barely changed that.
RAG systems pull documents from sources that outsiders can sometimes write to. Before any hidden instruction can act, the poisoned page first has to get past the retriever.
For each test query, they added up to 5 documents repeating its text, with or without an attack payload. They tested classic keyword and semantic retrievers plus the dense embedding model BGE-small on 623 queries.
Just 5 documents, about 0.1% of the corpus, cut classic retrievers' ranking quality by 40.2% to 61.5%. A filter that rejects query-echoing documents blocked them but halved clean ranking quality on NFCorpus, from 0.267 to 0.134.
If your RAG index accepts outside content, flag documents that mirror user queries, and measure any filter's cost on clean queries too.