aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分60

清华研究提出 AI 智能体记忆删除基准,测试系统难以应对遗忘请求

AI 导读

研究构建 BEMULA 基准,以 50 段合成对话中的 135 次遗忘请求,检验个人信息是否被删除、其他记忆是否保留,以及删除后能否被诱导提取。在测试的 6 种记忆系统和 3 种基础模型中,没有一种配置能端到端完成超过 1.5% 的遗忘请求。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/itR5sezpMz

Title: "Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents"

回复Rohan Paul@rohanpaul_ai
New Tsinghua paper finds that when people ask AI agents with long-term memory to forget a personal detail, the assistants almost never remove it cleanly, either keeping it or deleting too much. Agents with long-term memory store personal details, and users increasingly ask them to delete things like a home address. Existing memory benchmarks test only storing and recalling facts. They built BEMULA, 50 long synthetic chats with 135 forget requests, each paired with a copy that has no request. Probes check whether the fact is gone, other memories survive, and attacks can still extract it. No tested setup passed more than 1.5% of forget requests end to end, across 6 memory systems and 3 backbone models. After a forget request, systems blocked at most 35.7% of simple extraction prompts like role-play or yes-or-no guesses. If your agent remembers users, build and test a real delete path, since a confirmation message does not mean the data is gone.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错