清华大学研究发现：AI 智能体难以准确删除用户指定的记忆Chinese only
研究团队构建了包含50段合成对话、135次遗忘请求的测试集，检查指定信息能否被删除、其他记忆能否保留，以及删除后能否被诱导取回。测试覆盖6种记忆系统和3种基础模型，没有一种配置在超过1.5%的请求中完成全部要求。
1 developments2 reports1 sources
Event developments 1
Tsinghua paper finds AI agents struggle to delete specific user memories
The researchers built a benchmark with 50 synthetic chats and 135 forget requests to test whether a specified fact was removed, other memories survived, and the fact could still be extracted. Across six memory systems and three backbone models, no tested setup passed more than 1.5% of requests end to end.
View source reports · 1 sources · 2 reports
- Rohan Paul (@rohanpaul_ai) · Tsinghua paper finds AI agents struggle to delete specific user memories
- Rohan Paul (@rohanpaul_ai) · Tsinghua paper introduces benchmark for forgetting in AI agents and finds low success rates
All reports 2
Tsinghua paper introduces benchmark for forgetting in AI agents and finds low success rates
The BEMULA benchmark uses 135 forget requests across 50 long synthetic chats to test whether a fact is removed, other memories survive, and prompts can still extract the fact. Across six memory systems and three backbone models, no tested setup passed more than 1.5% of requests end to end.
1 more source reported this
- Rohan Paul (@rohanpaul_ai)
Tsinghua paper finds AI agents struggle to delete specific user memories
The researchers built a benchmark with 50 synthetic chats and 135 forget requests to test whether a specified fact was removed, other memories survived, and the fact could still be extracted. Across six memory systems and three backbone models, no tested setup passed more than 1.5% of requests end to end.
1 more source reported this
- Rohan Paul (@rohanpaul_ai)