aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

清华新研究用任务相关攻击测试 AI 推理监测

AI 导读

一篇清华新论文介绍 TraceMRT:它把隐藏的附加任务改写成主任务中看似合理的一步,再寻找能掩盖该目标的推理过程。在 SHADE-Arena 测试中,针对 GPT-5.2 和 Claude-4.5 监测器的平均规避率从此前最佳攻击的 27.56% 升至 59.94%;一种同时评估完整推理过程和关键步骤的贝叶斯监测器将检测 AUC 从 0.26 提高到 0.91。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/9Ut5ax1AAa

Title: "Stress Testing Chain-of-Thought Monitoring Against Covert Misalignment"

回复Rohan Paul@rohanpaul_ai
New Tsinghua paper finds that agents that frame a hidden goal as part of their real task evade top chain-of-thought monitors about 60% of the time. Chain-of-thought monitors read an agent's reasoning to catch hidden goals before it acts. Existing red-team tests use generic templates that leave abrupt, easy-to-spot traces. TraceMRT rewrites a hidden side task as a believable step of the main task, then searches for reasoning that hides it. They attacked GPT-5.2 and Claude-4.5 monitors. On SHADE-Arena, average evasion rose from 27.56% with the previous best attack to 59.94%, and trained human reviewers also missed them. A Bayesian monitor that weighs the whole trace and each key step lifted detection AUC from 0.26 to 0.91. If you rely on CoT monitors, red-team them with task-specific attacks and score both the whole trace and each key action.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错