原帖引用 · AI 翻译
标题：“VeriCUA：将验证作为计算机使用智能体的自主探索”
谷歌和香港大学的一篇新论文认为，当 AI 智能体使用计算机时，其自身发出的“任务已完成”消息绝不应被视为证据，而应检查已保存的文件和应用数据。
计算机使用智能体可能在修改从未保存或只完成部分任务时报告任务已完成。常见的检查器要么判断几张屏幕截图，要么运行 1 个生成的脚本，两者都可能漏掉决定性证据。
VeriCUA 会探索应用，建立需求核对清单，然后执行屏幕或代码检查（例如读取文件），直到每一项都得到核实。智能体自身声称成功，绝不算作证据。
在 ScienceBoard 上，它的假阳性率为 5.4%，而 VLM 评判器为 24.8%，生成脚本验证器为 21.7%。在 3 个在线环境中，它驳回了约 80% 的虚假完成报告。
如果智能体运行结果会用于你的评估、训练数据或强化学习奖励，就应检查机器上保存的状态，而不是相信屏幕截图或智能体的报告。
原文
New Google and University of Hong Kong paper argues that when AI agents use computers, their own “task done” message should never count as proof, and the saved files and app data should be checked instead. Computer-use agents can report a task done when edits were never saved or only part was finished. Common checkers either judge a few screenshots or run 1 generated script, and both can miss the deciding evidence. VeriCUA explores the app to build a requirement checklist, then runs screen or code checks, like file reads, until each item is settled. The agent's own claim of success never counts as proof. On ScienceBoard, its false-positive rate was 5.4%, against 24.8% for a VLM judge and 21.7% for a generated script verifier. Across 3 online environments, it rejected about 80% of false completion reports. If agent runs feed your evals, training data, or RL rewards, check saved machine state instead of trusting screenshots or the agent's report.