aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

Google 与香港大学论文提出 VeriCUA,核查电脑操作智能体是否真正完成任务

AI 导读

VeriCUA 会探索应用并列出任务要求,再通过屏幕或代码检查逐项核对;智能体自称完成不算证据。材料称,在 ScienceBoard 测试中,其误报率为 5.4%,低于视觉语言模型判别器的 24.8% 和生成脚本验证器的 21.7%。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/80piimpKtU

Title: "VeriCUA: Verification as Agentic Exploration for Computer-Using Agents"

回复Rohan Paul@rohanpaul_ai
New Google and University of Hong Kong paper argues that when AI agents use computers, their own “task done” message should never count as proof, and the saved files and app data should be checked instead. Computer-use agents can report a task done when edits were never saved or only part was finished. Common checkers either judge a few screenshots or run 1 generated script, and both can miss the deciding evidence. VeriCUA explores the app to build a requirement checklist, then runs screen or code checks, like file reads, until each item is settled. The agent's own claim of success never counts as proof. On ScienceBoard, its false-positive rate was 5.4%, against 24.8% for a VLM judge and 21.7% for a generated script verifier. Across 3 online environments, it rejected about 80% of false completion reports. If agent runs feed your evals, training data, or RL rewards, check saved machine state instead of trusting screenshots or the agent's report.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错