aifollow.news 搜索
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

北大与三星论文提出实时智能体评测,并用慢模型指导快模型

自动核验发布 · 本文由系统生成并完成证据核验,未经人工审稿。

AI 导读

这项研究构建了推理时游戏仍在运行的 RealtimeBench,涵盖《马里奥》《贪吃蛇》《俄罗斯方块》等6款游戏的180项任务。测试中,GPT-6.1-sol 在暂停计时的6款游戏中有5款胜过 Jev,但实时得分为1.51分,低于 Jev 的13.51分;让慢模型在局间分析失败并撰写指令后,Jev 经过两轮指导的平均分升至24.19分。

原帖引用 · 原文

该语言的正文暂不可用,当前显示已有版本。

@rohanpaul_ai

– https://t.co/FxN52zkZai

Title: "The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock"

回复Rohan Paul@rohanpaul_ai
New Peking University and Samsung paper shows that a slow reasoning model can coach a fast model offline, raising its live game score from 13.51 to 24.19. shows that the stronger reasoning model in paused games can fall far behind a faster model once the game keeps moving. Many game evaluations pause the world while the model thinks, so slow answers cost nothing. In real settings, a correct action can arrive too late. They built RealtimeBench, 180 tasks across 6 games like Mario, Snake, and Tetris that keep running during inference. GPT-6.1-sol beat the faster Jev in 5 of 6 games when paused, but scored 1.51 versus 13.51 out of 100 live. Their fix, TORSI, has the slow model study the fast model's failures between games and write instructions the fast model follows during play. After 2 rounds, Jev's average score rose from 13.51 to 24.19, improving all 6 game types. For real-time agents, keep a fast model in the loop and use the slow model offline to write and test its instructions.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错