aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

亚马逊论文研究环境变化后智能体技能失效,并提出选择性维护方法

AI 导读

一篇亚马逊论文测试了 Claude Opus 4.7 和 Haiku 4.5 在环境变化前后的表现,发现保存的技能细节可能从助力变成阻碍。其方法会重放已保存的检查,只修复或重建冲突内容;在 Terminal-Bench 2 测试中,一项技能曾让 Opus 得分提高 4.7 点,环境变化后却使其通过率比不使用技能低 11.4 点。

原帖引用 · 原文

该语言的正文暂不可用,当前显示已有版本。

@rohanpaul_ai

– https://t.co/XdlWSlNtqQ

Title: "When Agent Skills Go Stale: Runtime Behavior and Selective Maintenance in Evolving Environments"

回复Rohan Paul@rohanpaul_ai
New Amazon paper finds that saved agent skills can hurt once the environment changes, and checking and repairing them selectively restores their value. Agent skills store past experience, like file paths, field names, or commands, so agents can skip work later. When an app or API changes, some of that stored detail goes stale without warning. They tested Claude Opus 4.7 and Haiku 4.5 on 4 settings, including Terminal-Bench 2 and IRS tax forms, before and after a change. Their fix replays saved checks witho1ut model calls and repairs or rebuilds only the conflicting skill content. On Terminal-Bench 2, a skill that had added 4.7 points for Opus cut its pass rate 11.4 points below no skill after the change. Maintenance lifted Haiku's tax-form field accuracy from 30.5% with the stale skill to 97.5%. Check saved agent skills against the current environment before reuse, instead of waiting for tasks to fail.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错