aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

Amazon 研究发现:环境变化后,旧版智能体技能可能拖累任务表现

AI 导读

Amazon 的论文测试了 Claude Opus 4.7 和 Haiku 4.5 在四种环境变化前后的表现。Terminal-Bench 2 中,一项原本让 Opus 通过率提高 4.7 个百分点的技能,在环境变化后反使通过率比不用技能低 11.4 个百分点;有选择地检查并修复冲突内容,可恢复旧技能的价值。

原帖引用 · 原文

该语言的正文暂不可用,当前显示已有版本。

@rohanpaul_ai

New Amazon paper finds that saved agent skills can hurt once the environment changes, and checking and repairing them selectively restores their value.

Agent skills store past experience, like file paths, field names, or commands, so agents can skip work later. When an app or API changes, some of that stored detail goes stale without warning.

They tested Claude Opus 4.7 and Haiku 4.5 on 4 settings, including Terminal-Bench 2 and IRS tax forms, before and after a change. Their fix replays saved checks witho1ut model calls and repairs or rebuilds only the conflicting skill content.

On Terminal-Bench 2, a skill that had added 4.7 points for Opus cut its pass rate 11.4 points below no skill after the change. Maintenance lifted Haiku's tax-form field accuracy from 30.5% with the stale skill to 97.5%.

Check saved agent skills against the current environment before reuse, instead of waiting for tasks to fail.

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错