aifollow.news
← News
Event · Change Claude

New Amazon paper finds that saved agent skills can hurt once the environment changes, and checking and repairing them se

研究测试Claude Opus 4.7和Haiku 4.5后发现,环境变化使Opus在Terminal-Bench 2上使用旧技能时的通过率低于不使用技能,而维护技能将Haiku的税表字段准确率从30.5%提高到97.5%。

1 reports1 sources

Reports 1

Rohan Paul (@rohanpaul_ai) Machine translation

Amazon paper finds saved agent skills can hurt performance after environments change

The paper tested Claude Opus 4.7 and Haiku 4.5 across four settings before and after a change. On Terminal-Bench 2, a skill that had raised Opus's pass rate by 4.7 points left it 11.4 points below the no-skill baseline after the change; selectively checking and repairing conflicting content can restore a saved skill's value.

SourceRohan Paul (@rohanpaul_ai) ↗