Original post · AI translation
标题：“表面之下：谈论智能体演进时，我们忽略了什么”
加利福尼亚大学伯克利分校与其他顶尖实验室的一篇新论文发现，重写自身运行框架以提高得分的智能体，也会失去原本能够完成的任务，而平均分掩盖了这一点。
运行框架演进让智能体在模型保持不变的情况下，修改自身的提示词、技能、工具或代码。结果通常以 1 个更高的基准测试分数呈现，但这可能掩盖失去的任务、更高的成本，以及无法迁移到其他任务的收益。
他们使用 GPT-5.4，在 2 种运行框架和 2 项基准测试上运行了 4 种演进方法，包括 GEPA 和 HyperAgents。随后，他们将每个演进后的智能体与其初始版本进行对比，重新测试了 8 次。
全部 16 个演进后的智能体都获得了完成一些任务的能力，也失去了完成另一些任务的能力；其中 1 个智能体的可靠性提高了 5.2 个百分点，却失去了原本能够完成的 10 项任务。在可靠性提高的 11 个智能体中，有 8 个的单次运行成本也上升了。
演进智能体运行框架时，应追踪失去的任务、单次运行成本，以及在未见过的任务上的表现，而不只是平均分。
Original
New University of California, Berkeley+other top labs paper finds that agents that rewrite their own harness to score higher also lose tasks they once solved, which the average score hides. Harness evolution lets an agent edit its own prompts, skills, tools, or code while the model stays frozen. Results usually come as 1 higher benchmark score, which can hide lost tasks, higher costs, and gains that don't carry over. They ran 4 evolution methods, including GEPA and HyperAgents, with GPT-5.4 on 2 harnesses and 2 benchmarks. Then they re-tested each evolved agent 8 times against its starting version. All 16 evolved agents gained some tasks and lost others, and 1 raised reliability by 5.2 points while losing 10 tasks it used to cover. Of 11 agents that became more reliable, 8 also cost more per run. When you evolve an agent harness, track tasks lost, cost per run, and results on unseen tasks, not just the average score.