Original post · Original
The article text is unavailable in this language; an existing version is shown.
New University of California, Berkeley+other top labs paper finds that agents that rewrite their own harness to score higher also lose tasks they once solved, which the average score hides.
Harness evolution lets an agent edit its own prompts, skills, tools, or code while the model stays frozen. Results usually come as 1 higher benchmark score, which can hide lost tasks, higher costs, and gains that don't carry over.
They ran 4 evolution methods, including GEPA and HyperAgents, with GPT-5.4 on 2 harnesses and 2 benchmarks. Then they re-tested each evolved agent 8 times against its starting version.
All 16 evolved agents gained some tasks and lost others, and 1 raised reliability by 5.2 points while losing 10 tasks it used to cover. Of 11 agents that became more reliable, 8 also cost more per run.
When you evolve an agent harness, track tasks lost, cost per run, and results on unseen tasks, not just the average score.