Back Rohan Paul

Rohan Paul· @rohanpaul_ai · X· 2026-11-15 08:08 · Original publication time AI score54

Northwestern and Google paper proposes a way to improve agent harnesses for new tasksMachine translation

AI introduction

In the research, Harness2 runs an agent, has an editor model revise its prompts, tools, skills and memory, then reruns the task to keep useful changes. It stores lessons for later tasks. The source reports that one improvement step raised GLM-5’s WorkBuddy-Bench score by 11.5 points, while harness edits without test runs lowered coding scores from 73.4 to 63.2.

Original post · Original

Rohan Paul@rohanpaul_ai

New Northwestern and Google paper shows that agent harnesses can improve themselves on new tasks without answer keys, by test-running edits and keeping what helps.

An agent's harness is the prompts, tools, skills, and memory wrapped around the model. Tuning it by hand is slow, and new real-world tasks come with no answer key to tune against.

Harness2 runs the agent, has an editor model propose harness fixes, then reruns the task to see which fixes improved the agent's behavior. Lessons from each task go into a library that guides edits on the next one.

A single improvement step raised GLM-5's WorkBuddy-Bench score by 11.5 points, past Gemini 3.5 Flash on its stock harness. Editing the harness without those test runs instead dropped coding scores from 73.4 to 63.2.

Treat your agent's harness as something to revise per task, and keep only the edits that change its behavior on real runs.

08:08 · 2026-11-15 · UTC+8

Found an error? Send a correction