aifollow.news
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score54

Tsinghua and Microsoft paper proposes HarnessMM to skip model calls after preset checks passMachine translation

AI introduction

HarnessMM has the model specify future commands and checks in advance; when a tool result passes a check, the agent executes the next command without another model call. Across 98 HumanEval tasks, Codex maintained 90.82% success while using 29.54% fewer tokens; DeepSeek Harness success rose from 71.43% to 85.71% while using 87.93% fewer tokens.

Original post · Original

The article text is unavailable in this language; an existing version is shown.

@rohanpaul_ai

New Tsinghua and Microsoft paper finds that DeepSeek Harness used 87.93% fewer tokens than its original version on 98 HumanEval tasks while success rose from 71.43% to 85.71%.

Agent tools like Codex and DeepSeek Harness call the LLM after each tool result to choose the next step. Often that step was already decided, like running tests once the code compiles.

HarnessMM lets the model write future commands with exact arguments, plus a check for each, such as a successful build. When a tool result passes the check, the agent runs the command and skips the LLM call.

On 98 HumanEval tasks, Codex kept its 90.82% success while using 29.54% fewer tokens than plain Codex. DeepSeek Harness rose from 71.43% to 85.71% success and used 87.93% fewer tokens than the original.

If your agent runs predictable chains like write, build, test, have the model plan them once with pass checks instead of re-asking each step.

Source: Rohan Paul · x.com

Research
Found an error? Send a correction