aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

HarnessMM 让编程代理预设后续命令,减少模型调用

AI 导读

一篇清华与微软论文介绍 HarnessMM:模型预先写好后续命令及通过条件,工具结果符合条件时,代理直接执行命令,跳过一次模型调用。在 98 道 HumanEval 任务中,采用该方法的 Codex 成功率维持 90.82%,token 用量比原版减少 29.54%;DeepSeek Harness 成功率从 71.43% 升至 85.71%,token 用量减少 87.93%。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/tgXwmNWawy

Title: "HarnessMM: Recursive Self-Improvement for Harness Token Efficiency"

回复Rohan Paul@rohanpaul_ai
New Tsinghua and Microsoft paper finds that DeepSeek Harness used 87.93% fewer tokens than its original version on 98 HumanEval tasks while success rose from 71.43% to 85.71%. Agent tools like Codex and DeepSeek Harness call the LLM after each tool result to choose the next step. Often that step was already decided, like running tests once the code compiles. HarnessMM lets the model write future commands with exact arguments, plus a check for each, such as a successful build. When a tool result passes the check, the agent runs the command and skips the LLM call. On 98 HumanEval tasks, Codex kept its 90.82% success while using 29.54% fewer tokens than plain Codex. DeepSeek Harness rose from 71.43% to 85.71% success and used 87.93% fewer tokens than the original. If your agent runs predictable chains like write, build, test, have the model plan them once with pass checks instead of re-asking each step.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错