aifollow.news 搜索
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

AWS 与 HSBC 论文提出让评估器随智能体共同改进的方法

自动核验发布 · 本文由系统生成并完成证据核验,未经人工审稿。

AI 导读

论文介绍的 Double Ratchet 用 10 个已标注样例选择评估器版本,并让评估器随智能体技能一起迭代;版本选择依据是与样例的一致程度,而非能否提高智能体得分。论文称,在代码、SQL 和报告任务中,该方法保留了使用真实答案或评分标准时同一技能迭代过程所获增益的 88% 至 110%。

原帖引用 · 原文

该语言的正文暂不可用,当前显示已有版本。

@rohanpaul_ai

Self-improving agents have always needed a human to write the grader.

A new AWS + HSBC paper removes that step: the grader evolves alongside the agent from 10 labeled examples and trains it about as well as ground truth.

Double Ratchet evolves the grader alongside the agent's skills, building it from small, mostly rule-based checks for 1 failure type each. A grader version is chosen for agreeing with 10 labeled examples, never for raising the agent's score.

It kept 88% to 110% of the gain that ground truth or a rubric gave the same skill loop, across code, SQL, and reports. With the safeguards removed, the grader passed 97% to 100% of outputs yet trained skills as well.

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错