Back Rohan Paul

Rohan Paul· @rohanpaul_ai · X· 2026-10-12 14:11 · Original publication time AI score54

AWS and HSBC paper proposes evolving the grader alongside the agentMachine translation

Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.

AI introduction

The paper describes Double Ratchet, which uses 10 labeled examples to select a grader version and evolves the grader alongside the agent’s skills. Selection is based on agreement with the examples, not on raising the agent’s score. The paper reports that, across code, SQL, and reports, the method retained 88% to 110% of the gains achieved by the same skill loop using ground truth or a rubric.

Original post · Original

The article text is unavailable in this language; an existing version is shown.

Rohan Paul@rohanpaul_ai

Self-improving agents have always needed a human to write the grader.

A new AWS + HSBC paper removes that step: the grader evolves alongside the agent from 10 labeled examples and trains it about as well as ground truth.

Double Ratchet evolves the grader alongside the agent's skills, building it from small, mostly rule-based checks for 1 failure type each. A grader version is chosen for agreeing with 10 labeled examples, never for raising the agent's score.

It kept 88% to 110% of the gain that ground truth or a rubric gave the same skill loop, across code, SQL, and reports. With the safeguards removed, the grader passed 97% to 100% of outputs yet trained skills as well.

14:11 · 2026-10-12 · UTC+8

Found an error? Send a correction