Original post · AI translation
@PumpHustler 没错，这个说法很贴切。
@rohanpaul_ai 首次尝试表现更好，并不总意味着整体解决问题的能力更强。
Original
@rohanpaul_ai Better first attempts don’t always mean better problem solving overall
Meta 的一篇新论文发现，经过强化学习后训练的智能体首次尝试表现更好，但在获得 128 次尝试机会时，最终找到解决方案的能力反而更差。
人们认为强化学习会教会智能体调用工具之类的新技能。
这一说法主要来自数学和编程领域，而基础模型已经见过这些领域的大量数据。
他们在 3 个智能体基准测试中测试了 14 对基础模型和后训练模型，每项任务最多允许尝试 128 次。他们还构建了 PTGS，这是一种采样器，会对难提示词提高温度，对简单提示词降低温度。
在 WebShop 上，一个 gemma-4-31B 基础模型在 128 次尝试内解决了超过 85% 的任务，而后训练版本为 56%。他们计算出的 Sharpening Tax 在 42 个模型与基准测试组合中有 36 个为正，这意味着任务覆盖率缩小了。
在为可以重试的智能体选择后训练模型之前，应衡量它最终能解决多少任务，而不只是首次尝试的准确率。
Original
New Meta paper finds Reinforcement learning post-training makes an agent better on its first attempt and worse at ever finding a solution when it gets 128 attempts. People assume reinforcement learning teaches agents new skills like tool calling. That claim comes mostly from math and coding, where base models have already seen huge amounts of data. They ran 14 base and post-trained model pairs on 3 agentic benchmarks with up to 128 tries per task. They also built PTGS, a sampler that raises temperature on hard prompts and lowers it on easy ones. On WebShop a gemma-4-31B base model solved over 85% of tasks within 128 tries against 56% for the post-trained version. Their Sharpening Tax came out positive in 36 of 42 model and benchmark pairs, meaning coverage shrank. Before picking a post-trained model for an agent that can retry, measure how many tasks it ever solves, not just first-try accuracy.