Researchers derive a limit on reward gains from language model alignmentMachine translation

The study derives the maximum expected reward gain from alignment under a fixed budget for divergence from the base model, and proposes a way to estimate it using base-model samples alone. In experiments on safety and summarization, best-of-N came close to the theoretical limit, while PPO and GRPO remained substantially below it. The analysis also links errors in proxy rewards to reward hacking.