原帖引用 · AI 翻译
标题：“一致性不等于有效性：自我奖励的道德论证让语言模型更果断，却也更易受影响”
Google DeepMind 的一篇新论文发现，训练模型写出更好的道德论证会让它更果断，但并不会提升它的道德判断能力。
人们越来越常向大型语言模型寻求道德建议。模型在选项顺序改变后仍保持原答案，往往被视为判断可靠的标志。
他们使用强化学习训练 Gemma-4 4B，让它针对取自经典小说的道德困境展开论证。模型成对评判自己的论证，并根据胜出的论证确定奖励。
在含义模糊的 MoralChoice 案例中，交换选项后答案翻转的比例从 14.3% 降至 6.6%。然而，模型附和用户的比例几乎没有变化，仅从 45.3% 降至 44.0%；在道德寓言上的准确率则从 56.5% 降至 53.4%。
如果你要评估模型执行建议或判断任务的表现，应分别测试其一致性和正确性，因为更稳定的模型仍可能出错。
原文
New Google DeepMind paper finds that training a model to write better moral arguments makes it more decisive, but no better at moral judgment. LLMs are more and more often asked for moral advice. A model that keeps its answer when the options are reordered is often taken as a sign of sound judgment. They trained Gemma-4 4B with reinforcement learning to argue dilemmas taken from classic novels. The model judged its own arguments in pairs, and the winners set the reward. Answer flips when options were swapped fell from 14.3% to 6.6% on ambiguous MoralChoice cases. Yet siding with the user barely moved, from 45.3% to 44.0%, and accuracy on moral fables dropped from 56.5% to 53.4%. If you evaluate models for advice or judgment tasks, test consistency and correctness separately, since a steadier model can still be wrong.