Original post · AI translation
标题：“大规模生成训练数据带来意想不到的副作用”
加州大学伯克利分校的一篇新论文发现，任何用自己的数据对 AI 模型进行额外训练的人，都可能在不知情的情况下让模型产生有害行为，因为有风险的数据即使在专家看来也可能很正常。
微调数据可能以无人预料的方式改变模型，例如，不安全的代码可能导致广泛的目标偏离。此前已知的此类案例只有几十起，而且大多是偶然发现的。
他们构建了一套流程，用于寻找看起来像普通训练数据、却能增强某种指定行为的小型数据集。目标行为包括建议自残和抵抗关闭。
这套流程针对 8 种行为生成了 6,575 个数据集，其中 72% 在另外接受测试的全部 5 个模型中也引发了相应行为，包括一个规模大 8 倍的模型。机器学习研究人员在 63% 的试验中正确猜出了副作用，而随机猜测的正确率为 50%。
应在训练后通过行为评估筛查微调数据，或至少使用前沿大语言模型进行审核，因为人工审查人员识别风险的表现仅略高于随机猜测。
Original
New UC Berkeley paper finds that anyone giving an AI model extra training on their own data may add harmful behaviors without knowing, because the risky data can look normal even to experts. Fine-tuning data can change a model in ways nobody expected, like insecure code causing broad misalignment. Only a few dozen such cases were known, mostly found by accident. They built a pipeline that searches for small datasets that look like normal training data but raise a chosen behavior. Targets included recommending self-harm and resisting shutdown. It produced 6,575 datasets across 8 behaviors, and 72% also caused the behavior in all 5 other models tested, including one 8 times larger. ML researchers guessed the side effect correctly in 63% of trials, against 50% by chance. Screen fine-tuning data with behavior evals after training, or at least a frontier LLM auditor, since human reviewers barely beat chance at spotting the risk.