UC Berkeley paper uses data generation pipeline to reveal fine-tuning risksMachine translation

New UC Berkeley paper finds that anyone giving an AI model extra training on their own data may add harmful behaviors without knowing, because the risky data can look normal even to experts. Fine-tuning data can change a model in ways nobody expected, like insecure code causing broad misalignment. Only a few dozen such cases were known, mostly found by accident. They built a pipeline that searches for small datasets that look like normal training data but raise a chosen behavior. Targets included recommending self-harm and resisting shutdown. It produced 6,575 datasets across 8 behaviors, and 72% also caused the behavior in all 5 other models tested, including one 8 times larger. ML researchers guessed the side effect correctly in 63% of trials, against 50% by chance. Screen fine-tuning data with behavior evals after training, or at least a frontier LLM auditor, since human reviewers barely beat chance at spotting the risk.