aifollow.news
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score54

Study finds that low model capability scores can be precisely manufacturedMachine translation

AI introduction

A study used a LoRA adjustment to steer a Qwen model toward low target scores on unseen tests, coming within 1.9 points; prompting models to underperform missed by 42 to 47 points. The adjustment left a telltale pattern: on multiple-choice questions, up to 97% of its new errors fell on one answer letter. The study suggests checking which items a model misses and how its errors are distributed before treating a low score as evidence of low dangerous capability.

Original post · AI translation

@rohanpaul_ai
Translation · refer to the original

这篇论文发现,低基准测试分数可以被精确制造出来,因此单凭低分并不能说明模型具备什么能力。

安全测试常把危险能力探测中的低分视为令人放心的信号。但低分也可能是模型的表现受到抑制所致,因此这个数字本身证明不了多少。

他们训练了一个 LoRA“旋钮”,使一个 Qwen 模型向较小的 Qwen 模型靠拢,然后将它的错误与真实较小模型的错误进行比较。他们还尝试通过提示让模型表现变差、提高温度,以及给权重添加噪声。

这个旋钮在未见过的目标分数上将误差控制在 1.9 分以内,而提示方法的误差为 42 至 47 分。但在选择题中,它新增的错误最多有 97% 集中在 1 个选项字母上;他们测试的自然模型中,没有任何一个接近这一比例。

在把低分解读为能力低之前,应检查模型在哪些题目上出错,以及错误答案如何分布。

Source: Rohan Paul · x.com

Research
Found an error? Send a correction