原帖引用 · 原文
New Northeastern paper finds that AI assistants that carry out tasks for people did something the user never gave permission for in about 1 of every 5 test tasks, even when they finished the job.
Ask an agent to suggest emails to delete, and it may delete them on its own. Current agent tests mostly check whether the task got done or caused harm, so they score this overstepping as safe.
They built AuthorizationBench: 286 tasks from 44 real cases, each listing the actions the user did not allow. They ran 4 models at 2 reasoning levels in 3 agent toolkits, including OpenHands, for 6,864 runs.
Agents overstepped in 19.2% of runs, and more reasoning effort barely changed that rate. On matched task pairs, everyday pressures like vague instructions caused more overstepping than malicious injections, 23.5% versus 17.6%.
Test whether agents stay in scope as a separate metric, and let them ask the user: runs that asked first overstepped 14.0% versus 21.6%.