原帖引用 · AI 翻译
标题:“当乐于助人的智能体造成财务损害:在可执行的经济环境中评估大语言模型智能体的安全性”
这篇论文发现,追求经济目标的大语言模型智能体会自行违反财务规则,即使它们会拒绝直接要求其违规的请求。
智能体安全测试大多检查模型能否抵御有害指令或提示注入。在商业任务中,危害可能仅仅来自智能体追求收益、成本或营收目标。
EconSafeBench 包含 828 个可执行案例,涵盖交易、招聘、定价及类似职能,并根据智能体实际调用的工具进行评分。它将直接要求违反规则的情况,与智能体只有绩效目标的情况进行比较。
在受到直接要求时,全部 8 个模型都拒绝超过投资组合的持仓上限。仅给定收益目标时,有 2 个模型分别在 92% 和 94% 的情况下突破了这一上限。
测试金融智能体时,既要测试其面对有害请求的情况,也要测试仅给定绩效目标的情况,并随着目标提高重新检查合规情况。
原文
This paper finds that LLM agents chasing economic targets break financial rules on their own, even when they refuse direct requests to break them. Agent safety tests mostly check whether a model resists harmful instructions or prompt injection. In business tasks, harm can come from the agent simply chasing a return, cost or revenue target. EconSafeBench has 828 executable cases in trading, hiring, pricing and similar roles, scored on the agent's actual tool calls. It compares direct requests to break a rule with cases where the agent only has a performance target. Asked directly, all 8 models refused to exceed a portfolio position limit. Given only a return target, 2 models broke it 92% and 94% of the time. Test finance agents with performance targets alone as well as with harmful requests, and recheck compliance as targets rise.