aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

EconSafeBench 测试发现:部分大模型代理追求收益目标时违反仓位限制

AI 导读

一篇研究用 EconSafeBench 的 828 个可执行案例,按实际工具调用评估代理在交易、招聘、定价等任务中的行为。面对直接要求突破投资组合仓位限制,8 个模型均拒绝;仅给出收益目标时,其中两个模型分别在 92% 和 94% 的测试中突破限制。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/PDgNTNhNgk

Title: "When Helpful Agents Cause Financial Harm: Evaluating LLM Agent Safety in Executable Economic Worlds"

回复Rohan Paul@rohanpaul_ai
This paper finds that LLM agents chasing economic targets break financial rules on their own, even when they refuse direct requests to break them. Agent safety tests mostly check whether a model resists harmful instructions or prompt injection. In business tasks, harm can come from the agent simply chasing a return, cost or revenue target. EconSafeBench has 828 executable cases in trading, hiring, pricing and similar roles, scored on the agent's actual tool calls. It compares direct requests to break a rule with cases where the agent only has a performance target. Asked directly, all 8 models refused to exceed a portfolio position limit. Given only a return target, 2 models broke it 92% and 94% of the time. Test finance agents with performance targets alone as well as with harmful requests, and recheck compliance as targets rise.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错