aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分47

Deepnote 发布 Agent Initiative Benchmark,评测 AI 代理何时介入 Slack 对话

AI 导读

Deepnote 发布 Agent Initiative Benchmark,将主动型 AI 代理放入模拟公司的 Slack,测试其是否知道何时介入、说什么以及何时保持沉默。测试中,GPT-6 Astra 得分 8.97/10;评分由两个 LLM 评审模型参照参考答案给出。

原帖引用 · 原文

@rohanpaul_ai

@DeepnoteHQ beautiful product.

回复Deepnote@DeepnoteHQ
@rohanpaul_ai Thanks for sharing @rohanpaul_ai 💙
在 X 查看这条帖子 ↗
相关上下文Rohan Paul@rohanpaul_ai
This is a beautiful concept for a benchmark. Deepnote (@deepnotehq), an agent workspace company, released the Agent Initiative Benchmark, a test that drops proactive agents into a company Slack and scores whether they know when to jump in, what to say, and when to stay quiet. GPT-6 Astra led with 8.97 out of 10, and GPT-6.1 Sol followed at 8.65 for $0.11 per task, against $1.87 per task for third-placed Claude Fable 5.2. It runs every model through the same loop: > A thread starts in a simulated company’s Slack > The agent decides whether to act at all > It picks which of 11 services to check and investigates > It posts and tags the right people, or stays silent > Two LLM evaluators score the run 0 to 10 against a reference answer
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

产品
发现内容有误?提交纠错