aifollow.news 搜索
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

工具调用智能体研究:增加应用和数据库状态改善泛化表现

自动核验发布 · 本文由系统生成并完成证据核验,未经人工审稿。

AI 导读

一项关于工具调用智能体的研究构建了 SynthAPI,用语言模型生成带有真实 Python 工具和数据库的模拟应用,并自动检查任务是否完成。在固定 32,768 个任务的实验中,增加应用和数据库版本后,4B 模型在 AppWorld 挑战集上的成绩从 38.4% 升至 45.0%。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/DrscMqSwSN

Title: "Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis"

回复Rohan Paul@rohanpaul_ai
New Apple paper finds that, as training data grows, tool-use agents gain more from diverse apps and database states than from more tasks. that the mix of synthetic training data, not just its size, decides how well tool-use agents generalize to new benchmarks. Teams train tool-use agents on synthetic environments, and most pipelines focus on generating more tasks. It was unclear whether new tools, new tasks, or new starting data matter most. They built SynthAPI, which has an LLM code synthetic apps with real Python tools and databases, then write tasks with automatic success checks. They varied apps, tasks, and database versions across 849 trained models. At a fixed 32,768-task budget, more apps and database versions lifted a 4B model on AppWorld's challenge split from 38.4% to 45.0%. Their scaling law, which tracks data mix, predicted larger held-out runs with 0.0285 error, versus 0.1706 for a task-count-only law.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错