原帖引用 · AI 翻译
标题：“通过可控环境合成研究工具使用智能体的扩展定律”
苹果公司的一篇新论文发现，随着训练数据增加，工具使用智能体从多样化的应用和数据库状态中获得的收益，比从更多任务中获得的收益更大。
决定工具使用智能体在新基准测试上泛化能力的，是合成训练数据的组成，而不只是其规模。
团队在合成环境中训练工具使用智能体，而大多数流程侧重于生成更多任务。此前尚不清楚，新工具、新任务或新的初始数据，哪一项最重要。
他们构建了 SynthAPI：由一个大语言模型编写带有真实 Python 工具和数据库的合成应用，再编写带有自动成功判定机制的任务。他们在 849 个训练好的模型中调整了应用、任务和数据库版本。
在固定的 32,768 个任务预算下，增加应用和数据库版本，使一个 4B 模型在 AppWorld 挑战划分上的成绩从 38.4% 提升至 45.0%。他们将数据组成纳入考量的扩展定律，在预测规模更大的留出实验时误差为 0.0285，而仅考虑任务数量的定律误差为 0.1706。
原文
New Apple paper finds that, as training data grows, tool-use agents gain more from diverse apps and database states than from more tasks. that the mix of synthetic training data, not just its size, decides how well tool-use agents generalize to new benchmarks. Teams train tool-use agents on synthetic environments, and most pipelines focus on generating more tasks. It was unclear whether new tools, new tasks, or new starting data matter most. They built SynthAPI, which has an LLM code synthetic apps with real Python tools and databases, then write tasks with automatic success checks. They varied apps, tasks, and database versions across 849 trained models. At a fixed 32,768-task budget, more apps and database versions lifted a 4B model on AppWorld's challenge split from 38.4% to 45.0%. Their scaling law, which tracks data mix, predicted larger held-out runs with 0.0285 error, versus 0.1706 for a task-count-only law.