aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

Route Anything 研究发现:模型路由主要靠降低成本体现价值

AI 导读

Route Anything 在25项基准上比较了452种候选模型和策略,并按质量与成本评估路由效果。理想路由器可比最佳单一模型高5.4分,但学习得到的路由器在16项基准上仅提高0.3分;编码任务中,路由的单位美元效率达到始终调用 Opus 4.6 的25.5倍,质量低0.44分。

原帖引用 · 原文

该语言的正文暂不可用,当前显示已有版本。

@rohanpaul_ai

Great new study on model routing.

From University of California, Berkeley + other labs

Finds that model routers pay off significantly by saving money, averaging 8.82X more quality per unit cost while losing under 0.5 points.

Routers pick which model handles each input, hoping to match the best model for less or beat it. Most routing research has studied only language models.

Route Anything ran 452 candidate models and policies on the same inputs across 25 benchmarks, from text QA to robot control. They trained routers on these 374,401 scored runs and measured both quality and cost.

A perfect per-input router would beat the best single model by 5.4 points, but learned routers gained only 0.3 across 16 benchmarks. On coding tasks, routing reached 25.5× the dollar efficiency of always calling Opus 4.6 at a 0.44-point loss.

Use routers mainly to move work to cheaper models when quality is close, and compare candidates on the same inputs before trusting gains.

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错