aifollow.news 搜索
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分47

RouterPlus 上线,面向小团队提供合并采购的推理定价

自动核验发布 · 本文由系统生成并完成证据核验,未经人工审稿。

AI 导读

RouterPlus 将多个团队的推理需求汇集起来,与模型服务商谈一笔大额交易,各团队按自己的用量支付谈定的费率。它还会用团队的实际请求测试模型,挑选满足需求且成本最低的选项;材料未给出具体报价或节省幅度。

原帖引用 · 原文

该语言的正文暂不可用,当前显示已有版本。

@rohanpaul_ai

Inference pricing has a size penalty. The best per-token rates go to whoever can commit the most volume.

RouterPlus groups demand from many teams into one large deal with a provider, and each team pays that rate on its share of the traffic.

Large buyers have done this with cloud commits for years. Applied to tokens, I’d guess it works best for popular open models, where many teams already want the same weights served on the same kind of hardware.

For a small team, this is a route to bulk pricing that doesn’t require being a bulk buyer. They also test models on your real usage and pick the one that's good enough at the lowest cost. Many teams pay for a top model on every task when a cheaper one could do most of the work.

引用Alok Bishoyi@alokbishoyi97
Getting your inference right shouldn't be hard. @useRouterPlus (YC F26) is live: the right model and provider for your workload, at the best price. check out : https://t.co/HDYLVRYMEC https://t.co/8WbdbK2z6i
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

产品
发现内容有误?提交纠错