Original post · AI translation
Rohan Paul@rohanpaul_ai
萨姆·奥尔特曼谈本地推理。
“从全球资源分配的角度看，让每个人都把全球有限的 DRAM 供应中的一部分放在自己的电脑里，而且只有 5% 的时间用得上，这相当糟糕。”
按每个 token 计算，数据中心轻松胜出。批量大小为 1 的本地模型，无论按哪种利用率指标衡量都很浪费，但它也是整个技术栈中唯一没人能替你弃用的部分。https://t.co/w9A93jliU9
· UTC+8
Sam Altman questions the memory efficiency of local inferenceMachine translation
Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.
Sam Altman argues that, from a global resource allocation perspective, it is inefficient for limited DRAM to sit in personal computers and be used only 5% of the time. The post’s author says datacenters are more efficient per token, while local models cannot be deprecated by a service provider.