Back Rohan Paul

Rohan Paul· @rohanpaul_ai · X· 2026-10-30 07:24 · Original publication time AI score20

Sam Altman questions the memory efficiency of local inferenceMachine translation

Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.

AI introduction

Sam Altman argues that, from a global resource allocation perspective, it is inefficient for limited DRAM to sit in personal computers and be used only 5% of the time. The post’s author says datacenters are more efficient per token, while local models cannot be deprecated by a service provider.

Original post · Original

Rohan Paul@rohanpaul_ai

Sam Altman on local inference.

"from a global allocation perspective, it's quite terrible to have everybody, a bunch of the world's limited supply of DRAM sitting in their own computer and being used 5% of the time"

per token, the datacenter wins easily. A local model at batch size 1 is wasteful by any utilization metric, and it’s also the only part of the stack nobody can deprecate on you. https://t.co/w9A93jliU9

07:24 · 2026-10-30 · UTC+8

Found an error? Send a correction