Original post · Original
Bloomberg: Microsoft is switching off AI servers it rented from Oracle after declining to renew a 2023 contract built around Bing.
The capacity lost value over time because Oracle’s hardware ran older-generation Nvidia chips that no longer matched Microsoft’s specifications, according to people familiar with the matter.
Basically, Microsoft pays per GPU-hour. A Blackwell GPU produces several times more inference output than an H100 for similar power, so each Hopper or Ampere hour now buys far less work.
Current frontier models, often large mixture-of-experts designs, need lots of memory and fast links between GPUs. An H100 has 80GB of memory and talks quickly to only 8 GPUs in its server. Blackwell racks link 72 GPUs as one unit, with 192 to 288GB each. Big models either don’t fit on the old boxes or run slowly across their slower network.
Oracle says demand for its AI infrastructure remains strong, and in September it renewed contracts for 4-year-old GPUs at a 20% premium to the original price.