aifollow.news
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score34

Sciforium shows DeepSeek-V4.1-Flash generating 1,129 tokens per second on four AMD GPUsMachine translation

AI introduction

Sciforium shows DeepSeek-V4.1-Flash reaching 1,129 output tokens per second at P90 on four AMD Instinct MI355X GPUs while retaining DeepSeek’s original weights. Its traffic-tuned speculative decoding drafter increases acceptance of proposed tokens, while kernel work fuses some operations and combines all-reduce with compute.

Original post · AI translation

@rohanpaul_ai
Translation · refer to the original

Sciforium 展示了 DeepSeek-V4.1-Flash 在 4 块 AMD Instinct MI355X GPU 上以 P90 达到每秒输出 1,129 个 token,同时保留 DeepSeek 的原始权重。

这一提升来自对推测解码的改进。Sciforium 的 Traffic-tuned Drafter 根据客户流量进行调优,因此它提出的 token 有更多被接受,草稿长度也得以提高到 7 个 token。

另外,内核方面的工作融合了 mHC 和投影等小型操作,并将 all-reduce 与计算融合。

按这一速度,生成 10,000 个 token 大约需要 9 秒。

QuoteHassan Akbari@hassanience
Translation
不久前,人们还在质疑 AMD GPU 能否跟上。今天,我们正在它们上面创下纪录。P90 下每秒输出 1,129 个 token。真实的生产流量,不是玩具项目。 一点信心,大量工程工作。以下是我们如何做到的。https://t.co/3boDmk3LQF
Original

not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project. a little faith, a lot of engineering. here’s how we did it. https://t.co/3boDmk3LQF

View this post on X ↗

Source: Rohan Paul · x.com

Products
Found an error? Send a correction