aifollow.news
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score34

Sciforium shows DeepSeek-V4.1-Flash generating 1,129 tokens per second on four AMD GPUsMachine translation

AI introduction

Sciforium shows DeepSeek-V4.1-Flash reaching 1,129 output tokens per second at P90 on four AMD Instinct MI355X GPUs while retaining DeepSeek’s original weights. Its traffic-tuned speculative decoding drafter increases acceptance of proposed tokens, while kernel work fuses some operations and combines all-reduce with compute.

Original post · Original

@rohanpaul_ai

Sciforium shows DeepSeek-V4.1-Flash reaching 1,129 output tokens/sec at P90 on 4 AMD Instinct MI355X GPUs, keeping DeepSeek's original weights.

The gain comes from speculative decoding done better. Sciforium's Traffic-tuned Drafter is tuned on customer traffic, so more of its proposed tokens get accepted and draft length could be raised to seven tokens.

Separately, kernel work fused small ops like mHC and projections and fused all-reduce with compute.

At that rate, 10,000 tokens is about nine seconds of generation.

QuoteHassan Akbari@hassanience
not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project. a little faith, a lot of engineering. here’s how we did it. https://t.co/3boDmk3LQF
View this post on X ↗

Source: Rohan Paul · x.com

Products
Found an error? Send a correction