Original post · Original
The article text is unavailable in this language; an existing version is shown.
Sciforium shows DeepSeek-V4.1-Flash reaching 1,129 output tokens/sec at P90 on 4 AMD Instinct MI355X GPUs, keeping DeepSeek's original weights.
The gain comes from speculative decoding done better. Sciforium's Traffic-tuned Drafter is tuned on customer traffic, so more of its proposed tokens get accepted and draft length could be raised to seven tokens.
Separately, kernel work fused small ops like mHC and projections and fused all-reduce with compute.
At that rate, 10,000 tokens is about nine seconds of generation.
not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project.
a little faith, a lot of engineering. here’s how we did it. https://t.co/3boDmk3LQF
View this post on X ↗