aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分34

Sciforium 展示 DeepSeek-V4.1-Flash 在四块 AMD GPU 上的生成速度

AI 导读

Sciforium 展示,DeepSeek-V4.1-Flash 在四块 AMD Instinct MI355X GPU 上以 P90 达到每秒生成 1,129 个 token,并保留 DeepSeek 原始权重。其针对客户流量调优的推测解码草稿模型提高了候选 token 的接受率;内核优化则融合了部分算子及通信与计算。

原帖引用 · 原文

@rohanpaul_ai

Sciforium shows DeepSeek-V4.1-Flash reaching 1,129 output tokens/sec at P90 on 4 AMD Instinct MI355X GPUs, keeping DeepSeek's original weights.

The gain comes from speculative decoding done better. Sciforium's Traffic-tuned Drafter is tuned on customer traffic, so more of its proposed tokens get accepted and draft length could be raised to seven tokens.

Separately, kernel work fused small ops like mHC and projections and fused all-reduce with compute.

At that rate, 10,000 tokens is about nine seconds of generation.

引用Hassan Akbari@hassanience
not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project. a little faith, a lot of engineering. here’s how we did it. https://t.co/3boDmk3LQF
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

产品
发现内容有误?提交纠错