aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分27

帖子称 AMD GPU 在生产流量中达到每秒 1,129 个输出 token

AI 导读

一则帖子称,AMD GPU 在真实生产流量下达到 P90 每秒 1,129 个输出 token。另有评论指出,团队曾测试在独立流上运行 all-reduce,但因其争用内存带宽而放弃这一做法。

原帖引用 · 原文

@rohanpaul_ai

@hassanience Notable that they tested running all-reduce on a separate stream and dropped it because it competed for memory bandwidth. Useful negative result.

回复Hassan Akbari@hassanience
not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project. a little faith, a lot of engineering. here’s how we did it. https://t.co/3boDmk3LQF
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

行业
发现内容有误?提交纠错