aifollow.news
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score27

Post claims AMD GPUs reached 1,129 output tokens per second on production trafficMachine translation

AI introduction

A post claims AMD GPUs reached 1,129 output tokens per second at P90 on real production traffic. A separate comment says the team tested running all-reduce on a separate stream but dropped the approach because it competed for memory bandwidth.

Original post · AI translation

@rohanpaul_ai
Translation · refer to the original

@hassanience 值得注意的是,他们测试了在单独的流上运行 all-reduce,但因为它会争抢内存带宽而放弃了这一做法。这是一个有用的负面结果。

ReplyHassan Akbari@hassanience
Translation
就在不久前,人们还在质疑 AMD GPU 能否跟上。如今,我们正用它们创下纪录。P90 达到每秒 1,129 个输出 token。真正的生产流量,不是什么玩具项目。 一点信念,大量工程投入。我们是这样做到的。https://t.co/3boDmk3LQF
Original

not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project. a little faith, a lot of engineering. here’s how we did it. https://t.co/3boDmk3LQF

View this post on X ↗

Source: Rohan Paul · x.com

Industry
Found an error? Send a correction