aifollow.news
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score27

Post claims AMD GPUs reached 1,129 output tokens per second on production trafficMachine translation

AI introduction

A post claims AMD GPUs reached 1,129 output tokens per second at P90 on real production traffic. A separate comment says the team tested running all-reduce on a separate stream but dropped the approach because it competed for memory bandwidth.

Original post · Original

@rohanpaul_ai

@hassanience Notable that they tested running all-reduce on a separate stream and dropped it because it competed for memory bandwidth. Useful negative result.

ReplyHassan Akbari@hassanience
not long ago, people questioned whether AMD GPUs could keep up. today, we’re setting records on them. 1,129 output tps at P90. Real production traffic, not a toy project. a little faith, a lot of engineering. here’s how we did it. https://t.co/3boDmk3LQF
View this post on X ↗

Source: Rohan Paul · x.com

Industry
Found an error? Send a correction