Original post · Original
Rohan Paul@rohanpaul_ai
Meta just opened live video calls with its Muse AI agent to every US user.
Callers see an animated avatar they can restyle as a person, animal or other character, and it reacts while listening.
Muse Realtime Voice emits speech tokens carrying words and delivery, and Muse Realtime Avatar turns the same stream into lip-synced video.
To run that live, Meta trained a student model to reproduce in 2 passes per video chunk what a teacher built in 120.
the system streams 448x768 portrait video at 25 frames per second and starts replying about 870ms after a user stops talking.
Adding 4-bit quantization, caching and dynamic batching lets a single Nvidia GB200 run 12 video streams at once.
In company-run tests of 2- to 3-minute calls, raters preferred the avatar to Runway Characters 78% to 22% and HeyGen LiveAvatar 88% to 12%.
At 12 streams per GB200, 1M simultaneous callers would need about 83K of them for video alone, so serving cost could cap the avatar’s reach.
View this post on X ↗
· UTC+8