Original post · AI translation
你可以在这里查看实时结果
语音 AI 现在已经足够出色，可以将偏好度和类人程度作为两个独立目标进行评分。
@voicearena_ai 推出了一项名为 Turing TTS 的新基准测试，直接对后者评分：听众听同一句话分别由真人和模型说出，然后选出真人。
他们选中模型的频率就是“误认率”（Fool Rate）。50% 意味着听众无法分辨。
即使是最好的模型 Gemini 3.8 Flash TTS，也只在约 10% 的情况下骗过了听众。
在 Voice Arena 的偏好排行榜上，前 4 个模型在统计上并列。以真人语音为参照，整个领域仍有很长的路要走。
https://t.co/AUS2S4Wp5h
Original
Voice AI is now good enough that preference and human-likeness can be scored as separate targets. @voicearena_ai launched a new benchmark, Turing TTS to score the second one directly: listeners hear the same sentence from a person and from a model, then pick the person. How often they pick the model is the Fool Rate. 50% would mean listeners can’t tell. Even the best model, Gemini 3.8 Flash TTS, fooled them about 10% of the time. On Voice Arena’s preference board, the top four models are statistically tied. Measured against a human, the whole field still has a long way to go. https://t.co/AUS2S4Wp5h
第 2/15 条
我们实际衡量的，真的是我们最终希望语音 AI 模型实现的目标吗？
最好的 AI 语音正变得难以区分。
它们彼此之间难以区分，而不是与真人语音难以区分。
🚀 隆重推出 Turing TTS：一项评估，衡量真人听众有多频繁地将 AI 生成的语音误认为真人语音。
我们将领先的 TTS 模型与真人语音进行了对比测试。即使是最好的模型，也只在约 10% 的情况下骗过了听众。
我们希望通过 Turing TTS，为研究界关于如何衡量语音 AI 进展的更广泛讨论作出贡献。
如果我们的目标之一是开发出与真人语音无法区分的语音，那么目前的评估方法在多大程度上反映了我们朝这一目标取得的进展？
我们并不声称 Turing TTS 是正确的前进方向，也不认为与真人语音无法区分应当成为每个 TTS 系统的目标。
相反，我们希望这些发现能促使研究界共同审视：对自然度进行盲测式的人类两两比较是否足够，以及我们还可能需要衡量什么。
结果现已在 @voicearena_ai 上线。
Original
Drop 2/15 Are we actually measuring the things we ultimately want our voice AI models to achieve? The best AI voices are becoming hard to tell apart. From each other. Not from humans. 🚀 Introducing Turing TTS: an evaluation that measures how often human listeners mistake AI-generated speech for real human speech. We tested leading TTS models against real human speech. Even the best model fooled listeners only ~10% of the time. With Turing TTS, we hope to contribute to a broader discussion within the research community about how we measure progress in voice AI. If one of our goals is to develop speech that is indistinguishable from human speech, how well do our current evaluation methods capture progress toward that goal? We don’t claim that Turing TTS is the right way forward, or that human indistinguishability should be the goal of every TTS system. Rather, we hope these findings encourage the community to collectively examine whether blind pairwise human evaluations of naturalness are sufficient, and what else we might need to measure. Results are now live on @voicearena_ai.