正文 · AI 翻译
Thanks to IT Home reader 愚公骑马 for the tip!
IT Home reported on October 29 that NVIDIA announced the open-weight speech transcription model Nemotron 3.5 Transcribe yesterday (October 28) and simultaneously made its training data pipeline, Granary v2, public.
Nemotron 3.5 Transcribe combines transcription, speaker attribution, translation, and audio question answering. It is aimed primarily at uses such as meeting minutes, classroom recordings, interview transcription, and multilingual customer service.
Architecturally, Nemotron 3.5 Transcribe uses a mixture-of-experts (MoE) architecture, with approximately 34 billion total parameters and 4.3 billion active parameters. It can process recordings up to 1 hour long and supports transcription in 40 languages, 31 of which can be translated to and from English. For speaker identification, it can distinguish up to 8 speakers and supports speaker labels in 13 languages.
The model's main highlight is its ability to handle multiple people speaking at the same time. It can label the speaker for each utterance as it generates text. In an official demonstration, it generated subtitles by character even for a scene from the 1940 film His Girl Friday featuring extremely fast speech and extensive overlapping dialogue.
In public evaluations, the model ranked first among open-weight models in the English category of the Hugging Face Open ASR Leaderboard, with an average word error rate of 5.33%.
On Artificial Analysis's non-streaming word error rate index, it tied with Voxtral Small Transcribe for first place among open-source models at 2.8%. For speaker attribution, it recorded the lowest error rate, 18.26%, on David AI's speaker diarization benchmark covering 21 languages. IT Home includes the relevant image below:
The model is available for free on Hugging Face and ModelScope under the OpenMDW-1.1 license. Quantized versions are expected to be released later to facilitate running it on local devices such as RTX Spark, DGX Spark, and Jetson AGX Thor.
Granary v2 is an open, reusable training-data curation pipeline and dataset built by NVIDIA for multilingual speech models such as Nemotron 3.5 Transcribe.
It automatically processes raw audio or audio with existing annotations into multiple transcription formats, translations, subtitles, question-answer pairs, and instruction pairs for use in ASR, speech translation, and speech-instruction fine-tuning.
The published Granary v2 data inventory covers 463,062 hours of audio, 48 languages, and 16 source corpora, a substantial expansion from the 25 languages in Granary v1.
For the same recording, Granary v2 generates at least three transcription variants corresponding to different written styles that the model can be prompted to produce:
Text-normalized verbatim transcript (tn_raw): Preserves disfluencies such as “uh” and “um” but normalizes the text form of numbers and similar elements, for example, “I paid, uh, twenty-five euros.”
Inverse text normalization with disfluencies (itn_raw): Converts spoken numbers into written symbols, as in “I paid, uh, €25.”
Inverse text normalization without disfluencies (itn_clean): Produces clean, readable text, as in “I paid €25.”