Audio-Text-to-Text
Transformers
Safetensors
English
Chinese
moss_transcribe_diarize
text-generation
moss
audio
speech
asr
diarization
timestamp-asr
long-form-audio
multimodal
multilingual
custom_code
Eval Results
Instructions to use OpenMOSS-Team/MOSS-Transcribe-Diarize with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMOSS-Team/MOSS-Transcribe-Diarize with Transformers:
# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("OpenMOSS-Team/MOSS-Transcribe-Diarize", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update model card evaluation formatting
Browse files
README.md
CHANGED
|
@@ -25,7 +25,7 @@ pipeline_tag: audio-text-to-text
|
|
| 25 |
<a href="https://arxiv.org/abs/2601.01554"><img src="https://img.shields.io/badge/arXiv-2601.01554-b31b1b?logo=arxiv"></a>
|
| 26 |
</div>
|
| 27 |
|
| 28 |
-
MOSS-Transcribe-Diarize is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness.
|
| 29 |
|
| 30 |
Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as `[S01]`, `[S02]`, and beyond.
|
| 31 |
|
|
@@ -51,23 +51,20 @@ Given an audio or video file, the model generates a compact speaker-aware transc
|
|
| 51 |
|
| 52 |
## Introduction
|
| 53 |
|
| 54 |
-
MOSS-Transcribe-Diarize turns real-world long-form audio into structured, speaker-aware transcripts in one pass. Instead of stitching together separate ASR and diarization systems, it jointly performs speech transcription and speaker diarization, producing time-aligned text with consistent speaker labels.
|
| 55 |
|
| 56 |
The model is built for meetings, calls, podcasts, interviews, lectures, videos, and other long or messy multi-speaker recordings. It can also emit acoustic event annotations, giving downstream systems a richer view of what happened, who spoke, and when.
|
| 57 |
|
| 58 |
Core capabilities:
|
| 59 |
|
| 60 |
-
* **Long-form
|
| 61 |
-
* **Speaker-aware
|
| 62 |
-
* **
|
| 63 |
-
* **Timestamp prediction**: Generates segment-level start and end timestamps.
|
| 64 |
-
* **Audio and video input**: Supports common audio files and video containers through the source package utilities.
|
| 65 |
-
* **Promptable generation**: Supports custom transcription instructions and hotwords.
|
| 66 |
|
| 67 |
## Model Architecture
|
| 68 |
|
| 69 |
<p align="center">
|
| 70 |
-
<img src="Model_Architecture.png" alt="MOSS-Transcribe-Diarize model architecture" width="900">
|
| 71 |
</p>
|
| 72 |
|
| 73 |
| Component | Specification |
|
|
@@ -136,7 +133,7 @@ We evaluate MOSS-Transcribe-Diarize using three objective metrics: Character Err
|
|
| 136 |
<td>22.75</td><td>27.43</td><td>4.68</td>
|
| 137 |
<td>26.75</td><td>32.84</td><td>6.09</td>
|
| 138 |
<td>-</td><td>-</td><td>-</td>
|
| 139 |
-
<td>8.62</td><td>14.73</td><td>6.11</td>
|
| 140 |
</tr>
|
| 141 |
<tr>
|
| 142 |
<td style="white-space: nowrap;">VIBEVOICE ASR</td>
|
|
@@ -146,17 +143,17 @@ We evaluate MOSS-Transcribe-Diarize using three objective metrics: Character Err
|
|
| 146 |
<td>14.59</td><td>42.54</td><td>27.94</td>
|
| 147 |
</tr>
|
| 148 |
<tr>
|
| 149 |
-
<td style="white-space: nowrap;"><b>MOSS Transcribe Diarize</b></td>
|
| 150 |
-
<td>14.84</td><td>15.83</td><td>0.99</td>
|
| 151 |
-
<td>24.86</td><td>22.17</td><td>-2.69</td>
|
| 152 |
-
<td>5.97</td><td>7.37</td><td><b>1.40</b></td>
|
| 153 |
-
<td>6.36</td><td>12.76</td><td>6.40</td>
|
| 154 |
</tr>
|
| 155 |
<tr>
|
| 156 |
<td style="white-space: nowrap;"><b>MOSS Transcribe Diarize Pro</b></td>
|
| 157 |
<td><b>13.78</b></td><td><b>14.02</b></td><td><b>0.24</b></td>
|
| 158 |
<td><b>18.22</b></td><td><b>13.94</b></td><td><b>-4.27</b></td>
|
| 159 |
-
<td><b>4.46</b></td><td><b>6.97</b></td><td>2.51</td>
|
| 160 |
<td><b>5.86</b></td><td><b>11.78</b></td><td><b>5.92</b></td>
|
| 161 |
</tr>
|
| 162 |
</tbody>
|
|
@@ -254,7 +251,7 @@ More prompt recipes are available in the GitHub repository: <https://github.com/
|
|
| 254 |
|
| 255 |
### Serve with vLLM and SGLang
|
| 256 |
|
| 257 |
-
MOSS-Transcribe-Diarize supports vLLM serving through the OpenAI-compatible transcription API:
|
| 258 |
|
| 259 |
```bash
|
| 260 |
pip install vllm
|
|
@@ -338,11 +335,11 @@ In this format:
|
|
| 338 |
|
| 339 |
## License
|
| 340 |
|
| 341 |
-
MOSS-Transcribe-Diarize is licensed under the Apache License 2.0.
|
| 342 |
|
| 343 |
## Citation
|
| 344 |
|
| 345 |
-
If you use MOSS-Transcribe-Diarize, please cite the technical report:
|
| 346 |
|
| 347 |
```bibtex
|
| 348 |
@misc{moss_transcribe_diarize_2026,
|
|
|
|
| 25 |
<a href="https://arxiv.org/abs/2601.01554"><img src="https://img.shields.io/badge/arXiv-2601.01554-b31b1b?logo=arxiv"></a>
|
| 26 |
</div>
|
| 27 |
|
| 28 |
+
MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness.
|
| 29 |
|
| 30 |
Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as `[S01]`, `[S02]`, and beyond.
|
| 31 |
|
|
|
|
| 51 |
|
| 52 |
## Introduction
|
| 53 |
|
| 54 |
+
MOSS-Transcribe-Diarize 0.9B turns real-world long-form audio into structured, speaker-aware transcripts in one pass. Instead of stitching together separate ASR and diarization systems, it jointly performs speech transcription and speaker diarization, producing time-aligned text with consistent speaker labels.
|
| 55 |
|
| 56 |
The model is built for meetings, calls, podcasts, interviews, lectures, videos, and other long or messy multi-speaker recordings. It can also emit acoustic event annotations, giving downstream systems a richer view of what happened, who spoke, and when.
|
| 57 |
|
| 58 |
Core capabilities:
|
| 59 |
|
| 60 |
+
* **Long-form transcription**: Converts long audio or video recordings into timestamped text.
|
| 61 |
+
* **Speaker-aware diarization**: Assigns anonymous speaker labels such as `[S01]` and `[S02]` without a separate diarization pipeline.
|
| 62 |
+
* **Promptable generation**: Supports custom transcription instructions, hotwords, and acoustic event annotations.
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
## Model Architecture
|
| 65 |
|
| 66 |
<p align="center">
|
| 67 |
+
<img src="Model_Architecture.png" alt="MOSS-Transcribe-Diarize 0.9B model architecture" width="900">
|
| 68 |
</p>
|
| 69 |
|
| 70 |
| Component | Specification |
|
|
|
|
| 133 |
<td>22.75</td><td>27.43</td><td>4.68</td>
|
| 134 |
<td>26.75</td><td>32.84</td><td>6.09</td>
|
| 135 |
<td>-</td><td>-</td><td>-</td>
|
| 136 |
+
<td>8.62</td><td>14.73</td><td><u>6.11</u></td>
|
| 137 |
</tr>
|
| 138 |
<tr>
|
| 139 |
<td style="white-space: nowrap;">VIBEVOICE ASR</td>
|
|
|
|
| 143 |
<td>14.59</td><td>42.54</td><td>27.94</td>
|
| 144 |
</tr>
|
| 145 |
<tr>
|
| 146 |
+
<td style="white-space: nowrap;"><b>MOSS Transcribe Diarize 0.9B</b></td>
|
| 147 |
+
<td><u>14.84</u></td><td><u>15.83</u></td><td><u>0.99</u></td>
|
| 148 |
+
<td><u>24.86</u></td><td><u>22.17</u></td><td><u>-2.69</u></td>
|
| 149 |
+
<td><u>5.97</u></td><td><u>7.37</u></td><td><b>1.40</b></td>
|
| 150 |
+
<td><u>6.36</u></td><td><u>12.76</u></td><td>6.40</td>
|
| 151 |
</tr>
|
| 152 |
<tr>
|
| 153 |
<td style="white-space: nowrap;"><b>MOSS Transcribe Diarize Pro</b></td>
|
| 154 |
<td><b>13.78</b></td><td><b>14.02</b></td><td><b>0.24</b></td>
|
| 155 |
<td><b>18.22</b></td><td><b>13.94</b></td><td><b>-4.27</b></td>
|
| 156 |
+
<td><b>4.46</b></td><td><b>6.97</b></td><td><u>2.51</u></td>
|
| 157 |
<td><b>5.86</b></td><td><b>11.78</b></td><td><b>5.92</b></td>
|
| 158 |
</tr>
|
| 159 |
</tbody>
|
|
|
|
| 251 |
|
| 252 |
### Serve with vLLM and SGLang
|
| 253 |
|
| 254 |
+
MOSS-Transcribe-Diarize 0.9B supports vLLM serving through the OpenAI-compatible transcription API:
|
| 255 |
|
| 256 |
```bash
|
| 257 |
pip install vllm
|
|
|
|
| 335 |
|
| 336 |
## License
|
| 337 |
|
| 338 |
+
MOSS-Transcribe-Diarize 0.9B is licensed under the Apache License 2.0.
|
| 339 |
|
| 340 |
## Citation
|
| 341 |
|
| 342 |
+
If you use MOSS-Transcribe-Diarize 0.9B, please cite the technical report:
|
| 343 |
|
| 344 |
```bibtex
|
| 345 |
@misc{moss_transcribe_diarize_2026,
|