new

README.md

@@ -18,7 +18,7 @@ tags:
 pipeline_tag: audio-text-to-text
 ---
 
-# MOSS-Transcribe-Diarize 0.9B
+# MOSS-Transcribe-Diarize
 
 <div align="center">
   <a href="https://github.com/OpenMOSS/MOSS-Transcribe-Diarize"><img src="https://img.shields.io/badge/GitHub-OpenMOSS%2FMOSS--Transcribe--Diarize-black?logo=github"></a>
@@ -28,14 +28,12 @@ pipeline_tag: audio-text-to-text
 
 MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness.
 
-It supports transcription and diarization across **50+** languages, single-pass inference on audio recordings up to **90 minutes** long, and custom hotword prompting for domain-specific terms.
+It supports transcription and diarization across 50+ languages.
 
 Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as `[S01]`, `[S02]`, and beyond.
 
 ## News
 
-* 2026-07-22: The subtitle Web UI now supports both Simplified Chinese and English.
-* 2026-07-14: 🏆 MOSS-Transcribe-Diarize won first place in the [2nd MLC-SLM Challenge](https://www.nexdata.ai/competition/mlc-slm) at INTERSPEECH 2026, spanning 14 languages (English, French, German, Italian, Portuguese, Spanish, Japanese, Korean, Russian, Thai, Vietnamese, Tagalog, Urdu, Turkish).
 * 2026-07-09: Released MOSS-Transcribe-Diarize 0.9B.
 
 ## Contents
@@ -46,6 +44,7 @@ Given an audio or video file, the model generates a compact speaker-aware transc
 - [Quickstart](#quickstart)
   - [Environment Setup](#environment-setup)
   - [Python Usage](#python-usage)
+  - [Custom Prompt and Hotwords](#custom-prompt-and-hotwords)
   - [Serve with vLLM and SGLang](#serve-with-vllm-and-sglang)
   - [Subtitle Web App](#subtitle-web-app)
 - [Output Format](#output-format)
@@ -71,6 +70,14 @@ Core capabilities:
   <img src="Model_Architecture.png" alt="MOSS-Transcribe-Diarize 0.9B model architecture" width="900">
 </p>
 
+| Component | Specification |
+|---|---|
+| Text backbone | Qwen3-0.6B style causal decoder |
+| Audio encoder | Whisper-Medium encoder configuration |
+| Audio frontend | `WhisperFeatureExtractor`, 16 kHz, 80 mel bins, 30 s chunks |
+| Audio-text bridge | 4x temporal merge + MLP adaptor |
+| Fusion | Audio features replace <code>&lt;&#124;audio_pad&#124;&gt;</code> embeddings via `masked_scatter` |
+| Output format | Compact `[start][Sxx]text[end]` transcript with speaker tags such as `[S01]` |
 
 This Hugging Face repository includes the custom Transformers remote code required to load the model with `trust_remote_code=True`.
 
@@ -146,7 +153,7 @@ We evaluate MOSS-Transcribe-Diarize using three objective metrics: Character Err
       <td><u>6.36</u></td><td><u>12.76</u></td><td>6.40</td>
     </tr>
     <tr>
-      <td style="white-space: nowrap;"><a href="https://platform.mosi.cn/app/playground"><b>MOSS Transcribe Diarize Pro</b></a></td>
+      <td style="white-space: nowrap;"><b>MOSS Transcribe Diarize Pro</b></td>
       <td><b>13.78</b></td><td><b>14.02</b></td><td><b>0.24</b></td>
       <td><b>18.22</b></td><td><b>13.94</b></td><td><b>-4.27</b></td>
       <td><b>4.46</b></td><td><b>6.97</b></td><td><u>2.51</u></td>
@@ -245,9 +252,37 @@ To add hotwords, append a short hint to the default prompt:
 
 More prompt recipes are available in the GitHub repository: <https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/main/examples/prompts.md>
 
-### Serve with SGLang and VLLM
+### Serve with vLLM and SGLang
 
-The recommended way to serve MOSS-Transcribe-Diarize is [SGLang Omni](https://github.com/sgl-project/sglang-omni) through the OpenAI-compatible `/v1/audio/transcriptions` endpoint. If you are using a CUDA 12 environment, SGLang is currently not supported; use vLLM instead. Install `sglang-omni` by following the [installation guide](https://github.com/sgl-project/sglang-omni/blob/main/docs/get_started/installation.md), then download the model:
+MOSS-Transcribe-Diarize supports vLLM serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use `cu129`; for CUDA 13 environments, use `cu130`.
+
+```bash
+uv pip install -U vllm \
+  --torch-backend=auto \
+  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
+```
+
+or:
+
+```bash
+uv pip install -U vllm \
+  --torch-backend=auto \
+  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu130
+```
+
+```bash
+vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code
+```
+
+```bash
+curl http://localhost:8000/v1/audio/transcriptions \
+  -F model="OpenMOSS-Team/MOSS-Transcribe-Diarize" \
+  -F file=@"audio.wav" \
+  -F response_format="json" \
+  -F temperature="0"
+```
+
+The recommended way to serve MOSS-Transcribe-Diarize is [SGLang Omni](https://github.com/sgl-project/sglang-omni) through the OpenAI-compatible `/v1/audio/transcriptions` endpoint. Install `sglang-omni` by following the [installation guide](https://github.com/sgl-project/sglang-omni/blob/main/docs/get_started/installation.md), then download the model:
 
 ```bash
 hf download OpenMOSS-Team/MOSS-Transcribe-Diarize
@@ -273,6 +308,27 @@ curl -X POST http://localhost:8000/v1/audio/transcriptions \
   -F response_format=verbose_json
 ```
 
+```python
+import requests
+
+with open("audio.wav", "rb") as f:
+    resp = requests.post(
+        "http://localhost:8000/v1/audio/transcriptions",
+        data={
+            "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize",
+            "response_format": "verbose_json",
+        },
+        files={"file": ("audio.wav", f, "audio/wav")},
+        timeout=300,
+    )
+
+resp.raise_for_status()
+payload = resp.json()
+print(payload["text"])
+for segment in payload.get("segments", []):
+    print(f"[{segment['start']:.2f}-{segment['end']:.2f}] {segment['text']}")
+```
+
 For longer multi-speaker audio, raise `max_new_tokens` so the decoder can finish the full diarized transcript:
 
 ```bash
@@ -283,35 +339,37 @@ curl -X POST http://localhost:8000/v1/audio/transcriptions \
   -F max_new_tokens=65536
 ```
 
-MOSS-Transcribe-Diarize also supports **vLLM** serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use `cu129`; for CUDA 13 environments, use `cu130`.
-
-```bash
-uv pip install -U vllm \
-  --torch-backend=auto \
-  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
-```
-
-or:
-
-```bash
-uv pip install -U vllm \
-  --torch-backend=auto \
-  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu130
-```
-
-```bash
-vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code
-```
-
-```bash
-curl http://localhost:8000/v1/audio/transcriptions \
-  -F model="OpenMOSS-Team/MOSS-Transcribe-Diarize" \
-  -F file=@"audio.wav" \
-  -F response_format="json" \
-  -F temperature="0"
-```
-
-
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `file` | file | required | Audio file uploaded as multipart form data |
+| `model` | string | server default | Model identifier |
+| `language` | string | unset | Optional language hint |
+| `response_format` | string | `json` | `json`, `verbose_json`, or `text` |
+| `temperature` | float | model default (`0.0`) | Sampling temperature |
+| `max_new_tokens` | int | `5120` | Max generated tokens; raise for long audio, for example `65536` |
+| `prompt` | string | unset | Optional instruction override; omit to use the built-in transcribe+diarize prompt |
+
+For benchmarking, performance numbers, and implementation details, see the [SGLang Omni cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/moss_transcribe_diarize.md). The following single-H100 results are reported for short- and long-sequence multi-speaker ASR tasks.
+
+`movies` short-sequence ASR:
+
+| Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
+|---:|---:|---:|---:|---:|
+| 1 | 2.57 | 0.388 | 0.0612 | 29.76 |
+| 2 | 4.89 | 0.409 | 0.0659 | 56.55 |
+| 4 | 6.62 | 0.513 | 0.0790 | 76.64 |
+| 8 | 6.80 | 0.533 | 0.0810 | 78.70 |
+| 16 | 7.08 | 0.659 | 0.0922 | 81.98 |
+
+`aishell4_long` long-sequence ASR:
+
+| Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
+|---:|---:|---:|---:|---:|
+| 1 | 0.022 | 45.2 | 0.0197 | 50.64 |
+| 2 | 0.032 | 60.7 | 0.0265 | 74.25 |
+| 4 | 0.036 | 105.6 | 0.0461 | 81.64 |
+| 8 | 0.040 | 172.6 | 0.0754 | 90.62 |
+| 16 | 0.043 | 282.8 | 0.1237 | 98.83 |
 
 ### Subtitle Web App