Automatic Speech Recognition
Transformers
Safetensors
qwen3_asr
confucius4
r2t2
asr
streaming
real-time
low-latency
speech-recognition
vllm
multilingual
Instructions to use netease-youdao/Confucius4-R2T2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use netease-youdao/Confucius4-R2T2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="netease-youdao/Confucius4-R2T2")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("netease-youdao/Confucius4-R2T2") model = AutoModelForMultimodalLM.from_pretrained("netease-youdao/Confucius4-R2T2", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add library name and paper link
#5
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,4 +1,10 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
tags:
|
| 3 |
- confucius4
|
| 4 |
- r2t2
|
|
@@ -9,11 +15,6 @@ tags:
|
|
| 9 |
- speech-recognition
|
| 10 |
- vllm
|
| 11 |
- multilingual
|
| 12 |
-
base_model: Qwen/Qwen3-ASR-1.7B
|
| 13 |
-
pipeline_tag: automatic-speech-recognition
|
| 14 |
-
license: other
|
| 15 |
-
license_name: netease-model-use-license-agreement
|
| 16 |
-
license_link: https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/refs/heads/master/MODEL_LICENSE
|
| 17 |
---
|
| 18 |
|
| 19 |
<div align="center">
|
|
@@ -48,6 +49,8 @@ license_link: https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/r
|
|
| 48 |
</div>
|
| 49 |
<br>
|
| 50 |
|
|
|
|
|
|
|
| 51 |
Confucius4-R2T2 is a low-latency and high-accuracy true streaming Automatic Speech Recognition (ASR) model that features fine-grained and configurable decoding chunks from 80 ms to 2 s. The model operates in append-only output mode: committing transcript text permanently without revising previous words, which is critical for applications where text must be processed or acted upon instantly. This results in a smoother user experience, avoiding disruptive text revisions and visual flickering in real-time applications, such as Real-Time Live Captioning & Subtitling, Downstream NLP Pipelines & LLM Agents, Simultaneous Speech Translation, etc.
|
| 52 |
|
| 53 |
R2T2, short for Real Real-Time Transcription, is built upon the Qwen3-ASR model. And it is trained with a unique set of data construction techniques including stable-prefix data, forced time-alignment data, and token-level audio segmentation. Combined with a Longest Stable Prefix (LSP) learning paradigm (tech report will be released soon), R2T2 can dynamically determine when a stable prefix can be safely emitted and when additional audio context is needed. By exposing only stable prefixes, the model provides high-quality context that conditions subsequent predictions while guaranteeing that previously emitted text remains unchanged. Despite its streaming design, R2T2 maintains strong accuracy in offline recognition.
|
|
@@ -724,4 +727,4 @@ If you use this repository or the R2T2 checkpoint in your research, please cite
|
|
| 724 |
R2T2 uses **dual licensing** to distinguish the source code from the model weights:
|
| 725 |
|
| 726 |
- **Code** in the accompanying GitHub repository is released under the [Apache License 2.0](https://github.com/netease-youdao/Confucius4-R2T2/blob/master/LICENSE) and is free to use, modify, and redistribute (including commercially) under the terms of that license.
|
| 727 |
-
- **Model weights** are released under the [NetEase Model Use License Agreement](https://github.com/netease-youdao/Confucius4-R2T2/blob/master/MODEL_LICENSE).
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model: Qwen/Qwen3-ASR-1.7B
|
| 3 |
+
license: other
|
| 4 |
+
license_name: netease-model-use-license-agreement
|
| 5 |
+
license_link: https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/refs/heads/master/MODEL_LICENSE
|
| 6 |
+
pipeline_tag: automatic-speech-recognition
|
| 7 |
+
library_name: transformers
|
| 8 |
tags:
|
| 9 |
- confucius4
|
| 10 |
- r2t2
|
|
|
|
| 15 |
- speech-recognition
|
| 16 |
- vllm
|
| 17 |
- multilingual
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
---
|
| 19 |
|
| 20 |
<div align="center">
|
|
|
|
| 49 |
</div>
|
| 50 |
<br>
|
| 51 |
|
| 52 |
+
**Paper:** [Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription](https://huggingface.co/papers/2611.06043)
|
| 53 |
+
|
| 54 |
Confucius4-R2T2 is a low-latency and high-accuracy true streaming Automatic Speech Recognition (ASR) model that features fine-grained and configurable decoding chunks from 80 ms to 2 s. The model operates in append-only output mode: committing transcript text permanently without revising previous words, which is critical for applications where text must be processed or acted upon instantly. This results in a smoother user experience, avoiding disruptive text revisions and visual flickering in real-time applications, such as Real-Time Live Captioning & Subtitling, Downstream NLP Pipelines & LLM Agents, Simultaneous Speech Translation, etc.
|
| 55 |
|
| 56 |
R2T2, short for Real Real-Time Transcription, is built upon the Qwen3-ASR model. And it is trained with a unique set of data construction techniques including stable-prefix data, forced time-alignment data, and token-level audio segmentation. Combined with a Longest Stable Prefix (LSP) learning paradigm (tech report will be released soon), R2T2 can dynamically determine when a stable prefix can be safely emitted and when additional audio context is needed. By exposing only stable prefixes, the model provides high-quality context that conditions subsequent predictions while guaranteeing that previously emitted text remains unchanged. Despite its streaming design, R2T2 maintains strong accuracy in offline recognition.
|
|
|
|
| 727 |
R2T2 uses **dual licensing** to distinguish the source code from the model weights:
|
| 728 |
|
| 729 |
- **Code** in the accompanying GitHub repository is released under the [Apache License 2.0](https://github.com/netease-youdao/Confucius4-R2T2/blob/master/LICENSE) and is free to use, modify, and redistribute (including commercially) under the terms of that license.
|
| 730 |
+
- **Model weights** are released under the [NetEase Model Use License Agreement](https://github.com/netease-youdao/Confucius4-R2T2/blob/master/MODEL_LICENSE).
|