Abstract
Voxtral Realtime is a streaming speech recognition model trained end-to-end for sub-second latency with performance matching offline systems.
We introduce Voxtral Realtime, a natively streaming automatic speech recognition model that matches offline transcription quality at sub-second latency. Unlike approaches that adapt offline models through chunking or sliding windows, Voxtral Realtime is trained end-to-end for streaming, with explicit alignment between audio and text streams. Our architecture builds on the Delayed Streams Modeling framework, introducing a new causal audio encoder and Ada RMS-Norm for improved delay conditioning. We scale pretraining to a large-scale dataset spanning 13 languages. At a delay of 480ms, Voxtral Realtime achieves performance on par with Whisper, the most widely deployed offline transcription system. We release the model weights under the Apache 2.0 license.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Streaming Speech Recognition with Decoder-Only Large Language Models and Latency Optimization (2026)
- MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models (2026)
- Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization (2026)
- Qwen3-TTS Technical Report (2026)
- Stream-Voice-Anon: Enhancing Utility of Real-Time Speaker Anonymization via Neural Audio Codec and Language Models (2026)
- FastSLM: Hierarchical Frame Q-Former for Effective Speech Modality Adaptation (2026)
- dLLM-ASR: A Faster Diffusion LLM-based Framework for Speech Recognition (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper