omniASR-LLM-300M — vLLM-Compatible Fork

HuggingFace-compatible fork of facebook/omniASR-LLM-300M, converted for vLLM integration.
Model weights are identical to the original. Only the packaging has changed.

What Changed

Change	Reason
Added config.json (OmniASRConfig format)	Original has no HuggingFace config; vLLM needs one
Added tokenizer_config.json	HuggingFace tokenizer auto-detection
Added lang_lookup.json	Language ID lookup table for multi-language support
Converted checkpoint to model.safetensors	vLLM requires safetensors format
Renamed omniASR_tokenizer.model → tokenizer.model	HuggingFace tokenizer convention
Removed omniASR-LLM-300M.pt	Redundant with safetensors; saves ~6.5 GB

Usage with vLLM

Start the server:

vllm serve ziywang50/omniASR-LLM-300M \
    --enforce-eager \
    --dtype bfloat16 \
    --attention-backend TRITON_ATTN

Note: --attention-backend TRITON_ATTN is required because OmniASR's decoder uses head_dim=512, which exceeds FlashAttention's supported range.

Transcribe audio (OpenAI-compatible API):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")

with open("audio.wav", "rb") as f:
    result = client.audio.transcriptions.create(
        model="ziywang50/omniASR-LLM-300M",
        file=f,
        language="en",  # optional: ISO-639-1 or OmniASR native code (e.g., "eng_Latn")
    )
print(result.text)

Other Variants

Variant	Fork	Status
300M	ziywang50/omniASR-LLM-300M	✅ Tested
1B	ziywang50/omniASR-LLM-1B	✅ Tested
3B	ziywang50/omniASR-LLM-3B	✅ Tested
7B	ziywang50/omniASR-LLM-7B	✅ Tested

All forks are part of vLLM PR #36864 adding OmniASR support to vLLM.

---

Original Model Card

Everything below is from the original facebook/omniASR-LLM-300M model card.

Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages

🤗 Hugging Face | 🐙 GitHub | 🤖️ Demo | 📃 Paper | 📝 Blogpost | 📄 License: Apache 2.0

Model Card for omniASR-LLM-300M

Model Description

This model is part of the Omnilingual ASR family released by Meta AI. The original suite includes:

Model Name	Features	Parameters	Download Size (FP32)	Inference VRAM¹	Real-Time Factor¹ (relative speed)²
omniASR_W2V_300M	SSL	317_390_592	1.2 GiB
omniASR_W2V_1B	SSL	965_514_752	3.6 GiB
omniASR_W2V_3B	SSL	3_064_124_672	12.0 GiB
omniASR_W2V_7B	SSL	6_488_487_168	25.0 GiB
omniASR_CTC_300M	ASR	325_494_996	1.3 GiB	~2 GiB	0.001 (96x)
omniASR_CTC_1B	ASR	975_065_300	3.7 GiB	~3 GiB	0.002 (48x)
omniASR_CTC_3B	ASR	3_080_423_636	12.0 GiB	~8 GiB	0.003 (32x)
omniASR_CTC_7B	ASR	6_504_786_132	25.0 GiB	~15 GiB	0.006 (16x)
omniASR_LLM_300M	ASR with optional language conditioning	1_627_603_584	6.1 GiB	~5 GiB	0.090 (~1x)
omniASR_LLM_1B	ASR with optional language conditioning	2_275_710_592	8.5 GiB	~6 GiB	0.091 (~1x)
omniASR_LLM_3B	ASR with optional language conditioning	4_376_679_040	17.0 GiB	~10 GiB	0.093 (~1x)
omniASR_LLM_7B	ASR with optional language conditioning	7_801_041_536	30.0 GiB	~17 GiB	0.092 (~1x)
omniASR_LLM_7B_ZS	Zero-Shot ASR	7_810_900_608	30.0 GiB	~20 GiB	0.194 (~0.5x)

¹ (batch=1, audio_len=30s, BF16, A100)

² Relative speed to omniASR_LLM_7B

---

Installation

The models were developed using fairseq2, a research-focused sequence modeling toolkit. While we provide a reference inference pipeline that works across platforms, audio support requires libsndfile (Mac: brew install libsndfile; Windows may need an additional setup).

# using pip
pip install omnilingual-asr

# using uv
uv add omnilingual-asr

Inference

from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline

pipeline = ASRInferencePipeline(model_card="omniASR_LLM_7B")

audio_files = ["/path/to/eng_audio1.flac", "/path/to/deu_audio2.wav"]
lang = ["eng_Latn", "deu_Latn"]
transcriptions = pipeline.transcribe(audio_files, lang=lang, batch_size=2)

Supported Languages

To view the full list of 1600+ supported languages, you can access the language list programmatically:

from omnilingual_asr.models.wav2vec2_llama.lang_ids import supported_langs

# Print all supported languages
print(f"Total supported languages: {len(supported_langs)}")
print(supported_langs)

# Check if a specific language is supported
if "eng_Latn" in supported_langs:
    print("English (Latin script) is supported!")

Languages follow the format {language_code}_{script}, for example eng_Latn - English (Latin script), cmn_Hans - Mandarin Chinese (Simplified), ...

---

Training

To further finetune the released checkpoints on your own data, use our data preparation guide followed by the finetuning recipe guide.

---

Citation

BibTeX:

@misc{omnilingualasr2025,
  title={{Omnilingual ASR}: Open-Source Multilingual Speech Recognition for 1600+ Languages},
  author={{Omnilingual ASR Team} and Keren, Gil and Kozhevnikov, Artyom and Meng, Yen and Ropers, Christophe and Setzler, Matthew and Wang, Skyler and Adebara, Ife and Auli, Michael and Can, Balioglu and Chan, Kevin and Cheng, Chierh and Chuang, Joe and Droof, Caley and Duppenthaler, Mark and Duquenne, Paul-Ambroise and Erben, Alexander and Gao, Cynthia and Mejia Gonzalez, Gabriel and Lyu, Kehan and Miglani, Sagar and Pratap, Vineel and Sadagopan, Kaushik Ram and Saleem, Safiyyah and Turkatenko, Arina and Ventayol-Boada, Albert and Yong, Zheng-Xin and Chung, Yu-An and Maillard, Jean and Moritz, Rashel and Mourachko, Alexandre and Williamson, Mary and Yates, Shireen},
  year={2025},
  url={https://ai.meta.com/research/publications/omnilingual-asr-open-source-multilingual-speech-recognition-for-1600-languages/},
}

• Developed by: Meta AI / Omnilingual ASR Team(GitHub)
• Model type: End-to-end automatic speech recognition model (wav2vec2-style encoder with CTC head / encoder-decoder, depending on checkpoint).
• Language(s) (NLP): 1,600+ languages overall in Omnilingual ASR; this corpus release specifically covers 348 under-served languages across many writing systems (Latin, Arabic, Devanagari, etc.).(GitHub)
• License: Apache-2.0 (for the model and code), CC-BY-4.0 for the facebook/omnilingual-asr-corpus dataset.(GitHub)

---