README.md

---
language:
- ne
license: other
pipeline_tag: automatic-speech-recognition
tags:
- onnx
- ctc
- nemo
- nepali
- quantization
---

# Nepali Automatic Speech Recognition (ONNX CTC)

This repository contains a Nepali ASR model converted to ONNX and tuned for noisy conditions.
The base architecture is from:
https://huggingface.co/ai4bharat/indicconformer_stt_ne_hybrid_ctc_rnnt_large

Model summary:
- Parameters: ~129M
- Decoder: CTC (greedy decoding)
- Variants:
  - `model_ctc.onnx` (FP model, 420.22 MB)
  - `model_ctc_quantized.onnx` (INT8 quantized, 133.94 MB)

## Files

- `model_ctc.onnx`: full-size ONNX model
- `model_ctc_quantized.onnx`: quantized ONNX model
- `model_config.yaml`: NeMo config (preprocessor + vocabulary)
- `local_onnx_asr_inference.ipynb`: notebook for local testing
- `instruction.txt`: post-upload guide for Hugging Face Spaces


## Load and Use the Model

Install dependencies:

```bash
pip install onnxruntime soundfile scipy numpy pyyaml omegaconf torch "nemo_toolkit[asr]"
```

Example inference script:

```python
import numpy as np
import onnxruntime as ort
import soundfile as sf
import torch
import yaml
from omegaconf import OmegaConf
from scipy.signal import resample_poly
from nemo.collections.asr.modules import AudioToMelSpectrogramPreprocessor

ONNX_PATH = "model_ctc_quantized.onnx"  # or "model_ctc.onnx"
CONFIG_PATH = "model_config.yaml"
AUDIO_PATH = "sample.wav"

# Load config
try:
    conf = OmegaConf.load(CONFIG_PATH)
except Exception:
    with open(CONFIG_PATH, "r", encoding="utf-8") as f:
        conf = OmegaConf.create(yaml.safe_load(f))

preprocessor_cfg = OmegaConf.to_container(conf.preprocessor, resolve=True)
preprocessor_cfg.pop("_target_", None)
preprocessor = AudioToMelSpectrogramPreprocessor(**preprocessor_cfg)
preprocessor.eval()
SAMPLE_RATE = preprocessor_cfg["sample_rate"]

vocabulary = (
    conf.get("aux_ctc", {}).get("decoder", {}).get("vocabulary", None)
    or conf.get("decoder", {}).get("vocabulary", None)
)

session = ort.InferenceSession(ONNX_PATH, providers=["CPUExecutionProvider"])
session_ins = session.get_inputs()
main_input = next((x for x in session_ins if "length" not in x.name.lower()), session_ins[0])
length_input = next((x for x in session_ins if "length" in x.name.lower()), None)

def _length_dtype(meta):
    return np.int32 if meta and "int32" in meta.type else np.int64

def decode_ctc(logits, encoded_len, vocab):
    greedy = logits[0].argmax(axis=-1)[: int(encoded_len[0])]
    blank_id = logits.shape[-1] - 1
    collapsed, prev = [], None
    for t in greedy:
        t = int(t)
        if t == prev or t == blank_id:
            prev = t
            continue
        collapsed.append(t)
        prev = t

    if not vocab:
        return str(collapsed)

    text = ""
    for i in collapsed:
        if 0 <= i < len(vocab):
            tok = vocab[i]
            if tok.startswith("##"):
                text += tok[2:]
            elif tok.startswith("▁"):
                text += " " + tok[1:]
            else:
                text += tok
    return text.strip().replace("▁", " ")

def transcribe(audio_path: str) -> str:
    audio, sr = sf.read(audio_path)
    if audio.ndim == 2:
        audio = audio.mean(axis=1)
    if sr != SAMPLE_RATE:
        audio = resample_poly(audio, SAMPLE_RATE, sr)

    audio = np.clip(audio, -1.0, 1.0).astype(np.float32)
    audio_len = np.array([audio.shape[0]], dtype=np.int64)

    ort_inputs = {}
    if len(main_input.shape) == 2:
        ort_inputs[main_input.name] = audio[None, :]
        if length_input is not None:
            ort_inputs[length_input.name] = audio_len.astype(_length_dtype(length_input))
    elif len(main_input.shape) == 3:
        with torch.no_grad():
            mel, mel_len = preprocessor(
                input_signal=torch.from_numpy(audio[None, :]),
                length=torch.from_numpy(audio_len),
            )
        ort_inputs[main_input.name] = mel.numpy().astype(np.float32)
        if length_input is not None:
            ort_inputs[length_input.name] = mel_len.numpy().astype(_length_dtype(length_input))

    outputs = session.run(None, ort_inputs)
    logits = next((x for x in outputs if getattr(x, "ndim", 0) == 3), None)
    encoded_len = next((x for x in outputs if getattr(x, "ndim", 0) == 1), None)
    if encoded_len is None:
        encoded_len = np.array([logits.shape[1]], dtype=np.int64)

    return decode_ctc(logits, encoded_len, vocabulary)

print(transcribe(AUDIO_PATH))
```

## Use Directly from Hugging Face Hub
 
```python
from huggingface_hub import hf_hub_download
 
repo_id = "gam30/nepali-automatic-speech-recognition"
onnx_path = hf_hub_download(repo_id=repo_id, filename="model_ctc_quantized.onnx")
config_path = hf_hub_download(repo_id=repo_id, filename="model_config.yaml")
```
 
Then run inference with the same code above, replacing `ONNX_PATH` and `CONFIG_PATH`.
 
---
 
## Notes
 
- The notebook `local_onnx_asr_inference.ipynb` is the reference test workflow.
- For better quality, use clean 16 kHz mono audio where possible.
- Quantized model is faster/smaller; full model provides better accuracy with small margin only.
 
---
 
## Citation
 
If you use this model in your research, project, or application, please cite it as follows:
<br>
&nbsp;
 
**APA:**
<br>
```
gam30. (2025). Nepali Automatic Speech Recognition (ONNX CTC) [Model]. Hugging Face. https://huggingface.co/gam30/nepali-automatic-speech-recognition
```
<br>
&nbsp;
<br>
**BibTeX:**
 
<pre>
@misc{gam30_nepali_asr,
  author       = {sangam},
  title        = {Nepali Automatic Speech Recognition (ONNX CTC)},
  year         = {2025},
  publisher    = {Hugging Face},
  howpublished = {https://huggingface.co/gam30/nepali-automatic-speech-recognition}
}
</pre>
 
&nbsp;
 
> **Please note:** This model is based on the architecture from [ai4bharat/indicconformer_stt_ne_hybrid_ctc_rnnt_large](https://huggingface.co/ai4bharat/indicconformer_stt_ne_hybrid_ctc_rnnt_large). When citing, please also acknowledge the original base model authors.
 
---
 
## License
 
If you use or redistribute this model, you must credit **[gam30/nepali-automatic-speech-recognition](https://huggingface.co/gam30/nepali-automatic-speech-recognition)** as the source as well as AI4Bharat.