Lapillus-v2

Lapillus-v2 is an Armenian speech recognition model created by David Ghazaryan, tuned for call-center and telephone audio while keeping Lapillus-v1's accuracy on clean speech.

It is a FastConformer hybrid RNN-T + CTC model (115M parameters), continued from davitlive5/Lapillus-v1. It outputs text with punctuation and capitalisation. The repository also contains an optional 6-gram Armenian language model for beam-search decoding.

Official ArmBench results

Independent evaluation by Metric AI lab on ArmBench-ASR (%, lower is better). "norm" = after text normalisation (case, punctuation, numbers); raw scores also count punctuation, capitalisation and number formatting.

Dataset WER WER (norm) CER CER (norm)
Common Voice 26 7.02 4.34 1.42 0.99
FLEURS 12.83 5.43 2.97 1.82
Infocom (May 2026) 19.01 5.45 4.05 1.81
Poems 30.36 9.20 7.98 3.46
Movies 58.18 33.96 27.85 20.21
Combined 18.21 8.31 4.95 2.94

Most of the gap between raw and normalised scores comes from numbers (the model writes them as words, e.g. "ึ„ีฝีกีถีฐีซีถีฃ" instead of "25") and punctuation. Film dialogue (music, effects, overlapping speech) remains the hardest domain.

Our evaluation

Normalised WER (%), lower is better. "Phone" sets are the official test audio passed through a fixed call-center simulation (background noise / babble / music from held-out files, room reverb, 300โ€“3400 Hz band, 8 kHz, G.711 ฮผ-law / A-law, packet loss).

Test set Lapillus-v1 (greedy) Lapillus-v2 (beam 4 + LM) Relative
ArmBench Common Voice 26 test (6892 utts) 5.19 4.48 โˆ’14%
ArmBench FLEURS test (932 utts) 6.70 5.73 โˆ’14%
Phone Common Voice 26 test (2000 utts) 15.21 12.50 โˆ’18%
Phone FLEURS test (932 utts) 12.42 9.86 โˆ’21%

With the same beam + LM decoder, Lapillus-v1 scores 4.51 / 5.91 / 13.13 / 10.59, so the new weights are better on all four sets and the decoder adds most of the clean-speech gain.

Usage

# pip install "nemo_toolkit[asr]"
import nemo.collections.asr as nemo_asr

model = nemo_asr.models.ASRModel.from_pretrained("davitlive5/Lapillus-v2")
print(model.transcribe(["audio.wav"])[0].text)  # 16 kHz mono (8 kHz phone audio: resample to 16 kHz)

Best accuracy: beam search + language model

from huggingface_hub import hf_hub_download
from omegaconf import open_dict

lm = hf_hub_download("davitlive5/Lapillus-v2", "lm/hy_tok6_lm.nemo")
cfg = model.cfg.decoding
with open_dict(cfg):
    cfg.strategy = "malsd_batch"
    cfg.beam.beam_size = 4
    cfg.beam.return_best_hypothesis = True
    cfg.beam.ngram_lm_model = lm
    cfg.beam.ngram_lm_alpha = 0.3
model.change_decoding_strategy(cfg, decoder_type="rnnt")
print(model.transcribe(["audio.wav"])[0].text)

On a GPU this runs at about the same speed as greedy decoding (~0.13 s for a 7 s utterance on an RTX 5070). Split audio longer than about 30 seconds into shorter segments before transcribing.

Training

  1. Fine-tuning from Lapillus-v1 on about 87 h of public Eastern Armenian speech (OpenSLR SLR160, FLEURS train, Armenian audiobooks, VoxPopuli) with on-the-fly call-center augmentation: speed 0.9โ€“1.1, room impulse responses, MUSAN noise and music, 3โ€“6 speaker babble, telephone band-pass, 8 kHz resampling, G.711 codecs, packet loss and clipping. Loss 0.7 RNN-T + 0.3 CTC, AdamW lr 3e-5 with cosine schedule, EMA 0.999, 12k steps, bf16. Best EMA checkpoint at 10k steps.
  2. Weight averaging: the fine-tuned weights are averaged 50/50 with Lapillus-v1, which keeps v1's clean-speech accuracy and most of the telephone gain.
  3. Language model (lm/hy_tok6_lm.nemo): token-level 6-gram KenLM on 8M Armenian sentences from FineWeb-2 plus training transcripts.

Every training sentence and LM sentence that matches an ArmBench test sentence, or shares any 7-word sequence with one, was removed. No Common Voice 26 test clips were used.

Limitations

  • Eastern Armenian only. Expect more errors on Western Armenian.
  • Tested on simulated telephone audio; real call recordings may differ.
  • Accuracy drops on film dialogue, heavy overlapping speech, rare names and specialised terms. Numbers are written out as words.

License

CC-BY-4.0, derived from davitlive5/Lapillus-v1 and NVIDIA's stt_hy_fastconformer_hybrid_large_pc (CC-BY-4.0).

Citation

@misc{ghazaryan2026lapillusv2,
  title        = {Lapillus-v2: Armenian Speech Recognition for Call Centers},
  author       = {Ghazaryan, David},
  year         = {2026},
  url          = {https://huggingface.co/davitlive5/Lapillus-v2}
}
Downloads last month
31
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for davitlive5/Lapillus-v2

Finetuned
(1)
this model

Evaluation results

  • WER (normalised, official ArmBench) on ArmBench-ASR Common Voice 26 test
    self-reported
    4.340
  • CER (normalised, official ArmBench) on ArmBench-ASR Common Voice 26 test
    self-reported
    0.990
  • WER (normalised, official ArmBench) on ArmBench-ASR FLEURS test
    self-reported
    5.430
  • CER (normalised, official ArmBench) on ArmBench-ASR FLEURS test
    self-reported
    1.820
  • WER (normalised, official ArmBench) on ArmBench-ASR combined (MCV26, FLEURS, Infocom, Poems, Movies)
    self-reported
    8.310
  • CER (normalised, official ArmBench) on ArmBench-ASR combined (MCV26, FLEURS, Infocom, Poems, Movies)
    self-reported
    2.940