GGUF + pure-C++ runtime in CrispASR — FastConformer-CTC XL

by cstr - opened May 1

May 1

We've added the FastConformer-CTC XL size to CrispASR's fastconformer-ctc backend. C++ binary, GGUF — no NeMo. (Same runtime supports all three NeMo sizes — large, xlarge, xxlarge.)

Same FastConformer encoder as our parakeet / canary backends (shared core/fastconformer.h — conv subsampling + MHA with rel-PE), just a different decode head: greedy CTC. Lowest-latency English path in CrispASR — no autoregression.

CTC means no native punctuation, so the recipe is to add --punc-model fireredpunc-q8_0.gguf for English+Chinese caps/punc (or the XLM-R-based fullstop-punc-q4_k.gguf for DE/FR/IT). Word-level timing via -am canary-ctc-aligner.gguf.

Pre-quantised GGUFs (CC-BY-4.0): cstr/stt-en-fastconformer-ctc-xlarge-GGUF

./build/bin/crispasr --backend fastconformer-ctc \
    -m stt-en-fastconformer-ctc-xlarge-q4_k.gguf \
    -f audio.wav --punc-model fireredpunc-q8_0.gguf -osrt

Sibling sizes: large, xxlarge.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment