| --- |
| license: apache-2.0 |
| language: |
| - ln |
| tags: |
| - automatic-speech-recognition |
| - lingala |
| - wav2vec2-bert |
| - ctc |
| - low-resource |
| base_model: keystats/w2v-bert-2.0-lingala-main-best-3 |
| datasets: |
| - google/WaxalNLP |
| - KasuleTrevor/Lingala_100hrs |
| pipeline_tag: automatic-speech-recognition |
| --- |
| |
| # w2v-bert-2.0-lingala |
|
|
| A Lingala automatic speech recognition (ASR) model, further |
| fine-tuned from |
| [keystats/w2v-bert-2.0-lingala-main-best-3](https://huggingface.co/keystats/w2v-bert-2.0-lingala-main-best-3), |
| which was itself continued from |
| [keystats/w2v-bert-2.0-lingala-main-best](https://huggingface.co/keystats/w2v-bert-2.0-lingala-main-best), |
| which was fine-tuned from |
| [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0). |
|
|
| This checkpoint is the latest link in that chain: starting from |
| `main-best-3`'s weights, training continued for **1 additional epoch** |
| at a learning rate of **1e-5**, everything else — data, splits, |
| casing, filtering — kept the same. |
|
|
| ## Model description |
|
|
| `facebook/w2v-bert-2.0` — a large-scale, multilingual self-supervised |
| speech encoder pretrained with a BERT-style masked prediction |
| objective — is used as the backbone, with a character-level CTC |
| (Connectionist Temporal Classification) head. |
|
|
| **Text casing note:** training targets were kept in their raw, cased |
| form (same tokenizer/vocab used throughout this lineage since |
| `main-best`). |
|
|
| ## Training data |
|
|
| Same pool and split usage as `main-best` and `main-best-3` — WAXAL |
| `validation` held out, `test` untouched: |
|
|
| | Source | Role | |
| |---|---| |
| | [google/WaxalNLP](https://huggingface.co/datasets/google/WaxalNLP) (`lin_asr` config) | `train` split pooled into training, `validation` split held out untouched as the fixed evaluation benchmark | |
| | [KasuleTrevor/Lingala_100hrs](https://huggingface.co/datasets/KasuleTrevor/Lingala_100hrs) | All splits pooled into training | |
|
|
| **WAXAL's `validation` split is the only data used for evaluation, |
| and it was never included in training, across this entire lineage.** |
|
|
| ## Training procedure |
|
|
| - **Base checkpoint:** `keystats/w2v-bert-2.0-lingala-main-best-3` |
| (continued fine-tuning) |
| - **Architecture:** `Wav2Vec2BertForCTC`, `add_adapter=True` |
| - **Processor:** same `Wav2Vec2BertProcessor` / tokenizer used since |
| `main-best` (character-level, raw/cased text, `|` word delimiter, |
| `[PAD]` as CTC blank) |
| - **Sample rate:** 16 kHz mono |
| - **Epochs:** 1 additional epoch on top of `main-best-3` |
| - **Learning rate:** 1e-5, cosine schedule, 10% warmup |
| - **Effective batch size:** 32 (per-device batch size 4 × gradient |
| accumulation 8) |
| - **Precision:** fp16, gradient checkpointing enabled |
| - **Regularization:** attention/hidden/feature-projection dropout 0.05 |
| - **Data filtering:** same CTC-feasibility filtering used throughout |
| this lineage |
| - **Seed:** 42 (deterministic — same seed for Python/NumPy/PyTorch/CUDA) |
|
|
| ## Evaluation results |
|
|
| Evaluated on the WAXAL Lingala `validation` split, greedy decoding |
| vs. greedy + KenLM (`keystats/waxal-kenlm-models-best`). Adding the |
| KLM gives a consistent, meaningful WER/CER improvement over greedy |
| decoding alone — pair the two for the best results. |
|
|
| ## How to use |
|
|
| ### Option 1 — model alone (greedy decoding) |
|
|
| ```python |
| import torch |
| import librosa |
| from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor |
| |
| MODEL_ID = "keystats/w2v-bert-2.0-lingala" |
| DEVICE = "cuda" if torch.cuda.is_available() else "cpu" |
| |
| processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID) |
| model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval() |
| |
| audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True) |
| inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt") |
| with torch.no_grad(): |
| logits = model(input_features=inputs.input_features.to(DEVICE)).logits |
| |
| predicted_ids = torch.argmax(logits, dim=-1) |
| transcription = processor.batch_decode(predicted_ids)[0] |
| |
| print(transcription) # cased, punctuated Lingala text |
| ``` |
|
|
| ### Option 2 — model + KLM (recommended, higher accuracy) |
|
|
| ```python |
| # pip install pyctcdecode |
| # pip install https://github.com/kpu/kenlm/archive/master.zip |
| |
| import torch |
| import librosa |
| from huggingface_hub import hf_hub_download |
| from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor |
| from pyctcdecode import build_ctcdecoder |
| |
| MODEL_ID = "keystats/w2v-bert-2.0-lingala" |
| KLM_REPO_ID = "keystats/waxal-kenlm-models-best" |
| DEVICE = "cuda" if torch.cuda.is_available() else "cpu" |
| |
| processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID) |
| model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval() |
| |
| klm_path = hf_hub_download(repo_id=KLM_REPO_ID, repo_type="dataset", |
| filename="lingala/lingala_5gram_correct-best.arpa") |
| |
| def build_vocab_list(tokenizer, vocab_size): |
| vocab_dict = tokenizer.get_vocab() |
| vocab_list = [None] * vocab_size |
| for tok, idx in sorted(vocab_dict.items(), key=lambda kv: kv[1]): |
| if idx < vocab_size: |
| vocab_list[idx] = tok |
| pad_id = tokenizer.pad_token_id |
| if pad_id is not None and pad_id < len(vocab_list): |
| vocab_list[pad_id] = "" |
| word_delim = getattr(tokenizer, "word_delimiter_token", None) |
| if word_delim: |
| delim_id = vocab_dict.get(word_delim) |
| if delim_id is not None: |
| vocab_list[delim_id] = " " |
| return vocab_list |
| |
| vocab_list = build_vocab_list(processor.tokenizer, model.config.vocab_size) |
| decoder = build_ctcdecoder( |
| vocab_list, |
| kenlm_model_path=klm_path, |
| alpha=0.5, |
| beta=0.7, |
| ) |
| |
| audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True) |
| inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt") |
| with torch.no_grad(): |
| logits = model(input_features=inputs.input_features.to(DEVICE)).logits |
| |
| transcription = decoder.decode(logits.cpu().numpy()[0], beam_width=100) |
| print(transcription) |
| ``` |
|
|
| ## Intended uses & limitations |
|
|
| - Intended for transcribing spoken Lingala audio into cased, |
| punctuated text. |
| - As a CTC-based model, it assumes single-speaker, forward-only |
| audio and has no mechanism for overlapping speech from multiple |
| speakers. |
| - This is the fourth checkpoint in a continued fine-tuning lineage |
| (`main-best` → `main-best-3` → this model). If earlier checkpoints |
| had already converged, this additional low-LR epoch may yield only |
| marginal gains — compare validation metrics against `main-best-3` |
| before choosing between them. |
|
|
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{keystats_wav2vec2bert_lingala_final, |
| title={w2v-bert-2.0-lingala: A Lingala ASR model, continued fine-tune of w2v-bert-2.0-lingala-main-best-3}, |
| author={keystats}, |
| year={2026}, |
| howpublished={\url{https://huggingface.co/keystats/w2v-bert-2.0-lingala}} |
| } |
| |
| @misc{waxal, |
| title={WAXAL: A Multilingual African Speech Dataset}, |
| author={Google}, |
| howpublished={\url{https://huggingface.co/datasets/google/WaxalNLP}} |
| } |
| |
| @misc{kasule_lingala_100hrs, |
| title={Lingala\_100hrs}, |
| author={KasuleTrevor}, |
| howpublished={\url{https://huggingface.co/datasets/KasuleTrevor/Lingala_100hrs}} |
| } |
| |
| @inproceedings{w2vbert2, |
| title={Seamless: Multilingual Expressive and Streaming Speech Translation}, |
| author={Seamless Communication and others}, |
| year={2023}, |
| howpublished={\url{https://huggingface.co/facebook/w2v-bert-2.0}} |
| } |
| ``` |
