phase2_corrected/COMPLIANCE.md

# Compliance statement — data, models and licences

Google WAXAL ASR Challenge, Phase 2 · user `Pricilegangbe` · public score `0.761204`

This document discloses **every** data source and pre-trained checkpoint used to build the
submitted system, as required by the organisers.

---

## 1. Applicable rule on external datasets

The Rules page states *"You may use only the datasets specified for this challenge"*, while the
Phase 1 section states that participants *"may supplement the provided challenge data with other
publicly available open-source speech or language datasets"*. This inconsistency was raised on
the challenge discussion board and **clarified by the organisers**
([discussion 33616](https://zindi.africa/competitions/google-waxal-asr-challenge/discussions/33616),
answer by `meganomaly` (Zindi), 28 June 2026):

> "The intended rule is that participants **may use additional publicly available open-source
> speech or language datasets** during both Phase 1 and when developing the model they ultimately
> use for Phase 2. Any external datasets used must: be publicly accessible and appropriately
> licensed for research or development; comply with the competition rules; and **be clearly
> disclosed in your final solution documentation**."

The present section 2 is that disclosure.

---

## 2. Data sources used in the submitted system

| Dataset | Provenance | Licence | Used for |
|---|---|---|---|
| **`google/WaxalNLP`** — labelled train split | Official challenge dataset | Challenge dataset (CC-BY 4.0 sharing) | Fine-tuning of all three acoustic models; KenLM text; language-ID statistics; held-out development split |
| **`google/WaxalNLP`** — **unlabelled** Shona split | Official challenge dataset | idem | Self-training: pseudo-labels for the Shona model (labels generated by our own model, no external labels) |
| **`KasuleTrevor/lg_100hrs`** | **External**, public Hugging Face dataset, no gating. ~68.8 k Luganda audio–text pairs. Derived from **Mozilla Common Voice** (audio identifiers are `common_voice_lg_*`). | Underlying Common Voice corpus is released under **CC0-1.0** (public-domain dedication). The Hugging Face mirror carries no separate licence tag. | Additional **Luganda** training material for the multilingual acoustic model `joint_cont` |

### Exactly which model used which data

| Model | Role in the submission | Training material | Size |
|---|---|---|---|
| `joint_cont_best` | **Lingala** decoder | WAXAL lin + WAXAL lug + **`lg_100hrs` (external)** + WAXAL sna, concatenated | 95,467 clips / 291.6 h |
| `sna_ps_best` | **Shona** decoder | WAXAL sna (labelled) + pseudo-labelled WAXAL sna **unlabelled** split | 61,956 clips / 362.6 h |
| `sna_r2_best` | Shona n-best rescorer | WAXAL sna (labelled) only | 13,799 clips / 79.6 h |
| KenLM 5-gram | Lingala shallow fusion | **13,960 WAXAL Lingala training transcripts only** — no external text | — |

Note on `joint_cont`: the external Luganda material is **not** in either of the two evaluated
Phase 2 languages. It was included because multilingual training improves the shared encoder;
Luganda was one of the three focus languages when the model was developed.

### Not used in the final system

Several exploration scripts kept in `code/` for completeness mention datasets that are **not**
part of the submitted system — `AfriVoice` (referenced in `prep_data.py` job list and in the
`train_wbert.py` docstring example) and `BantuLanguagesInitiative/CSRC`. Neither appears in the
training manifest of any model used in the final submission; this can be verified from the
training manifests listed in the table above.

---

## 2-bis. Exact configuration of the submitted system

The submission is `sub_LMA06.csv` (public score `0.761204`, WER 0.3662 / CER 0.1121).
Test clips are routed by language using `artifacts/test_lang.json` (446 Lingala / 446 Shona).

| Language | Acoustic model | Decoding |
|---|---|---|
| **Lingala** | `joint_cont_best` | CTC beam search (`pyctcdecode`), **beam width 64**, shallow fusion with the KenLM 5-gram: **alpha = 0.6, beta = 1.0, lm_score_boundary = True**. The case of the first character is then copied from the greedy hypothesis. |
| **Shona** | `sna_ps_best` | CTC beam search **without any language model**, beam width 256; the 50-best list is re-scored by `sna_r2_best` (CTC forward score, weight 4.0) and the highest-scoring hypothesis is kept. |

`alpha = 0.6` was selected **on the public leaderboard**, not on a held-out split: our offline
development sets proved anti-correlated with the leaderboard on the language-model axis. The
measured curve is 0.5 → 0.759972, **0.6 → 0.761204**, 0.7 → 0.760865, 0.9 → 0.755362,
1.2 → 0.721312.

### Explored and **not** part of the submitted system

For the avoidance of doubt, the following were built and measured but are **absent** from the
submission:

- a **Shona KenLM** (5-gram over WAXAL Shona transcripts) — measured −0.00089, rejected;
- a **Lingala KenLM enriched** with pseudo-labelled text from the WAXAL unlabelled split —
  measured −0.00444, rejected;
- a **second round of Shona self-training** (`sna_ps2`) — measured −0.00018, rejected;
- a **character-level neural rescoring LM** — rejected.

The submitted KenLM therefore uses **only the 13,960 Lingala training transcripts**, as stated in
section 2.

---

## 3. Data-leakage statement

- **No Phase 1 test transcriptions** were used for training, tuning or pseudo-labelling, at any
  point, even though they are publicly available.
- **No Phase 2 test audio or transcriptions** were used for training or adaptation.
- **No pseudo-labels obtained from closed or commercial APIs.** The only pseudo-labels used
  (Shona) were produced by our own model on the challenge's own unlabelled split.
- No publicly released checkpoint that had been fine-tuned on the WAXAL corpus was used, in line
  with the organisers' answer in the same discussion thread.

---

## 4. Pre-trained checkpoint

| Checkpoint | Role | Licence |
|---|---|---|
| [`facebook/w2v-bert-2.0`](https://huggingface.co/facebook/w2v-bert-2.0) | Initialisation of all three acoustic models (speech encoder of SeamlessM4T v2, self-supervised pre-training by Meta AI, openly available to everyone). | **MIT** |

This is the only pre-trained checkpoint in the system. It was **not** trained on WAXAL data.
The three fine-tuned artifacts are derivatives of it.

---

## 5. Third-party software

| Component | Role | Licence |
|---|---|---|
| `transformers`, `datasets`, `accelerate` | model implementation and training loop | Apache-2.0 |
| `torch` | tensor backend | BSD-3-Clause |
| `pyctcdecode` | CTC beam search with shallow fusion | Apache-2.0 |
| `kenlm` | n-gram language model | LGPL-2.1 |
| `soundfile` / `libsndfile` | audio I/O | BSD-3-Clause / LGPL-2.1 |
| `jiwer` | our own WER/CER evaluation | Apache-2.0 |
| `ffmpeg` | 48 kHz → 16 kHz resampling of the test clips | LGPL-2.1 / GPL-2.0 |

All packages are open-source, as required. No component carries a non-commercial restriction.
`facebook/mms-1b-all` (CC-BY-NC-4.0) was evaluated during exploration and is **not** part of the
final system.

## 6. Reproducibility

- The development split is deterministic (`seed 1234`), regenerated by
  `code/build_devhard_linsna.py`.
- The full pipeline is described step by step in `README.md` §3.2.
- Training used one NVIDIA L40S GPU; inference requires substantially less.

## 7. Credentials

No access token, key or credential is present in this package. Scripts that need Hugging Face
access read the token from the `HF_TOKEN` environment variable or the local Hugging Face cache;
nothing is hard-coded.