--- license: mit pipeline_tag: audio-classification tags: - language-identification - spoken-language-identification - audio-classification - speech - multilingual - wav2vec2 - mms - xls-r - pytorch-lightning datasets: - google/fleurs --- # BA-Lang - Spoken Language Identification checkpoints Checkpoints for **spoken language identification (LID)**: a self-supervised speech backbone, pooled with **MHFA** (Multi-Head Factorized Attention) and scored with either a standard cosine-softmax classifier or **BA-LR** (Binary Attribute Likelihood Ratio), a Bernoulli classifier over a learned binary attribute embedding. - Training code / recipe: [https://github.com/AMIAD-Research/BA-Lang] - BA-LR / BA-Lang model: Y. Jelassi et al., *"Reconnaissance de la langue parlée explicable avec BA-LR"*, JEP 2026 — [isca-archive.org/jep_2026/jelassi26_jep.html](https://www.isca-archive.org/jep_2026/jelassi26_jep.html) ## Architecture ``` raw audio (16 kHz) │ ▼ SSL backbone (MMS-1B or wav2vec2-XLS-R) │ ▼ Multi-Head Factorized Attentive pooling (MHFA) │ ▼ classification head ├─ Linear, or └─ BA-Lang: binary encoder → per-class Bernoulli log-likelihood │ ▼ language logits ``` ## Checkpoints | File | Backbone | Corpus | Languages | Head | Step | Val. accuracy | Size | |---|---|---|---|---|---|---|---| | `MMS-Fleurs-step-010360-val_classif_acc-0.9168118238449097.ckpt` | MMS-1B | FLEURS | 102 | Linear | 10360 | 91.68% | 3.86 GB | | `MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt` | MMS-1B | FLEURS | 102 | BA-Lang (e2e) | 8856 | 91.68% | 3.87 GB | | `MMS-Fleurs-e2e-balang-MIR-rho-0.95-step-032472-val_classif_acc-0.90935212373733...ckpt` | MMS-1B | FLEURS | 102 | BA-Lang (e2e) + Matryoshka-inspired regularization, ρ=0.95 | 32472 | 90.94% | 3.87 GB | | `MMS-Fleurs-e2e-balang-ND-rho-0.857-step-047232-val_classif_acc-0.89306861162185...ckpt` | MMS-1B | FLEURS | 102 | BA-Lang (e2e) + Nested Dropout, ρ=0.857 | 47232 | 89.31% | 3.87 GB | | `MMS-Tidylang-step-008533-val_classif_acc-0.9778640866279602.ckpt` | MMS-1B | Tidylang | 35 | Linear | 8533 | 97.79% | 3.86 GB | | `MMS-Tidylang-e2e-balang-step-005565-val_classif_acc-0.9929152131080627.ckpt` | MMS-1B | Tidylang | 35 | BA-Lang (e2e) | 5565 | 99.29% | 3.87 GB | | `XLSR-Fleurs-step-005904-val_classif_acc-0.8094159960746765.ckpt` | wav2vec2-XLS-R | FLEURS | 102 | Linear | 5904 | 80.94% | 1.27 GB | | `XLSR-Fleurs-e2e-balang-step-047232-val_classif_acc-0.8120864033699036.ckpt` | wav2vec2-XLS-R | FLEURS | 102 | BA-Lang (e2e) | 47232 | 81.21% | 1.27 GB | | `XLSR-Tidylang-step-008904-val_classif_acc-0.9662737846374512.ckpt` | wav2vec2-XLS-R | Tidylang | 35 | Linear | 8904 | 96.63% | 1.27 GB | | `XLSR-Tidylang-e2e-balang-step-032648-val_classif_acc-0.9860262870788574...ckpt` | wav2vec2-XLS-R | Tidylang | 35 | BA-Lang (e2e) | 32648 | 98.60% | 1.27 GB | ## Datasets Training is done separately on two corpora — pick the checkpoint matching the languages you need. - **[FLEURS](https://huggingface.co/datasets/google/fleurs)** (102 languages, ISO 639-3): `afr, amh, ara, asm, ast, aze, bel, ben, bos, bul, cat, ceb, ces, ckb, cmn, cym, dan, deu, ell, eng, est, fas, fil, fin, fra, ful, gle, glg, guj, hau, heb, hin, hrv, hun, hye, ibo, ind, isl, ita, jav, jpn, kam, kan, kat, kaz, kea, khm, kir, kor, lao, lav, lin, lit, ltz, lug, luo, mal, mar, mkd, mlt, mon, mri, msa, mya, nep, nld, nob, nso, nya, oci, ori, orm, pan, pol, por, pus, ron, rus, slk, slv, sna, snd, som, spa, srp, swa, swe, tam, tel, tgk, tha, tur, ukr, umb, urd, uzb, vie, wol, xho, yor, yue, zul`. - **Tidylang** (35 languages): `TidyVoiceX_ASV` (train/dev) and `TidyVoiceX2_ASV` (eval), from Mozilla Data Collective ([train/dev](https://mozilladatacollective.com/datasets/cmihtsewu023so207xot1iqqw), [eval](https://mozilladatacollective.com/datasets/cmkv32i5e02tumg07j79d3c35)): `ara, bak, bel, ben, bul, cat, chv, cym, deu, div, ell, eng, fas, fra, hin, hye, jpn, kat, lit, lug, mal, mar, nld, ori, pol, por, rus, tam, tha, tuk, tur, uig, uzb, yue, zho`. ```python import yaml lang_list = yaml.safe_load(open("configs/data/fleurs.yaml"))["language_list"] # or configs/data/tidylang.yaml for the Tidylang checkpoints ``` ## Quickstart — inference on your own audio ```bash git clone https://github.com/AMIAD-Research/BA-Lang cd BA-Lang python -m venv .venv && source .venv/bin/activate pip install -e ".[dev]" pip install huggingface_hub python -c " from huggingface_hub import hf_hub_download hf_hub_download( repo_id='yjelassi/BA-Lang', filename='MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt', local_dir='.', ) " ``` Then, on a single audio file (no manifest needed — this loads the checkpoint directly and runs one forward pass): ```python import torch import torchaudio import yaml from balr_lid.models.lid_model import LanguageIdentificationModel CHECKPOINT = "MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt" LANGUAGES = yaml.safe_load(open("configs/data/fleurs.yaml"))["language_list"] model = LanguageIdentificationModel.load_from_checkpoint( CHECKPOINT, map_location="cpu", weights_only=False ) model.eval() waveform, sr = torchaudio.load("my_audio.wav") if waveform.shape[0] > 1: waveform = waveform.mean(dim=0, keepdim=True) if sr != 16000: waveform = torchaudio.functional.resample(waveform, sr, 16000) with torch.no_grad(): out = model(waveform) probs = out["logits"].softmax(dim=-1)[0] top5 = probs.topk(5) for score, idx in zip(top5.values, top5.indices): print(f"{LANGUAGES[idx]}: {score.item():.3f}") ``` ### Batch inference over a dataset To evaluate on many files at once (accuracy, confusion matrix, embedding dump), build a manifest CSV (see [docs/data_manifest.md](https://github.com/AMIAD-Research/BA-Lang)) and use the repo's batch script instead: ```bash python -m balr_lid.inference \ --checkpoint MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt \ --manifest data/manifests/fleurs-test.csv \ --audio-root /path/to/fleurs/audio \ --language-list afr amh ara ... \ --output predictions.csv ``` `--language-list` must match the checkpoint's training corpus, in order (see [Datasets](#datasets) above).