---
license: cc-by-4.0
language: [kk, en]
library_name: pytorch
pipeline_tag: automatic-speech-recognition
tags: [target-speaker-asr, speech-recognition, kazakh, english, overlapping-speech, wavlm, ctc]
---

# Persona-ASR

Bilingual (Kazakh–English) **target-speaker ASR** for overlapping speech, from the [Persona-ASR](https://github.com/IS2AI/Persona_ASR) project. Given a three-speaker mixture and a short enrollment utterance, the model transcribes only the enrolled target speaker, and rejects the utterance (emitting `<no_target>`) when the target is absent.

## Checkpoints
- `asr_backbone.pt` — recognizer + activity-detection backbone: a frozen ECAPA-TDNN speaker embedding modulates a WavLM-Base+ encoder via FiLM, feeding two language-specific CTC heads (English/Latin, Kazakh/Cyrillic) and a frame-level VAD head.
- `presence_gate.pt` — utterance-level target-presence gate (enrollment–mixture matching + speaker-conditioned attention + attentive-statistics pooling), applied on top of the frozen backbone.
- `config.json` — backbone configuration (vocab sizes, hyperparameters).

## Results (three-speaker test sets)
| Test set | Raw WER | Gated WER | Detection BAcc |
|---|---|---|---|
| English (Libri3Mix-100h) | 29.35 | 36.92 | 80.04 |
| Kazakh (Kazakh3Mix-100h) | 43.47 | 50.71 | 86.78 |

Presence-gate thresholds (calibrated on validation): τ_EN = 0.502, τ_KK = 0.586.

## Training
Backbone loss L = 0.7·CTC + 0.3·VAD; gate trained with binary cross-entropy on the frozen backbone. Trained on LibriMix (English) and [KazMix-3](https://huggingface.co/datasets/issai/KazMix-3) (Kazakh); also evaluated on [PersonaMix](https://huggingface.co/datasets/issai/PersonaMix).

## Usage
Inference and evaluation code: [github.com/IS2AI/Persona_ASR](https://github.com/IS2AI/Persona_ASR).

## License and citation
Released under CC BY 4.0. Please cite Persona-ASR.

```bibtex
@article{persona_asr,
  title = {Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh--English Overlapping Speech},
  year  = {2026}
}
```