--- license: apache-2.0 library_name: pytorch tags: - forced-alignment - word-alignment - speech - audio - gguf --- # FuseAlign multilingual (noisy student, step 60,000) Word-level forced aligner: given a recording and its transcript, it predicts each word's start and end on a 1 ms grid plus a presence probability. A reproduction of FuseAlign ([Vaidya et al., 2026](https://arxiv.org/abs/2609.33650)) by Zyphra: one multilingual noisy student, trained 60,000 steps from the multilingual model's CJK fine-tune on about 25,000 h of audio in 73 languages, mostly labelled on clean audio by a teacher per language and learnt from augmented audio. Code, usage, long-form alignment and the C++/ggml CLI: [github.com/Zyphra/fusealign-multilingual](https://github.com/Zyphra/fusealign-multilingual). ## Files | File | Size | Use | | --- | ---: | --- | | `fusealign-multilingual-ns4-step60000.safetensors` | 1.34 GB | PyTorch package (`torch.hub`, `fusealign_hub`): FuseAlign fp32, Dia text encoder bf16 | | `fusealign-multilingual-ns4-step60000-f32.gguf` | 1.35 GB | ggml port: FuseAlign F32, Dia BF16, Silero VAD F32 | | `fusealign-multilingual-ns4-step60000-q8_0.gguf` | 0.51 GB | ggml port: transformer matmuls Q8_0, other Dia tensors BF16, the rest F32 | ## Usage ```python import soundfile as sf import torch aligner = torch.hub.load("Zyphra/fusealign-multilingual", "fusealign", trust_repo=True, device="cuda") wav, sr = sf.read("speech.wav", dtype="float32") words = aligner.align(torch.from_numpy(wav), sr, "the transcript of the recording") # [WordAlignment(word='the', start=..., end=..., presence=..., placed=True), ...] ``` ## Evaluation % of word boundaries off by more than 50 ms (the paper's asymmetric error), against [Qwen3-ForcedAligner-0.6B](https://huggingface.co/Qwen/Qwen3-ForcedAligner-0.6B) (qwen-asr 0.0.6, bf16, whole clips) on the same clips, transcripts and references (FuseAlign: the PyTorch package, CUDA, one clip per forward): | Corpus | FuseAlign | Qwen3-FA | | --- | ---: | ---: | | TIMIT | 7.92 | 20.17 | | Buckeye | 7.23 | 24.60 | | AMI-silver | 34.22 | 58.22 | | TIMIT-long | 8.21 | 20.04 | | BZNSYP Mandarin (gold) | 15.44 | 28.04 | | Seoul Corpus Korean (gold) | 22.43 | 33.58 | | FLEURS Mandarin | 17.97 | 11.06 | | FLEURS Japanese | 13.34 | 8.38 | | FLEURS Korean | 7.80 | 11.53 | | FLEURS Spanish | 15.43 | 11.71 | | FLEURS Portuguese | 9.20 | 10.24 | | FLEURS French | 13.34 | 12.35 | | FLEURS German | 13.99 | 13.91 | | FLEURS Russian | 10.60 | 11.18 | FLEURS references are Montreal Forced Aligner output, not hand labels; BZNSYP's and the Seoul Corpus's are hand-labelled (Seoul: interviewee turns of 40 speakers' spontaneous interviews, at most 30 s). ## License Apache-2.0. The checkpoint and GGUFs include the Dia text encoder weights from [nari-labs/Dia-1.6B-0626](https://huggingface.co/nari-labs/Dia-1.6B-0626) (Apache-2.0); the GGUFs include [Silero VAD](https://github.com/snakers4/silero-vad) weights (MIT). ## Citation ```bibtex @article{vaidya2026fusealign, title = {FuseAlign: Forced Alignment in the Wild}, author = {Vaidya, Mithilesh and Bailey, Stephen and Badam, Sumukh and Bendel, Matthew and He, Xingzhe}, journal = {arXiv preprint arXiv:2609.33650}, year = {2026} } ```