--- license: apache-2.0 base_model: Lucasgitguy-co/Fleet-1 language: [en, de, fr, es, it, nl, pl, ko] pipeline_tag: automatic-speech-recognition datasets: [facebook/voxpopuli, Bingsu/zeroth-korean, google/fleurs] tags: [whistle, cactus, lora, speech-to-text, distillation, pseudo-labels] --- # Fleet-2 A 30 MB on-device speech-to-text model (Cactus Whistle architecture) for English, German, French, Spanish, Italian, Dutch, Polish and Korean, trained on transcripts written by [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo). **Collection:** [Speech recognition](https://huggingface.co/collections/Lucasgitguy-co/speech-recognition-6aef2dcbe0ea208136b0ddc9) **Clean text:** [Fleet-Clean-0.36B](https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B) chains Fleet-2 with a word tagger that removes fillers, repeats and false starts and fixes punctuation ([GGUF](https://huggingface.co/Lucasgitguy-co/Fleet-Clean-0.36B-GGUF)). ## What changed from Fleet-1 - **Teacher:** Whisper-large-v3-turbo transcribed new audio, and Fleet-2 learned from those transcripts. This is sequence-level distillation, as in Distil-Whisper. Whisper and Whistle use different tokenizers, so no logits were matched. - **New audio:** VoxPopuli (European Parliament speech) for the seven European languages and Zeroth-Korean for Korean. Fleet-1 used FLEURS only. - **Start point:** chosen by validation score between Sky-Kim's Korean Whistle and Fleet-1. This run started from `Lucasgitguy-co/Fleet-1`. ## Evaluation Each language has 100 short FLEURS test clips. These are the same clips, filtered the same way, as Fleet-1's table. All models ran in the same native Cactus engine with automatic language detection. Lower is better. Whistle has no Korean. | Language | Metric | Whistle | Sky-Kim | Fleet-1 | Fleet-2 | |---|---|---:|---:|---:|---:| | en | WER | 13.86% | 14.87% | 14.39% | **14.50%** | | de | WER | 22.45% | 22.71% | 22.18% | **22.02%** | | fr | WER | 22.15% | 25.73% | 25.18% | **24.93%** | | es | WER | 11.83% | 11.83% | 11.93% | **11.93%** | | it | WER | 17.44% | 16.60% | 16.32% | **16.66%** | | nl | WER | 32.95% | 29.39% | 29.53% | **30.06%** | | pl | WER | 45.38% | 43.68% | 43.13% | **41.62%** | | ko | CER | n/a | 33.68% | 52.35% | **42.80%** | Fleet-2 beats Fleet-1 in 4 of 8 languages (de, fr, pl, ko). Korean CER is no worse than Sky-Kim: no. **Improvement over Fleet-1 was NOT demonstrated. Check each language; some may regress.**
Out-of-domain clips (VoxPopuli / Zeroth-Korean test splits) | Language | Source | Clips | Whistle | Sky-Kim | Fleet-1 | Fleet-2 | |---|---|---:|---:|---:|---:|---:| | en | VoxPopuli | 50 | 10.56% | 12.69% | 10.91% | **11.98%** | | de | VoxPopuli | 50 | 17.32% | 19.66% | 19.53% | **18.36%** | | fr | VoxPopuli | 50 | 13.06% | 12.24% | 12.04% | **13.57%** | | es | VoxPopuli | 50 | 9.99% | 11.89% | 11.70% | **11.04%** | | it | VoxPopuli | 50 | 34.45% | 40.21% | 38.22% | **33.19%** | | nl | VoxPopuli | 50 | 23.06% | 23.29% | 23.17% | **22.95%** | | pl | VoxPopuli | 50 | 22.02% | 23.53% | 22.98% | **23.53%** | | ko | Zeroth-Korean | 50 | n/a | 25.19% | 15.65% | **13.24%** | Fleet-2 trained on other clips from these corpora, so this table shows in-corpus generalisation, not a fully independent test.
These results come from short clips of 12 s or less. They are not full benchmark scores and not significance tests. Accuracy on long, noisy or conversational audio was not measured. Upstream models may have seen FLEURS training data before. ## Training - **Data:** 4326 clips kept out of 6400 that the teacher labeled: 3014 new VoxPopuli/Zeroth clips and 1312 of Fleet-1's FLEURS training clips. All clips are 12 s or less. - **Teacher decoding:** language forced, temperature 0, no conditioning on previous text. Runtime: the MLX conversion `mlx-community/whisper-large-v3-turbo`. - **Filter:** a clip was kept only if all of these held: - compression ratio ≤ 2.4 - average log-probability ≥ -1.0 - no-speech probability ≤ 0.6 - teacher text within 10% WER of the human transcript (CER for Korean) The training target is the teacher's text. - **Excluded from training:** validation and test clips, and any transcript that matches an evaluation transcript. - **Optimisation:** the start adapter (LoRA plus Korean vocabulary rows) continued with AdamW at 2e-05, warmup/cosine schedule, batch 1, 2 epochs (8652 steps), clip 1.0, W4/A8 quantisation-aware training on CPU (Apple M4). - **Checkpoint selection:** step 2163, chosen by macro validation error on Fleet-1's FLEURS validation clips (0.2212, against 0.2283 for the start model). Test clips were not used for selection. - **Release:** published whatever the scores. The comparison above is reported as measured, including any regressions. `training_manifest.json` records every clip, teacher transcript and pinned revision. `evaluation.json` has every score. `fleet2.py`, `teacher.py` and `run.sh` reproduce the run. ## Usage ```python import cactus from huggingface_hub import hf_hub_download path = hf_hub_download("Lucasgitguy-co/Fleet-2", "whistle.cact") print(cactus.transcribe("clip.wav", weights=path)["text"]) ``` Use 16 kHz mono WAV and `cactus-compute` at revision `ee2ffc67fe0f9537d4928e158440c7d228a63100`. This is a Cactus model, not a Transformers checkpoint. ## License and attribution - **Model:** Apache License 2.0. The original Whistle LICENSE is kept verbatim. Architecture and pretrained weights: Cactus Compute's Whistle (Mroz et al., 2026). Korean vocabulary and adapter lineage: [Sky-Kim/whistle-finetuned-ko](https://huggingface.co/Sky-Kim/whistle-finetuned-ko) and [Lucasgitguy-co/Fleet-1](https://huggingface.co/Lucasgitguy-co/Fleet-1). See `UPSTREAM_MODEL_CARD.md`. - **Teacher:** [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo), MIT License. Only its transcripts were used; none of its weights are included. - **Data:** - [VoxPopuli](https://huggingface.co/datasets/facebook/voxpopuli): CC0 - [Zeroth-Korean](https://www.openslr.org/40/): CC BY 4.0 - [FLEURS](https://huggingface.co/datasets/google/fleurs) (Conneau et al., 2022): CC BY 4.0 No audio is included in this repository. See `NOTICE`.