File size: 3,751 Bytes
50019c3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ---
license: other
license_name: mixed-see-below
tags:
- automatic-speech-recognition
- translation
- whisper.cpp
- ctranslate2
- vrstt
---
# vrSTT models
Model weights for [vrSTT](https://github.com/kittynXR/vrSTT) — local speech-to-text
for VR and streaming. Everything vrSTT runs is here so the app can fetch the one
model a user chose, verify it, and never touch the network again.
**This repository redistributes third-party models. It trains nothing and
modifies nothing except file format.** Licences differ per directory and are
stated below; the upstream terms travel with the weights.
## Layout
```
manifest.json # what the fetcher reads first
whisper/ggml-<name>.bin # whisper.cpp ggml format, one file each
translation/<name>/ # CTranslate2 format, a directory each
```
`manifest.json` carries a `sha256` and a byte count for every file. The fetcher
verifies both before handing a path to whisper.cpp or CTranslate2 — a truncated
ggml file otherwise fails at load with an error about magic bytes, which sends
you looking at the wrong thing.
## What is here
**Speech recognition** — [whisper.cpp](https://github.com/ggerganov/whisper.cpp)
ggml conversions, byte-identical to
[ggerganov/whisper.cpp](https://huggingface.co/ggerganov/whisper.cpp). Mirrored
rather than linked so the bytes vrSTT downloads are pinned and hashed.
`tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`,
`medium.en`, `large-v1`, `large-v2`, `large-v3`, `large-v3-turbo`
vrSTT defaults to **`small.en` for commits and `base.en` for live previews** —
the split is deliberate, previews are allowed to be wrong and are replaced within
a second, so they should not spend the larger model.
**Translation** — [NLLB-200](https://ai.meta.com/research/no-language-left-behind/)
converted to CTranslate2 with `ct2-transformers-converter --quantization
float16`, from `facebook/nllb-200-*`.
`nllb-200-distilled-600M` (default), `nllb-200-distilled-1.3B`, `nllb-200-1.3B`
float16 rather than int8 because CTranslate2 can cast **down** at load time but
not up — one artifact serves both, and converting to int8 here would cap quality
for everyone. 600M is the default because vrSTT's stated minimum-VRAM target is
8 GB, and a 1.3B model beside the ASR stages and a game does not fit
comfortably there.
## Licences — read this before redistributing further
| Directory | Licence | Commercial use |
|---|---|---|
| `whisper/` | **MIT** (OpenAI Whisper; ggml conversions by Georgi Gerganov) | permitted |
| `translation/` | **CC-BY-NC-4.0** (Meta AI, NLLB-200) | **NOT permitted** |
**NLLB-200 is non-commercial.** Redistribution is allowed with attribution and
the licence must travel with it — which is what this section is — but neither
Meta's weights nor anything derived from them may be used commercially. If vrSTT
ever becomes a paid product, the translation models have to be replaced rather
than relicensed.
NLLB-200 also carries Meta's own caveats, which apply unchanged here: it is
research work, not certified for production translation, it was trained on
Wikimedia-domain text and does not target long-form or domain-specific input,
and translation quality varies widely by language pair. Mistranslations can
carry real consequences and vrSTT displays them live on a stream — the
`include_original` setting exists so the source text stays visible beside the
translation for exactly this reason, and it is on by default.
## Citation
Whisper — Radford et al., *Robust Speech Recognition via Large-Scale Weak
Supervision*, OpenAI, 2022.
NLLB-200 — NLLB Team et al., *No Language Left Behind: Scaling Human-Centered
Machine Translation*, Meta AI, 2022.
|