| --- |
| license: cc-by-nc-4.0 |
| language: |
| - en |
| - fr |
| - de |
| - es |
| - pt |
| base_model: |
| - gradium/phonon-40m-en |
| - gradium/phonon-40m-fr |
| - gradium/phonon-40m-de |
| - gradium/phonon-40m-es |
| - gradium/phonon-40m-pt |
| - gradium/phonon-40m-multilingual |
| pipeline_tag: text-to-speech |
| tags: |
| - text-to-speech |
| - webgpu |
| - phonon |
| --- |
| |
| # Phonon 40M for WebGPU |
|
|
| This repository holds Gradium's six Phonon 40M models, converted for [blixt/phonon](https://github.com/blixt/phonon), a WebGPU runtime that runs them in the browser. Each model has two files: its converted weights (`MODEL.int8.bin`, 53.7 MB in place of the 122 MB checkpoint) and its tokenizer (`MODEL.tokenizer.5.0.0.json`, Gradium's `tokenizer.5.0.0.json` unchanged). |
|
|
| | Model | Languages | Source revision | |
| |---|---|---| |
| | [gradium/phonon-40m-en](https://huggingface.co/gradium/phonon-40m-en) | English | `bc7e8ae7e543a7a3e4604753e42fbda76ddf0781` | |
| | [gradium/phonon-40m-fr](https://huggingface.co/gradium/phonon-40m-fr) | French | `d91086970206b88efd2c88ca971d0628b5968f0f` | |
| | [gradium/phonon-40m-de](https://huggingface.co/gradium/phonon-40m-de) | German | `79cfc7b47fcfc0b666657b981eb366415023a862` | |
| | [gradium/phonon-40m-es](https://huggingface.co/gradium/phonon-40m-es) | Spanish | `e5e052b26e31abd5918ade71d3135e69d51a66f8` | |
| | [gradium/phonon-40m-pt](https://huggingface.co/gradium/phonon-40m-pt) | Portuguese | `4cbfd742a1555d60eb5399c6d296e9f2e07389e9` | |
| | [gradium/phonon-40m-multilingual](https://huggingface.co/gradium/phonon-40m-multilingual) | English, French, German, Spanish, Portuguese | `1537ce46c64f42903e0d608b132ccb073dac46d2` | |
|
|
| The models are by [Gradium](https://gradium.ai), and build on Pocket TTS and the Mimi codec by Kyutai. Gradium did not make this conversion. For an Apache-2.0 model, see [Phonon 31M](https://huggingface.co/gradium/phonon-31m-en) and its conversion, [blixt/phonon-31m-en-webgpu](https://huggingface.co/blixt/phonon-31m-en-webgpu). |
|
|
| ## License |
|
|
| The weights and the tokenizers stay under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/), as in Gradium's repositories: you may share and adapt them with credit to Gradium, and not for commercial purposes. The conversion changes the weights as described below. |
|
|
| ## What the conversion does |
|
|
| - Tensors with 16,384 values or more become int8, with one f16 scale for each 32 values (scale = max |value| / 127, values rounded to f16 first, quantized values rounded half to even). All other tensors are f16. |
| - The rows of q and k move from interleaved rotary pairs to halves in each head. |
| - The flow head's time embedding for its one step is added to the bias of `cond_embed`. |
| - The codec's latent projection is merged into its upsampling weights, and its layer scales into the weights they scale. |
| - Convolution weights are reordered so that each output gathers its inputs. |
|
|
| Each file starts with a JSON header (each tensor's offset, length and scale offset, and the vectors the speed control needs), followed by the vectors and the weights. `packModel` and `unpackModel` in [src/checkpoint.ts](https://github.com/blixt/phonon/blob/main/src/checkpoint.ts) write and read it. |
|
|
| ## Rebuild a file |
|
|
| ```sh |
| git clone https://github.com/blixt/phonon && cd phonon && npm install |
| npm run build-weights -- path/to/gradium/phonon-40m-fr/model.safetensors phonon-40m-fr.int8.bin |
| shasum -a 256 phonon-40m-fr.int8.bin |
| ``` |
|
|
| SHA-256 of each file: |
|
|
| - `phonon-40m-en.int8.bin`: `9826ea6e49c57bbc3db3bd057acd56e625bcc4721e81e0f0028d03c51273284e` |
| - `phonon-40m-fr.int8.bin`: `203eb5198ba19578a3cb2c7eb3bfa9f816da0edd3a370f815fe0a3a4fb2f6121` |
| - `phonon-40m-de.int8.bin`: `741e333aeb19761042a9ee7733270933f429cbe487b451dddac7851d8ea6d435` |
| - `phonon-40m-es.int8.bin`: `9fcd254567ddd387cc076e00d9c9dfec41d32f6755589f3eea7aecbcb0b4afc4` |
| - `phonon-40m-pt.int8.bin`: `0b8d0286136cf2460b7658dceda617fd44b40405f1115b82cc346fb8d7619e9a` |
| - `phonon-40m-multilingual.int8.bin`: `b9935b027a8914ba5397940e7c019b80f5608f1454674565894a47e3c8faad5f` |
|
|
