Phonon 40M models (en, fr, de, es, pt, multilingual) converted for blixt/phonon, CC BY-NC 4.0

README.md

@@ -0,0 +1,67 @@
+---
+license: cc-by-nc-4.0
+language:
+  - en
+  - fr
+  - de
+  - es
+  - pt
+base_model:
+  - gradium/phonon-40m-en
+  - gradium/phonon-40m-fr
+  - gradium/phonon-40m-de
+  - gradium/phonon-40m-es
+  - gradium/phonon-40m-pt
+  - gradium/phonon-40m-multilingual
+pipeline_tag: text-to-speech
+tags:
+  - text-to-speech
+  - webgpu
+  - phonon
+---
+
+# Phonon 40M for WebGPU
+
+This repository holds Gradium's six Phonon 40M models, converted for [blixt/phonon](https://github.com/blixt/phonon), a WebGPU runtime that runs them in the browser. Each model has two files: its converted weights (`MODEL.int8.bin`, 53.7 MB in place of the 122 MB checkpoint) and its tokenizer (`MODEL.tokenizer.5.0.0.json`, Gradium's `tokenizer.5.0.0.json` unchanged).
+
+| Model | Languages | Source revision |
+|---|---|---|
+| [gradium/phonon-40m-en](https://huggingface.co/gradium/phonon-40m-en) | English | `bc7e8ae7e543a7a3e4604753e42fbda76ddf0781` |
+| [gradium/phonon-40m-fr](https://huggingface.co/gradium/phonon-40m-fr) | French | `d91086970206b88efd2c88ca971d0628b5968f0f` |
+| [gradium/phonon-40m-de](https://huggingface.co/gradium/phonon-40m-de) | German | `79cfc7b47fcfc0b666657b981eb366415023a862` |
+| [gradium/phonon-40m-es](https://huggingface.co/gradium/phonon-40m-es) | Spanish | `e5e052b26e31abd5918ade71d3135e69d51a66f8` |
+| [gradium/phonon-40m-pt](https://huggingface.co/gradium/phonon-40m-pt) | Portuguese | `4cbfd742a1555d60eb5399c6d296e9f2e07389e9` |
+| [gradium/phonon-40m-multilingual](https://huggingface.co/gradium/phonon-40m-multilingual) | English, French, German, Spanish, Portuguese | `1537ce46c64f42903e0d608b132ccb073dac46d2` |
+
+The models are by [Gradium](https://gradium.ai), and build on Pocket TTS and the Mimi codec by Kyutai. Gradium did not make this conversion. For an Apache-2.0 model, see [Phonon 31M](https://huggingface.co/gradium/phonon-31m-en) and its conversion, [blixt/phonon-31m-en-webgpu](https://huggingface.co/blixt/phonon-31m-en-webgpu).
+
+## License
+
+The weights and the tokenizers stay under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/), as in Gradium's repositories: you may share and adapt them with credit to Gradium, and not for commercial purposes. The conversion changes the weights as described below.
+
+## What the conversion does
+
+- Tensors with 16,384 values or more become int8, with one f16 scale for each 32 values (scale = max |value| / 127, values rounded to f16 first, quantized values rounded half to even). All other tensors are f16.
+- The rows of q and k move from interleaved rotary pairs to halves in each head.
+- The flow head's time embedding for its one step is added to the bias of `cond_embed`.
+- The codec's latent projection is merged into its upsampling weights, and its layer scales into the weights they scale.
+- Convolution weights are reordered so that each output gathers its inputs.
+
+Each file starts with a JSON header (each tensor's offset, length and scale offset, and the vectors the speed control needs), followed by the vectors and the weights. `packModel` and `unpackModel` in [src/checkpoint.ts](https://github.com/blixt/phonon/blob/main/src/checkpoint.ts) write and read it.
+
+## Rebuild a file
+
+```sh
+git clone https://github.com/blixt/phonon && cd phonon && npm install
+npm run build-weights -- path/to/gradium/phonon-40m-fr/model.safetensors phonon-40m-fr.int8.bin
+shasum -a 256 phonon-40m-fr.int8.bin
+```
+
+SHA-256 of each file:
+
+- `phonon-40m-en.int8.bin`: `9826ea6e49c57bbc3db3bd057acd56e625bcc4721e81e0f0028d03c51273284e`
+- `phonon-40m-fr.int8.bin`: `203eb5198ba19578a3cb2c7eb3bfa9f816da0edd3a370f815fe0a3a4fb2f6121`
+- `phonon-40m-de.int8.bin`: `741e333aeb19761042a9ee7733270933f429cbe487b451dddac7851d8ea6d435`
+- `phonon-40m-es.int8.bin`: `9fcd254567ddd387cc076e00d9c9dfec41d32f6755589f3eea7aecbcb0b4afc4`
+- `phonon-40m-pt.int8.bin`: `0b8d0286136cf2460b7658dceda617fd44b40405f1115b82cc346fb8d7619e9a`
+- `phonon-40m-multilingual.int8.bin`: `b9935b027a8914ba5397940e7c019b80f5608f1454674565894a47e3c8faad5f`

phonon-40m-de.int8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:741e333aeb19761042a9ee7733270933f429cbe487b451dddac7851d8ea6d435
+size 53730600

phonon-40m-en.int8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9826ea6e49c57bbc3db3bd057acd56e625bcc4721e81e0f0028d03c51273284e
+size 53730600

phonon-40m-es.int8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fcd254567ddd387cc076e00d9c9dfec41d32f6755589f3eea7aecbcb0b4afc4
+size 53730600

phonon-40m-fr.int8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:203eb5198ba19578a3cb2c7eb3bfa9f816da0edd3a370f815fe0a3a4fb2f6121
+size 53730600

phonon-40m-multilingual.int8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9935b027a8914ba5397940e7c019b80f5608f1454674565894a47e3c8faad5f
+size 53730600

phonon-40m-pt.int8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b8d0286136cf2460b7658dceda617fd44b40405f1115b82cc346fb8d7619e9a
+size 53730600