Hugging Face's logo

blixt
/
phonon-40m-webgpu

Text-to-Speech
webgpu
phonon
Model card Files
xet
 Community
phonon-40m-webgpu / README.md
blixt's picture
blixt
Phonon 40M models (en, fr, de, es, pt, multilingual) converted for blixt/phonon, CC BY-NC 4.0
dde41aa verified
preview code
|
Raw History Blame Contribute Delete
3.98 kB
metadata 
license: cc-by-nc-4.0
language:
  - en
  - fr
  - de
  - es
  - pt
base_model:
  - gradium/phonon-40m-en
  - gradium/phonon-40m-fr
  - gradium/phonon-40m-de
  - gradium/phonon-40m-es
  - gradium/phonon-40m-pt
  - gradium/phonon-40m-multilingual
pipeline_tag: text-to-speech
tags:
  - text-to-speech
  - webgpu
  - phonon

Phonon 40M for WebGPU

This repository holds Gradium's six Phonon 40M models, converted for blixt/phonon, a WebGPU runtime that runs them in the browser. Each model has two files: its converted weights (MODEL.int8.bin, 53.7 MB in place of the 122 MB checkpoint) and its tokenizer (MODEL.tokenizer.5.0.0.json, Gradium's tokenizer.5.0.0.json unchanged).

Model Languages Source revision
gradium/phonon-40m-en English bc7e8ae7e543a7a3e4604753e42fbda76ddf0781
gradium/phonon-40m-fr French d91086970206b88efd2c88ca971d0628b5968f0f
gradium/phonon-40m-de German 79cfc7b47fcfc0b666657b981eb366415023a862
gradium/phonon-40m-es Spanish e5e052b26e31abd5918ade71d3135e69d51a66f8
gradium/phonon-40m-pt Portuguese 4cbfd742a1555d60eb5399c6d296e9f2e07389e9
gradium/phonon-40m-multilingual English, French, German, Spanish, Portuguese 1537ce46c64f42903e0d608b132ccb073dac46d2

The models are by Gradium, and build on Pocket TTS and the Mimi codec by Kyutai. Gradium did not make this conversion. For an Apache-2.0 model, see Phonon 31M and its conversion, blixt/phonon-31m-en-webgpu.

License

The weights and the tokenizers stay under CC BY-NC 4.0, as in Gradium's repositories: you may share and adapt them with credit to Gradium, and not for commercial purposes. The conversion changes the weights as described below.

What the conversion does

  • Tensors with 16,384 values or more become int8, with one f16 scale for each 32 values (scale = max |value| / 127, values rounded to f16 first, quantized values rounded half to even). All other tensors are f16.
  • The rows of q and k move from interleaved rotary pairs to halves in each head.
  • The flow head's time embedding for its one step is added to the bias of cond_embed.
  • The codec's latent projection is merged into its upsampling weights, and its layer scales into the weights they scale.
  • Convolution weights are reordered so that each output gathers its inputs.

Each file starts with a JSON header (each tensor's offset, length and scale offset, and the vectors the speed control needs), followed by the vectors and the weights. packModel and unpackModel in src/checkpoint.ts write and read it.

Rebuild a file 

git clone https://github.com/blixt/phonon && cd phonon && npm install
npm run build-weights -- path/to/gradium/phonon-40m-fr/model.safetensors phonon-40m-fr.int8.bin
shasum -a 256 phonon-40m-fr.int8.bin

SHA-256 of each file:

  • phonon-40m-en.int8.bin: 9826ea6e49c57bbc3db3bd057acd56e625bcc4721e81e0f0028d03c51273284e
  • phonon-40m-fr.int8.bin: 203eb5198ba19578a3cb2c7eb3bfa9f816da0edd3a370f815fe0a3a4fb2f6121
  • phonon-40m-de.int8.bin: 741e333aeb19761042a9ee7733270933f429cbe487b451dddac7851d8ea6d435
  • phonon-40m-es.int8.bin: 9fcd254567ddd387cc076e00d9c9dfec41d32f6755589f3eea7aecbcb0b4afc4
  • phonon-40m-pt.int8.bin: 0b8d0286136cf2460b7658dceda617fd44b40405f1115b82cc346fb8d7619e9a
  • phonon-40m-multilingual.int8.bin: b9935b027a8914ba5397940e7c019b80f5608f1454674565894a47e3c8faad5f