Orpheus-3B-ONNX-WebGPU

Built with Llama.

Canopy Labs' Orpheus 3B 0.1 ft text-to-speech model, with the decoder of the SNAC 24 kHz audio codec, as ONNX graphs with quantized weights for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference. A Llama 3.2 3B finetune writes SNAC codes (7 tokens per 85 ms frame) for one of eight voices; the decoder turns them into 24 kHz audio. English, with tags such as <laugh>, <chuckle>, <sigh>, <cough>, <sniffle>, <groan>, <yawn> and <gasp> in the text.

Changes from the original

Converted in Quick Inference (from the identical ungated copy unsloth/orpheus-3b-0.1-ft):

• The language model: ONNX Runtime GenAI's model builder, -p int4 -e webgpu, algo_config=k_quant_mixed (4-bit k-quant, some layers in 8 bits), block size 32; the output layer 4-bit (round to nearest); float16 embeddings, activations and KV cache.
• The SNAC decoder: from onnx-community/snac_24khz-ONNX's float decoder_model.onnx, convolution weights quantized to 4 bits without calibration (round to nearest, symmetric, blocks of 32).

These are modified files: their outputs are close to, but not identical to, the original model's. On eight test sentences (voice tara, the sampling below), Whisper small.en heard them with an 11.4% word error rate (ONNX Runtime CPU; 9.1% in Chrome) against 6.8% for the original model; it samples, so single runs vary.

Files

• onnx/model.onnx, onnx/model.onnx.data, onnx/genai_config.json: the language model. input_ids int64 [1, S], attention_mask int64 [1, P + S], past_key_values.{0..27}.{key,value} float16 [1, 8, P, 128] → logits float16 [1, S, 156940], present.*.
• onnx/snac_decoder_q4.onnx: audio_codes.0/1/2 int64 [1, n], [1, 2n], [1, 4n] → audio_values float32 [1, 1, 2048 n].
• tokenizer.json, tokenizer_config.json: the original's.

Generation (canopyai's engine)

1. ids = [128259, 128000] + tokens("{voice}: {text}") + [128009, 128260, 128261, 128257]. Voices: tara, leah, jess, leo, dan, mia, zac, zoe.
2. Sample (repetition penalty 1.3 over the prompt and the tokens so far, temperature 0.6, top-p 0.8) until 128258 or 1200 new tokens.
3. Audio tokens are 128266 + (i % 7) × 4096 + code; every 7 form a frame [L1, L2a, L3a, L3b, L2b, L3c, L3d].
4. Decode all frames with SNAC and drop the first 2048 samples.

Voices

The eight voices are built into Canopy Labs' finetune and come with it under its licence; Canopy Labs does not say who recorded them. Canopy Labs asks that the model not be used for impersonation without consent, misinformation or deception (including fake news or fraudulent calls), or anything illegal or harmful.

Licence

• Orpheus 3B 0.1 ft: Apache 2.0 (LICENSE). It is a finetune of Llama 3.2 3B Instruct, so the Llama 3.2 Community License applies too (LICENSE-Llama-3.2.txt), with Meta's Acceptable Use Policy. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
• SNAC: MIT (LICENSE-SNAC).