ragtag-ai commited on
Commit
1fdbac7
·
verified ·
1 Parent(s): 974a22d

Add model card

Browse files
Files changed (1) hide show
  1. README.md +100 -0
README.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - audio
7
+ - speech
8
+ - text-to-speech
9
+ - voice-cloning
10
+ - neucodec
11
+ library_name: safetensors
12
+ base_model: neuphonic/neucodec
13
+ inference: false
14
+ ---
15
+
16
+ # NeuCodec Decoder (safetensors weights)
17
+
18
+ These are the [NeuCodec](https://huggingface.co/neuphonic/neucodec) decoder weights, extracted to `safetensors` for use with a pure-Rust CPU decoder (ndarray + rustfft) — no native ONNX runtime, no PyTorch at inference.
19
+
20
+ The decoder converts the discrete speech codes produced by the NeuTTS backbone back into a 24 kHz audio waveform, completing the zero-shot voice cloning pipeline.
21
+
22
+ This artifact exists to enable voice cloning in [Ragtag](https://ragtag-ai.app), a local-first desktop AI application, under strict architectural constraints: no native ONNX runtime (no `onnxruntime` / `ort`), no GPL dependencies. The companion encoder is [ragtag-ai/neucodec-encoder-rten](https://huggingface.co/ragtag-ai/neucodec-encoder-rten).
23
+
24
+ ## Model details
25
+
26
+ - **Source model:** [neuphonic/neucodec](https://huggingface.co/neuphonic/neucodec) (decoder portion only)
27
+ - **Format:** `safetensors` (extracted from the original `pytorch_model.bin`)
28
+ - **Runtime:** a pure-Rust CPU decoder (ndarray + rustfft) — the implementation in [neutts-rs](https://github.com/eugenehp/neutts-rs), vendored into Ragtag
29
+ - **Precision:** fp32
30
+ - **File size:** approximately 880 MB
31
+ - **Input:** NeuCodec speech codes (50 per second of audio)
32
+ - **Output:** 24 kHz mono audio waveform
33
+
34
+ ## Provenance and extraction process
35
+
36
+ Upstream distributes the decoder only as ONNX ([neuphonic/neucodec-onnx-decoder](https://huggingface.co/neuphonic/neucodec-onnx-decoder)); the pure-Rust decoder loads `safetensors`. These weights were extracted from the original PyTorch model:
37
+
38
+ 1. **PyTorch source:** `pytorch_model.bin` from [neuphonic/neucodec](https://huggingface.co/neuphonic/neucodec)
39
+ 2. **Extraction:** the decoder sub-graph (VocosBackbone: FSQ projection, conv embed, ResNet + RoPE transformer blocks, ISTFT head) is filtered out and written to `safetensors` (script: `convert_weights.py`, or the dependency-free `convert_weights_nopytorch.py`, in the Ragtag repo).
40
+
41
+ No weights are modified — this is a format conversion only.
42
+
43
+ ## Quality verification
44
+
45
+ The pure-Rust decoder was verified against the reference implementations:
46
+
47
+ - **Decoder parity:** 0.99 correlation with the upstream Rust decoder on identical codes
48
+ - **End-to-end:** codes encoded from real audio, decoded through this path, produce an intelligible reconstruction of the speaker (ASR-confirmed)
49
+
50
+ NeuCodec is a generative (vocoder) decoder, so two valid implementations produce intelligible but not phase-identical waveforms; intelligibility, not waveform correlation, is the meaningful check.
51
+
52
+ ## Usage
53
+
54
+ This is not a standalone TTS system. It is the decoder stage of Ragtag's clone pipeline, used together with:
55
+
56
+ - The encoder [ragtag-ai/neucodec-encoder-rten](https://huggingface.co/ragtag-ai/neucodec-encoder-rten)
57
+ - A NeuTTS backbone (e.g. [neuphonic/neutts-nano-q4-gguf](https://huggingface.co/neuphonic/neutts-nano-q4-gguf))
58
+ - A G2P frontend producing IPA phonemes (Ragtag uses piper-plus-g2p)
59
+
60
+ The decoder runs on CPU.
61
+
62
+ ## Licence and attribution
63
+
64
+ Licensed under **Apache License 2.0**, derived from the original NeuCodec which is also Apache 2.0 licensed. Please retain attribution to the original authors:
65
+
66
+ ```
67
+ NeuCodec by Neuphonic
68
+ https://huggingface.co/neuphonic/neucodec
69
+ Licensed under Apache 2.0
70
+ ```
71
+
72
+ The safetensors extraction is contributed by Ragtag / Captivated Ltd, also under Apache 2.0.
73
+
74
+ ## Related resources
75
+
76
+ - Original NeuCodec: [neuphonic/neucodec](https://huggingface.co/neuphonic/neucodec)
77
+ - Companion encoder: [ragtag-ai/neucodec-encoder-rten](https://huggingface.co/ragtag-ai/neucodec-encoder-rten)
78
+ - NeuTTS backbone: [neuphonic/neutts-air](https://huggingface.co/neuphonic/neutts-air)
79
+ - neutts-rs (Rust port reference): [github.com/eugenehp/neutts-rs](https://github.com/eugenehp/neutts-rs)
80
+ - Ragtag: [ragtag-ai.app](https://ragtag-ai.app)
81
+
82
+ ## Citation
83
+
84
+ ```bibtex
85
+ @misc{neucodec2024,
86
+ author = {Neuphonic},
87
+ title = {NeuCodec},
88
+ year = {2024},
89
+ publisher = {Hugging Face},
90
+ url = {https://huggingface.co/neuphonic/neucodec}
91
+ }
92
+
93
+ @misc{neucodec-decoder-safetensors,
94
+ author = {Mallett, Leon},
95
+ title = {NeuCodec Decoder (safetensors weights)},
96
+ year = {2026},
97
+ publisher = {Hugging Face},
98
+ url = {https://huggingface.co/ragtag-ai/neucodec-decoder}
99
+ }
100
+ ```