Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- audio
|
| 7 |
+
- speech
|
| 8 |
+
- text-to-speech
|
| 9 |
+
- voice-cloning
|
| 10 |
+
- neucodec
|
| 11 |
+
library_name: safetensors
|
| 12 |
+
base_model: neuphonic/neucodec
|
| 13 |
+
inference: false
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# NeuCodec Decoder (safetensors weights)
|
| 17 |
+
|
| 18 |
+
These are the [NeuCodec](https://huggingface.co/neuphonic/neucodec) decoder weights, extracted to `safetensors` for use with a pure-Rust CPU decoder (ndarray + rustfft) — no native ONNX runtime, no PyTorch at inference.
|
| 19 |
+
|
| 20 |
+
The decoder converts the discrete speech codes produced by the NeuTTS backbone back into a 24 kHz audio waveform, completing the zero-shot voice cloning pipeline.
|
| 21 |
+
|
| 22 |
+
This artifact exists to enable voice cloning in [Ragtag](https://ragtag-ai.app), a local-first desktop AI application, under strict architectural constraints: no native ONNX runtime (no `onnxruntime` / `ort`), no GPL dependencies. The companion encoder is [ragtag-ai/neucodec-encoder-rten](https://huggingface.co/ragtag-ai/neucodec-encoder-rten).
|
| 23 |
+
|
| 24 |
+
## Model details
|
| 25 |
+
|
| 26 |
+
- **Source model:** [neuphonic/neucodec](https://huggingface.co/neuphonic/neucodec) (decoder portion only)
|
| 27 |
+
- **Format:** `safetensors` (extracted from the original `pytorch_model.bin`)
|
| 28 |
+
- **Runtime:** a pure-Rust CPU decoder (ndarray + rustfft) — the implementation in [neutts-rs](https://github.com/eugenehp/neutts-rs), vendored into Ragtag
|
| 29 |
+
- **Precision:** fp32
|
| 30 |
+
- **File size:** approximately 880 MB
|
| 31 |
+
- **Input:** NeuCodec speech codes (50 per second of audio)
|
| 32 |
+
- **Output:** 24 kHz mono audio waveform
|
| 33 |
+
|
| 34 |
+
## Provenance and extraction process
|
| 35 |
+
|
| 36 |
+
Upstream distributes the decoder only as ONNX ([neuphonic/neucodec-onnx-decoder](https://huggingface.co/neuphonic/neucodec-onnx-decoder)); the pure-Rust decoder loads `safetensors`. These weights were extracted from the original PyTorch model:
|
| 37 |
+
|
| 38 |
+
1. **PyTorch source:** `pytorch_model.bin` from [neuphonic/neucodec](https://huggingface.co/neuphonic/neucodec)
|
| 39 |
+
2. **Extraction:** the decoder sub-graph (VocosBackbone: FSQ projection, conv embed, ResNet + RoPE transformer blocks, ISTFT head) is filtered out and written to `safetensors` (script: `convert_weights.py`, or the dependency-free `convert_weights_nopytorch.py`, in the Ragtag repo).
|
| 40 |
+
|
| 41 |
+
No weights are modified — this is a format conversion only.
|
| 42 |
+
|
| 43 |
+
## Quality verification
|
| 44 |
+
|
| 45 |
+
The pure-Rust decoder was verified against the reference implementations:
|
| 46 |
+
|
| 47 |
+
- **Decoder parity:** 0.99 correlation with the upstream Rust decoder on identical codes
|
| 48 |
+
- **End-to-end:** codes encoded from real audio, decoded through this path, produce an intelligible reconstruction of the speaker (ASR-confirmed)
|
| 49 |
+
|
| 50 |
+
NeuCodec is a generative (vocoder) decoder, so two valid implementations produce intelligible but not phase-identical waveforms; intelligibility, not waveform correlation, is the meaningful check.
|
| 51 |
+
|
| 52 |
+
## Usage
|
| 53 |
+
|
| 54 |
+
This is not a standalone TTS system. It is the decoder stage of Ragtag's clone pipeline, used together with:
|
| 55 |
+
|
| 56 |
+
- The encoder [ragtag-ai/neucodec-encoder-rten](https://huggingface.co/ragtag-ai/neucodec-encoder-rten)
|
| 57 |
+
- A NeuTTS backbone (e.g. [neuphonic/neutts-nano-q4-gguf](https://huggingface.co/neuphonic/neutts-nano-q4-gguf))
|
| 58 |
+
- A G2P frontend producing IPA phonemes (Ragtag uses piper-plus-g2p)
|
| 59 |
+
|
| 60 |
+
The decoder runs on CPU.
|
| 61 |
+
|
| 62 |
+
## Licence and attribution
|
| 63 |
+
|
| 64 |
+
Licensed under **Apache License 2.0**, derived from the original NeuCodec which is also Apache 2.0 licensed. Please retain attribution to the original authors:
|
| 65 |
+
|
| 66 |
+
```
|
| 67 |
+
NeuCodec by Neuphonic
|
| 68 |
+
https://huggingface.co/neuphonic/neucodec
|
| 69 |
+
Licensed under Apache 2.0
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
The safetensors extraction is contributed by Ragtag / Captivated Ltd, also under Apache 2.0.
|
| 73 |
+
|
| 74 |
+
## Related resources
|
| 75 |
+
|
| 76 |
+
- Original NeuCodec: [neuphonic/neucodec](https://huggingface.co/neuphonic/neucodec)
|
| 77 |
+
- Companion encoder: [ragtag-ai/neucodec-encoder-rten](https://huggingface.co/ragtag-ai/neucodec-encoder-rten)
|
| 78 |
+
- NeuTTS backbone: [neuphonic/neutts-air](https://huggingface.co/neuphonic/neutts-air)
|
| 79 |
+
- neutts-rs (Rust port reference): [github.com/eugenehp/neutts-rs](https://github.com/eugenehp/neutts-rs)
|
| 80 |
+
- Ragtag: [ragtag-ai.app](https://ragtag-ai.app)
|
| 81 |
+
|
| 82 |
+
## Citation
|
| 83 |
+
|
| 84 |
+
```bibtex
|
| 85 |
+
@misc{neucodec2024,
|
| 86 |
+
author = {Neuphonic},
|
| 87 |
+
title = {NeuCodec},
|
| 88 |
+
year = {2024},
|
| 89 |
+
publisher = {Hugging Face},
|
| 90 |
+
url = {https://huggingface.co/neuphonic/neucodec}
|
| 91 |
+
}
|
| 92 |
+
|
| 93 |
+
@misc{neucodec-decoder-safetensors,
|
| 94 |
+
author = {Mallett, Leon},
|
| 95 |
+
title = {NeuCodec Decoder (safetensors weights)},
|
| 96 |
+
year = {2026},
|
| 97 |
+
publisher = {Hugging Face},
|
| 98 |
+
url = {https://huggingface.co/ragtag-ai/neucodec-decoder}
|
| 99 |
+
}
|
| 100 |
+
```
|