Instructions to use agiws/AGIWSVoice with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use agiws/AGIWSVoice with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-to-speech", model="agiws/AGIWSVoice")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("agiws/AGIWSVoice", device_map="auto") - Notebooks
- Google Colab
- Kaggle
AGIWSVoice
Research and engineering on a TTS / voice-conversion engine built around a single
audio decoder: a latent of shape [T, 256] at 25 fps is mapped to 24 kHz audio.
Motivation
OmniVoice consists of a Qwen3 backbone, a discrete 8-codebook head, and a DAC
decoder. The backbone learns text -> latent, but it is not streamable: it is
bidirectional and runs 32 full MaskGIT passes per chunk (142-199 ms per step,
about 5-6 s per chunk). This project does not use the backbone or the discrete
codebooks. Only the decoder is kept; the latent is produced and edited by our
own tooling.
phonemes + durations + curves (F0 / energy) -> latent [T,256] -> DAC decoder -> 24 kHz
^
inversion of real audio
Decoder weights come from the original PyTorch checkpoint; the GGUF copy is kept only as a cross-check that both hold the same tensors.
What decoder inversion enables
- artifact detection β residual
audio - decode(latent); clicks, clipping, hum, dropouts and compression damage are not explained by the decoder and land in the residual, so no labels are needed; - voice cloning β timbre from the mean latent of a reference utterance, without any speaker encoder;
- speech-to-speech β content and curves from the source, timbre from the target speaker;
- prosody control β sensitivity directions
d(F0, energy) / d(latent)and counterfactual stress edits; - ASR add-ons β emotion from curve coordinates, stress placement, and rescoring of external ASR hypotheses by residual.
Reference blocks
The upstream PyTorch implementation is exposed block by block, so both forward and backward passes can be run through a single block or the whole pipeline:
RVQ dequant -> fc2 (1024 -> 256) -> conv1 -> decoder blocks -> conv2
DAC encoder / HuBERT / quantizer (input side)
Structure
docs/β internal plan and work journal;src/agiws_voice/β library code;scripts/β publishing, job submission, extraction jobs;tests/β tests.
External dependencies
agiws/OmniVoice-0.6Bβ PyTorch reference implementation and codec weights;agiws/OmniVoice-0.6B-Q8-GGUF-llama.cppβ GGUF copy of the codec weights;agiws/NeuralQuantβ quantized modules, used only by later phases;- a Colab T4 compute worker behind the AGIWS Gateway for heavy runs.