Instructions to use xumo/reader-speech-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use xumo/reader-speech-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf xumo/reader-speech-models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf xumo/reader-speech-models:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf xumo/reader-speech-models:Q4_K_M
Use Docker
docker model run hf.co/xumo/reader-speech-models:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use xumo/reader-speech-models with Ollama:
ollama run hf.co/xumo/reader-speech-models:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use xumo/reader-speech-models with Docker Model Runner:
docker model run hf.co/xumo/reader-speech-models:Q4_K_M
- Lemonade
How to use xumo/reader-speech-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull xumo/reader-speech-models:Q4_K_M
Run and chat with the model
lemonade run user.reader-speech-models-Q4_K_M
List all available models
lemonade list
- Atomic Chat
|
Download README.md from xumo/reader-speech-models: direct link, hf CLI and curl.
- Browser
- Download file 4.87 kB
-
https://huggingface.co/xumo/reader-speech-models/resolve/main/README.md
- Command line
-
hf download hf://xumo/reader-speech-models/README.md
-
curl -L -o README.md https://huggingface.co/xumo/reader-speech-models/resolve/main/README.md
4.87 kB
| license: apache-2.0 | |
| base_model: | |
| - Qwen/Qwen3-ASR-0.6B | |
| - Qwen/Qwen3-ASR-1.7B | |
| - Qwen/Qwen3-ForcedAligner-0.6B | |
| - Qwen/Qwen3-TTS-12Hz-0.6B-Base | |
| library_name: gguf | |
| tags: | |
| - reader | |
| - gguf | |
| # Reader accepted local speech artifacts | |
| Reader-converted artifacts, not an official Qwen release. Files retain the bytes | |
| accepted by Reader; `models.lock.json` pins size and SHA-256. Bind downloads to the | |
| full publication commit, never a moving main branch. | |
| ## Included artifacts | |
| - `asr-0.6b-f16` | |
| - `asr-1.7b-f16` | |
| - `asr-1.7b-q4-k-m` | |
| - `asr-1.7b-q8-0` | |
| - `aligner-q8-0` | |
| - `mmproj-asr-0.6b-reader-compat` | |
| - `mmproj-asr-1.7b-reader-compat` | |
| - `mmproj-forcedaligner-reader-compat` | |
| - `qwen3-tts-0.6b-base-f16` | |
| - `synthetic-heart` | |
| - `synthetic-george` | |
| ## Sources and conversion | |
| Qwen Base revision: `5d83992436eae1d760afd27aff78a71d676296fc`. | |
| Converter: ggml-org/llama.cpp `67107ac34f62882d8b53ea41ad68010ea635b9be`. | |
| Reader packaging source commit: `14d2029ba099c97ea4b2544bef0c58de4f11ed79` (may precede its source-repository push). | |
| The complete recipe and reconstruction evidence are included in this model repo. | |
| Pinned public converter: | |
| https://github.com/ggml-org/llama.cpp/blob/67107ac34f62882d8b53ea41ad68010ea635b9be/convert_hf_to_gguf.py | |
| The Qwen TTS GGUF pair includes the talker and projector/codec. The generated | |
| `reader-model.json` preserves the original source-file receipt; raw safetensors | |
| listed in that receipt are source inputs and are not required for inference. | |
| Rebuilding is a separate operation; consult `conversion.lock.json` and `rebuild.json`. | |
| Use the locked Python/package versions, install the converter's pinned gguf-py, | |
| download all raw files from the pinned Base revision, and preserve the source | |
| directory basename recorded by the recipe (it can affect GGUF naming metadata). | |
| Run the two argument sets in the recipe through the pinned converter, providing | |
| the raw model directory as positional input and fresh output paths. Keep the | |
| `speech_tokenizer` subdirectory beside the raw model. Reconstruction is CPU-only; | |
| the accepted record uses OMP/MKL/OpenBLAS thread counts of 4 and offline HF mode. | |
| Synthetic Heart/George WAVs, when included, are reference audio resources, not | |
| model weights or user recordings. Their Kokoro model revision and source voice | |
| are recorded in each artifact origin. Preserve the supplied WAV bytes to retain | |
| the accepted speaker conditioning; generating new reference WAVs creates new assets. | |
| ## Runtime compatibility and limits | |
| Use the Reader runtime matching the locked upstream revision. Reader validates | |
| Qwen synthesis for English/Chinese; other upstream languages are not claimed by | |
| this release. Validated host: Linux x64, GTX 1080 Ti, CUDA SM61. Model files and | |
| platform runtime binaries have separate lifecycles. This repository contains no | |
| runtime binary, personal recordings, account data, or historical ASR/FA files with | |
| unverified provenance. | |
| ## Attribution | |
| Qwen3-TTS: https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base | |
| Kokoro reference source: https://huggingface.co/onnx-community/Kokoro-82M-v1.0-ONNX | |
| Upstream model cards mark Apache-2.0. See LICENSE. Reader changes: conversion to | |
| F16 GGUF and explicit packaging/identity metadata. The publication commit is not | |
| a model retraining or a new quality claim. | |
| ## ASR/FA conversion provenance | |
| The accepted text GGUF and Reader-compatible projectors were reconstructed exactly | |
| from the official revisions and raw-file hashes in `asr-conversion.lock.json`. | |
| `asr-rebuild.json` compares all eight outputs to their accepted SHA values. | |
| To reconstruct: obtain public ggml-org/llama.cpp commit | |
| `66d65ec29ba7c440cbc31b6f63b74a17b536ba65`, apply the included | |
| `legacy-asr-conversion.patch`, and check the four source hashes in | |
| `legacy-asr-converter.lock.json`. This freezes conversion-only sources; it does | |
| not restore an old inference backend. Use the locked Python environment and the | |
| patched source's gguf-py. Convert each official raw model to F16 text GGUF and an | |
| F16 mmproj. The original model folder basename affects metadata and must match. | |
| Build the CPU llama-quantize tool from current upstream commit | |
| `67107ac34f62882d8b53ea41ad68010ea635b9be`. Quantize ASR 1.7B F16 to Q8_0/Q4_K_M | |
| and FA F16 to Q8_0 with four threads. Apply the included standalone Reader | |
| projector converter (`conversion/`) with `--encoder-compat`, using gguf-py from | |
| that current pinned commit. No tensors are retrained; compare final SHA values | |
| against the model lock before use. | |
| The Reader-compatible mmproj files require Reader's encoder/aligner compatibility | |
| adapter. Ordinary upstream projectors are not interchangeable with these accepted | |
| files. See the per-artifact origins for source revisions and compatibility tags. | |
| Converter code is covered separately by LICENSE.llama.cpp; model provenance and | |
| Apache-2.0 attribution remain in LICENSE and the official Qwen model cards. | |