Instructions to use xumo/reader-speech-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use xumo/reader-speech-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf xumo/reader-speech-models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf xumo/reader-speech-models:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf xumo/reader-speech-models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf xumo/reader-speech-models:Q4_K_M
Use Docker
docker model run hf.co/xumo/reader-speech-models:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use xumo/reader-speech-models with Ollama:
ollama run hf.co/xumo/reader-speech-models:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use xumo/reader-speech-models with Docker Model Runner:
docker model run hf.co/xumo/reader-speech-models:Q4_K_M
- Lemonade
How to use xumo/reader-speech-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull xumo/reader-speech-models:Q4_K_M
Run and chat with the model
lemonade run user.reader-speech-models-Q4_K_M
List all available models
lemonade list
- Atomic Chat
File size: 3,630 Bytes
74e09ef | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | #!/usr/bin/env python3
"""Repackage the legacy ForcedAligner projector for pinned upstream mtmd.
Requires numpy and the pinned upstream gguf-py on PYTHONPATH. Renames tensors
and compatible shape metadata; never requantizes or changes tensor data bytes.
"""
import argparse
import hashlib
import json
from pathlib import Path
import gguf
def sha256(path):
with path.open("rb") as source:
return hashlib.file_digest(source, "sha256").hexdigest()
def convert(source, output, encoder_compat=False):
if output.exists() or output.with_suffix(".conversion.json").exists():
raise FileExistsError("choose a fresh output path")
reader = gguf.GGUFReader(source)
if "clip.audio.classify_num" not in reader.fields or not any(t.name == "a.classifier.weight" for t in reader.tensors):
raise ValueError("not a legacy ForcedAligner projector")
writer = gguf.GGUFWriter(output, "clip")
if encoder_compat:
writer.add_bool("clip.audio.reader_aligner_compat", True)
for key, field in reader.fields.items():
if key.startswith("GGUF.") or key == "general.architecture":
continue
new_key = "clip.audio.projector_type" if key == "clip.projector_type" else key
writer.add_key_value(new_key, field.contents(), field.types[0],
field.types[1] if len(field.types) > 1 else None)
tensor_hashes, tensor_types, tensor_shapes = {}, {}, {}
for tensor in reader.tensors:
name = tensor.name.replace("a.conv1d.", "a.conv2d.").replace("a.pre_encode.out.", "a.conv_out.")
data = tensor.data
if name.startswith("a.conv2d.") and name.endswith(".bias"):
data = data.reshape(-1, 1, 1)
writer.add_tensor(name, data, raw_dtype=tensor.tensor_type)
tensor_hashes[name] = hashlib.sha256(tensor.data.tobytes()).hexdigest()
tensor_types[name] = tensor.tensor_type
# GGUF dimensions are stored in the reverse order of NumPy dimensions.
tensor_shapes[name] = list(data.shape[::-1]) if name.startswith("a.conv2d.") and name.endswith(".bias") else tensor.shape.tolist()
writer.write_header_to_file()
writer.write_kv_data_to_file()
writer.write_tensors_to_file()
writer.close()
actual = gguf.GGUFReader(output)
if len(actual.tensors) != len(tensor_hashes) or any(
t.name not in tensor_hashes or
hashlib.sha256(t.data.tobytes()).hexdigest() != tensor_hashes[t.name] or
t.tensor_type != tensor_types[t.name] or t.shape.tolist() != tensor_shapes[t.name]
for t in actual.tensors):
raise RuntimeError("tensor byte/type/shape identity check failed")
output.with_suffix(".conversion.json").write_text(json.dumps({
"inputSha256": sha256(source), "outputSha256": sha256(output),
"upstreamCommit": "67107ac34f62882d8b53ea41ad68010ea635b9be",
"unchangedTensorBytes": len(tensor_hashes), "tensorSha256": tensor_hashes,
"readerAlignerEncoderCompatibility": encoder_compat,
}, indent=2) + "\n")
print(f"PASS: {len(tensor_hashes)} tensors retain identical data bytes")
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--input", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--encoder-compat", action="store_true",
help="Opt into Reader legacy alignment arithmetic (encoder and CUDA decoder)")
args = parser.parse_args()
convert(args.input, args.output, args.encoder_compat)
if __name__ == "__main__":
main()
|