timfduffy/zero-GGUF

Zero (MC001) - GGUF

GGUF quantizations of movingcastles/zero.

Zero is a character model: Qwen3-8B-Base fine-tuned to embody a single, limited, anti-servile character. See the original model card and the full training report for what it is and how it was made. All credit for the model goes to Moving Castles. This repo only repackages the weights for llama.cpp and compatible runtimes.

Files

File	Quant	Size	Notes
zero-Q8_0.gguf	Q8_0	8.71 GB	Near-lossless. Use this if it fits.
zero-Q6_K.gguf	Q6_K	6.73 GB	Very close to Q8_0 in quality.
zero-Q4_K_M.gguf	Q4_K_M	5.03 GB	Good default for 8 GB GPUs and CPU-only use.

Usage notes

These come from the original model card and matter more than usual for this model.

• No system prompt. The model was trained only on user and assistant turns in ChatML. The character is in the weights, and a system prompt is out of distribution.
• Two end-of-turn tokens. At non-zero temperature the model ends turns with either <|im_end|> (151645) or <|endoftext|> (151643). The GGUF metadata sets <|im_end|> as EOS, and llama.cpp also treats <|endoftext|> as end-of-generation, so turns stop cleanly there. If you see run-on turns in another runtime, add 151643 as a stop token.
• Sampling. The settings used for all results in the training report: temperature 0.7, top_p 1.0, top_k off, min_p 0.0, repetition penalty 1.0, presence penalty 1.5, max 1024 new tokens.
• Chat template. The training-side ChatML template from the original repo is embedded in each file. Use it as shipped.
• Context. Native context is 32768. The authors serve it at 16384.

Running

llama.cpp:

llama-cli -hf timfduffy/zero-GGUF:Q4_K_M -c 16384 --temp 0.7 --top-p 1.0 --top-k 0 --min-p 0.0 --presence-penalty 1.5 -n 1024

Ollama:

ollama run hf.co/timfduffy/zero-GGUF:Q4_K_M

The files also load in LM Studio, KoboldCpp, and other llama.cpp-based apps. Set the sampling parameters above by hand in those.

How these were made

• Converted from the original bfloat16 safetensors with llama.cpp's convert_hf_to_gguf.py.
• Q8_0 was written directly by the converter. Q6_K and Q4_K_M were produced with llama-quantize (build b11007) from a bf16 GGUF, not requantized from Q8_0.
• Static quants. No importance matrix was used.
• Source revision: 5a09d31.