Voltron-1 Mini GGUF

GGUF quantizations of Akahsizrr/voltron-1-mini, a LoRA post-train of LiquidAI/LFM3-1.7B by @RoliumGens.

The model shipped as BF16 safetensors on 13 November 2026 with no GGUF anywhere, so these files were converted and quantized at Guideboard Labs to run it on consumer GPUs. Same architecture, tokenizer and chat template as the base model. Nothing about the weights was changed beyond quantization.

Files

File Quant Size BPW Notes
voltron-1-mini-Q4_K_M.gguf Q4_K_M 1.10 GB 5.05 Benchmarked here. Tensor map identical to Liquid's own LFM3-1.7B Q4_K_M
voltron-1-mini-Q8_0.gguf Q8_0 1.85 GB 8.51 Near-lossless, for comparison runs

The Q4_K_M is a clean drop-in for Liquid's official quant. Both files carry 287 tensors and the same type map (136 q4_K, 18 q6_K, 133 f32), and the two files differ by 352 bytes. That means a quality or speed comparison against LiquidAI/LFM3-1.7B-GGUF at Q4_K_M measures the fine-tune, not the quantization recipe.

How these were built

Conversion and quantization ran with llama.cpp at 0.7.0-dev, build 50, commit 338fce651, the tree that first carried conversion/lfm3.py locally. That converter is upstream now: LFM3 support landed in PR #31252, merged 2026-10-27 as 19cce69.

# 1. weights
hf download Akahsizrr/voltron-1-mini --local-dir voltron-1-mini --exclude "eval/*"

# 2. convert (BF16 safetensors -> F16 GGUF)
python3 convert_hf_to_gguf.py voltron-1-mini --outtype f16 --outfile voltron-1-mini-F16.gguf

# 3. quantize
llama-quantize voltron-1-mini-F16.gguf voltron-1-mini-Q4_K_M.gguf Q4_K_M
llama-quantize voltron-1-mini-F16.gguf voltron-1-mini-Q8_0.gguf   Q8_0

Verified to load and generate on a plain upstream build (0.9.0-dev, build 1, commit 3ba1ce0), no patch needed.

Run it

llama-server -m voltron-1-mini-Q4_K_M.gguf \
  -ngl 999 -fa on -c 32768 --jinja \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

Give it a wide window. Thinking is on by default in the chat template and the trace lands in reasoning_content, separated from content by llama.cpp. On a trivial code prompt this model spent 4,860 characters of reasoning and 1,174 completion tokens before answering, so a small -c or max_tokens gets spent inside the trace and returns an empty reply. 16K works, 32K is comfortable, and the KV cost is small: 1.59 GiB of VRAM in total at 32K on a 1.7B model.

Sampling above is the model's own generation_config.json (temperature 0.1, top_k 50, repetition penalty 1.1). The model card's own evals used temperature 0.6 and top_p 0.95, so try both.

Measured performance

One RX 5700 XT (RDNA1, 8 GB), Ryzen 5 1600, 32 GB RAM, llama.cpp Vulkan, Q4_K_M:

decode 135.3 t/s (509k tokens over 66 requests, median 137.3)
prefill ~1,700 t/s
VRAM 1.59 GiB at a 32K window, 0.28 GiB GTT, no spill
load about 2 s
file 1.10 GB

Benchmark: Agon-Bench, 66 points

Full three-event pass, three runs per event, against the base model measured on the same harness:

event max LFM3-1.7B Voltron-1 Mini delta
Agent 24 18.00 18.67 +0.67
Code 25 13.00 17.67 +4.67
Reason 17 15.67 15.67 0.00
Total 66 46.67 52.00 +5.33

Rank 21 of 36 visible entries on the Agon-Bench board (harness and board: GuideboardLabs/agon-arena), at the same 1.1 GB and the same speed as the base.

Three disclosures, because a table without them is a claim too strong:

  • These are automated Agon scores, checked against the harness's known grading defects but not hand-graded task by task.
  • The code event was re-run at a 32K window. At the base entry's 16K window this model scored 16.67 on code, with four responses cut off mid-answer at around 70,000 characters, which is a window limit and not a capability result. On the fully comparable 16K configuration the total is 51.01.
  • The base model's 46.67 was measured at 16K, so the +4.67 on code compares a widened window to a narrow one. The like-for-like figure is +4.34 overall.

What it is good for, and what it is not

Best on code tasks that fit in one file: string handling, parsers, a multi-file CLI. Agent tasks are mostly fine. On the hardest code task in that suite (a regex engine written from scratch) all three runs produced scripts that crash on load, so the 17.67 is not evidence of that capability.

At 1.73B it is small. It will lose to any competent 7B and up on open-ended work. Its case is speed and footprint: 1.1 GB of weights at 135 t/s on an 8 GB card, with a real gain over the base model.

Attribution and license

  • Fine-tune: Akahsizrr / @RoliumGens (model)
  • Base model: Liquid AI (LFM3-1.7B)
  • GGUF conversion: Guideboard Labs

Licensed under the LFM Open License v1.0, the same license as the base model. A copy ships as LICENSE in this repo, and redistribution requires you to keep it. Section 5 limits commercial use by entities above the license's revenue threshold, so read the license before using this commercially. Model names and marks belong to their owners, used here only to describe origin.

Checksums

a4b5fd8249851528101c548e37f16e5911fcb1ea1378474817ca684ce3be2191  voltron-1-mini-Q4_K_M.gguf
1c7c3ce8f269f6dde72a2bf28133cbe00d2114f30426bc969ae331cfe9e38bdf  voltron-1-mini-Q8_0.gguf
Downloads last month
416
GGUF
Model size
2B params
Architecture
lfm3
Hardware compatibility
Log In to add your hardware

4-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for guideboardlabs/Voltron-1-Mini-GGUF

Quantized
(2)
this model