Victoria

70.0% on Terminal-Bench 2.1 (avg@3) from a model that uses 5.9B parameters per token, at 279.6 tok/s on one stream.

Victoria is Qwen/Qwen3.8-Flash-Next with 44% of its experts removed, 512 down to 288 per layer, then retrained so it still works well at 4-bit. This update replaces the NVFP4 build with a retrained checkpoint (step 810) and ships a draft head trained for it.

Why this model

• Fast decode. 5.9B active parameters per token, the same as Qwen3.8-Flash-Next: pruning removed experts, and each token still goes through 10 of the 288 that are left. With the included draft head on, the NVFP4 build decodes 279.6 tok/s on one stream on one NVIDIA B300, 2.08x its 134.7 tok/s with the head off.
• 70.0% on Terminal-Bench 2.1, avg@3. Three full runs of all 89 tasks at 8 hours per task scored 75.3%, 68.5% and 66.3%. The previous NVFP4 build scored 62.5% on the identical protocol, run side by side in the same job.
• Size. 48.0 GiB of weights including the draft head, plus the model's 95.4 GiB n-gram lookup table. The GGUF build holds 49.17 GiB of weights on llama.cpp.
• Fewer wasted tokens. Across those three runs the new build used 35% fewer output tokens than the previous NVFP4 build (69.4M against 107.3M), and its share of empty agent turns fell from 10-11% to 4-7%.
• Two runtimes. NVFP4 for vLLM on NVIDIA Blackwell GPUs, and GGUF Q4_K_M for llama.cpp.

Builds

The two builds are different checkpoints. The NVFP4 build is step 810 of the NVFP4 retraining. The GGUF build is step 410 of the first retraining run, the checkpoint published with the first release. Each build's scores below were measured on that build only.

Build	Checkpoint	Runtime	Download	Resident weights	Terminal-Bench 2.1, 8 h per task	HumanEval
nvfp4/	step 810, with a draft head trained for it	vLLM, NVIDIA B300 or B200	154.0 GB (143.4 GiB)	48.0 GiB of weights plus the 95.4 GiB lookup table; vLLM holds both on the GPU	70.0% avg@3 (75.3 / 68.5 / 66.3)	97.0% (159/164), one sample per problem
gguf/ Q4_K_M	step 410	llama.cpp	107.20 GB (smaller download, same model) or 155.20 GB (full-precision lookup table)	52.79 GB (49.17 GiB)	75.28% (67/89), one run	93.2% avg@5

Quick start

Download:

hf download rmonsurate/Victoria --include "nvfp4/*" --local-dir .

# full GGUF set, 155.20 GB
hf download rmonsurate/Victoria --include "gguf/victoria-s410-bitexact-0*" --local-dir .

# smaller download, 107.20 GB: the model's lookup table is stored at 8-bit, GPU memory use is the same
hf download rmonsurate/Victoria --include "gguf/victoria-s410-bitexact-tbl8-*" --local-dir .

nvfp4/SHA256SUMS lists the sha256 of every file in the NVFP4 build.

NVFP4 on vLLM, one GPU, draft head on. This is the command the decode speeds below were measured with:

vllm serve /path/to/nvfp4 \
  --served-model-name victoria \
  --tensor-parallel-size 1 \
  --max-model-len 20480 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.80 \
  --enable-prefix-caching \
  --compilation-config '{"inductor_compile_config":{"triton.autotune_at_compile_time":false}}' \
  --kernel-config '{"enable_flashinfer_autotune":false}' \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

• The --compilation-config line is required: compile-time Triton autotuning on this model either hangs or blows out startup time.
• For long agent sessions, the Terminal-Bench runs used --max-model-len 262144 --max-num-seqs 64 with everything else the same, minus --speculative-config.
• We ran vLLM from the Qwen3.8-Flash-Next vLLM image with two fixes backported for a prefix-caching crash on this architecture (vllm#53798 and vllm#54076, open upstream when we built it). On a vLLM without those fixes, leave out --enable-prefix-caching.

Run the GGUF files with speculative decoding off, since public llama.cpp cannot use this model's draft head. We ran llama.cpp build b10667, 8 slots at 16384 context each, across two GPUs. The block below is UNTESTED as written and you will need to work out the offload flags for your own machine:

llama-server \
  -m victoria-s410-bitexact-00001-of-00003.gguf \
  -c 16384 \
  --parallel 8 \
  --host 0.0.0.0 --port 8080

Evaluation

NVFP4 build, step 810

Model	Terminal-Bench 2.1, avg@3	Run 1	Run 2	Run 3
Victoria NVFP4, step 810 (this release)	70.04%	67/89	61/89	59/89
Victoria NVFP4, step 510 (previous NVFP4 build)	62.54%	57/89	54/89	56/89

Counting passes out of three runs per task, step 810 does better than step 510 on 18 tasks and worse on 6, and ties on the rest (two-sided exact sign test, p = 0.023). Both builds ran in one job on one 8x B300 node, four vLLM replicas each, the same serving flags and the same concurrency, speculative decoding off. Harbor 0.22.0 with the Terminus 2 agent, 8 hours per task, 262144 context, all 89 tasks in every denominator; a trial with no verdict counts as a fail. Per-task results for every run are in docs/TB21_S810_AVG3.md.

HumanEval on the step-810 NVFP4 weights is 97.0% (159/164): one sample per problem, thinking on, temperature 0.6, top_p 0.95, top_k 20. That is a single run, not an average over five runs like the GGUF figure, so the two numbers are not directly comparable. The previous NVFP4 build scored 93.9% (154/164) on the same single-run protocol.

Decode speed and the draft head

Setup	Output tok/s, mean of 3 runs (each run)	Draft acceptance	Tokens per forward pass of the main model
Draft head off	134.7 (133.7, 133.6, 136.9)	n/a	1
Draft head as pruned, never retrained	269.3 (270.4, 269.8, 267.6)	64.1%	2.92
Trained draft head, shipped in nvfp4/	279.6 (284.0, 278.5, 276.2)	67.6%	3.03

One NVIDIA B300, one stream, the vLLM command above, 50 fixed prompts (20 coding, 30 agentic), temperature 0.6, top_p 0.95, thinking off, up to 3 draft tokens per step, a fresh server for each run. Acceptance is accepted draft tokens over proposed draft tokens.

The draft head is the model's built-in multi-token-prediction layer, which vLLM uses through its mtp speculative method. Pruning cut its experts from 512 to 288 like the rest of the model, and the retraining never updated it, so it was trained afterwards on its own: the step-810 model was frozen and the head was trained to match that model's own next-token distribution on its replies (self-distillation), 500 updates of 262,144 tokens each. The rest of the NVFP4 files are byte-identical to the checkpoint scored on Terminal-Bench above. The draft head only proposes tokens and the main model verifies every one, so the head changes speed, not the weights that produce the answers. The trained head beat the untrained one in every run, and its slowest run was faster than the untrained head's fastest.

GGUF build, step 410

Model	Terminal-Bench 2.1	Kept vs original
Victoria GGUF Q4_K_M, step 410	75.28% (67/89)	84.8%
Qwen3.8-Flash-Next, the model Victoria came from	88.76% (79/89)	baseline
Qwen3.8-27B dense, avg@3 with Claude Code	73.0%	not applicable
Claude Opus 4.6 (Max)	78.2%	as listed on Qwen's Qwen3.8-27B model card

The last two rows come from Qwen's Qwen3.8-27B model card and were not run on our setup, so treat them as a rough guide.

Setup: Harbor 0.22.0 with the Terminus 2 agent, 8 hours per task, avg@1, all 89 tasks in the denominator, 262144 context, one B300 node. One avg@1 run over 89 tasks carries a standard error of roughly 5 points. HumanEval on the same weights is 93.2% avg@5, sd 1.3, at n=164. We evaluated text only. Per-task detail is in docs/TB21_8H_SCORES.md. The full write-up is at monsurate.com/post/victoria-release.

How it was made

We removed 44% of the model's experts, the ones contributing least to its output, using a method called REAP. We then retrained what was left to match the full model's output, over 128 million tokens of coding and office tool-use data it generated for itself, with the training run at 4-bit so the weights land on llama.cpp's Q4_K_M values exactly and nothing has to be refitted afterwards. That retraining follows NVIDIA's Quantization-Aware Distillation work. The GGUF build is step 410 of that run.

The NVFP4 build continues the same retraining in NVIDIA's 4-bit format. The first release shipped its step 510. For this update we fixed two problems we found in the training pipeline: documents packed into one training sequence could see each other, and the distillation loss covered prompt and tool-output tokens as well as the model's own replies. Training resumed from step 510 with each document isolated and the loss on the model's replies only, and step 810 is the checkpoint published here. The draft head was trained after that, as described above.

License

Victoria is a derivative work of Qwen3.8-Flash-Next by Farpoint Technologies and carries the same license, the Qwen Community License 1.0, whose full text ships in this repo as LICENSE. Two conditions travel with it: the copyright and permission notice must be included in all copies, and a licensee running a Model as a Service or AI Work Assistant business must obtain a separate license from Qwen before any commercial use. Read the license text and talk to Qwen if that describes what you are building. Attribution goes to Qwen for Qwen3.8-Flash-Next and to Farpoint for Victoria.

Citations

• QAD: Xin, Mao, Ping, Catanzaro, Han et al. (NVIDIA). Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery. arXiv:2601.20088.
• REAP: Lasby, Lazarevich, Sinnadurai, Lie, Ioannou, Thangarasa (Cerebras). REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression. arXiv:2510.13999, ICLR 2026.
• Next-latent prediction: Teoh, Tomar, Ahn, Hu, Pearce, Sharma, Krishnamurthy, Islam, Lamb, Langford (Microsoft Research). Next-Latent Prediction Transformers Learn Compact World Models. arXiv:2511.05963.
• Engram and conditional memory, the idea behind Flash-Next's n-gram table: Cheng, Tian, Zeng, Dai et al. (DeepSeek). Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models. arXiv:2601.07372.

Acknowledgements

This work was made possible by the support of Dell, who provided lab access to the 8x NVIDIA B300 system it was built and evaluated on, and of Mario at Buzz (NASDAQ: HIVE). Thanks to the NVIDIA researchers behind Quantization-Aware Distillation and the NVFP4 format, to the Qwen team for Qwen3.8-Flash-Next, and to the maintainers of llama.cpp, vLLM, harbor and Terminal-Bench.