kingjones777/ZAYA1-8B-ROCmFP4-GGUF

⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

The zaya architecture is not merged upstream (draft PR #23112). Ignore the auto-generated "Use this model" commands above — use the ROCmFPX patch in patches/.

📦 4.86 GiB, smaller than Q4_K_M (5.19 GiB), from a 16.52 GiB BF16 source. ⚠️ Speed is parity, not a win — see the honest benchmark below.

ZAYA1-8B — ROCmFP4 (tier 102 COHERENT) GGUF

A 4-bit ROCmFP4 quantization of Zyphra/ZAYA1-8B for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo).

File	ZAYA1-8B-Q4_0_ROCMFP4_COHERENT.gguf
Size	4.8629 GiB (5,221,551,424 bytes)
BPW	4.71
ftype	Q4_0_ROCMFP4_COHERENT (102)
Source	BF16 GGUF (16.52 GiB) — lossless source, not a requantization
sha256	e663e10d86bd63b1d4cccb5cd3bde5eb0d7d7442f9cd0f5e3857d0fe2739357c

---

⚠️ Read this before choosing 4-bit for ZAYA1

ZAYA1 is not weight-bandwidth-bound, so 4-bit buys size — not speed. We measured all three builds on the same machine, median of 3, warm-up discarded, on an otherwise idle box:

build	size	median decode	range
ROCmFP4 (this)	4.86 GiB	15.8 tok/s	[14.1 – 20.49]
Q4_K_M	5.19 GiB	19.26 tok/s	[15.26 – 19.88]
BF16	16.52 GiB	17.0 tok/s	[16.38 – 17.41]

BF16 reads 3.4× more bytes per token and decodes at the same rate. A bandwidth-bound model would show roughly 3× separation here; ZAYA1 shows none. The bottleneck is compute — the Compressed Convolutional Attention grouped-conv path (cca_conv_grp, 10 groups per even layer).

All three ranges overlap, so we make no speed claim in either direction — this build is neither meaningfully faster nor slower than Q4_K_M. Take it for the 0.32 GiB size saving and the fact that it fits alongside more models, not for throughput.

Where the time actually goes

We localized decode cost by structurally skipping the CCA grouped convolution and re-measuring (rocprof is unavailable on this hardware):

build	baseline	conv skipped	conv share
ROCmFP4	25.0 / 24.5 tok/s	56.7 / 52.7	~54%
Q4_K_M	24.9 / 24.5 tok/s	55.5	~56%

The CCA grouped convolution is roughly 55% of decode time, and it is identical across quantizations — the conv weights are BF16 in both. Fitting t = fixed + bytes/BW to the BF16 and 4-bit points puts ~37 ms of ~42 ms per token in weight-independent fixed cost, i.e. decode is ~88% independent of the weight format.

⇒ No 4-bit format can make this model faster. A proper grouped-conv kernel could be worth up to ~2× end-to-end, but that is a llama.cpp kernel/conversion fix, not a quantization one. We prototyped a single-GEMM rewrite and did not ship it — it was not numerics-identical, and a speedup that changes output is not a speedup.

Correctness

Official-sampling checks, reading both reasoning_content and content with a 2500-token budget (ZAYA1 reasons at length — a small cap will make it look broken when it is not):

check	result
17 × 23	✅ 391
capital of Japan	✅ Tokyo
days in 2024	✅ 366

Per-tensor types (audited in the finished file, 1283 tensors)

tensor class	type
token_embd.weight	Q6_K
cca_conv_grp.weight	BF16 — never quantized
cca_conv_grp.bias, cca_k_scale	F32
ssm_conv1d.{weight,bias} (80)	F32
ffn_gate_inp router (80)	F32
norms (121)	F32
routed experts, cca_val_proj1/2	4-bit

⚠️ tie_word_embeddings is true on this model, so there is no separate output.weight and --output-tensor-type is a silent no-op. --token-embedding-type q6_K is the flag that actually lands — and on a 262,272-token vocabulary that matters a great deal.

cca_conv_grp is excluded from quantization in the patch. Those tensors drive the compressed convolutional attention and are highly sensitive; other ZAYA1 quantizers exclude them too.

---

What was NOT measured

• No perplexity run, and no quality A/B against Q4_K_M or BF16. The checks above are memorized-fact prompts — necessary but not sufficient.
• No long-context testing (the model supports 131,072).
• No tool-calling evaluation.
• No coding or reasoning benchmark.

Model

ZayaForCausalLM / zaya. 40 layers, all hybrid · hidden 2048 · vocab 262,272 · 16 experts, 1 active · ssm_d_conv = 2 · 1283 tensors · context 131,072 · tie_word_embeddings: true. ~8.4B total / ~760M active.

Base model licence: Apache-2.0 (inherited). Credit for the model itself goes to Zyphra.

All quant variants

Three builds of this model, all measured in one session on one box with one binary (Ryzen AI MAX+ 395, gfx1151, ROCm 7.2.4, ROCmFPX-2809dc5) — so these rows are directly comparable. Median of 3, warm-up discarded, otherwise-idle box.

variant	ftype	size	bpw	decode (median)	range	repo
4-bit COHERENT	102	4.86 GiB	4.71	23.04	22.83 – 23.70	ZAYA1-8B-ROCmFP4-GGUF
8-bit AGENT	115	8.72 GiB	8.45	21.02	20.95 – 21.47	ZAYA1-8B-ROCmFPX-Q8_0-AGENT-GGUF
8-bit plain	111	8.59 GiB	8.32	21.08	20.99 – 21.20	ZAYA1-8B-ROCmFPX-Q8_0-GGUF

⚠️ Decode is ~88% weight-independent on this architecture (the CCA grouped conv is ~55% of decode). All three builds land within ~10% of each other; the 4-bit is smallest and marginally fastest. No 8-bit or 4-bit format will make this model meaningfully faster.

What AGENT actually changes: it keeps far more tensors at true Q8_0 instead of the packed 8-bit type — measured in these files, 154 tensors vs 1 tensor. On models with an MTP draft head that raises draft acceptance and wins ~6%; these two models have no MTP head, and here the two 8-bit builds are within noise of each other.