kompress-ultra-bitnet-benchmarks

BitNet b1.58 ternary quantization metrics: memory reduction, perplexity, and Apple Silicon Metal throughput.

Files

• bitnet_b158_compression_results.json — the metrics payload.

Denominators (read these first)

The JSON intentionally carries two compression numbers because they mean different things:

Number	Meaning	Denominator
8.1x	measured memory reduction vs fp16	2 bits/weight (packed ternary layout; 16/2 = 8.0x + packing overhead) — the honest headline
1.58	theoretical entropy of one ternary weight	log2(3) = 1.585 — the information-theoretic bound, NOT the shipped layout

16 / 1.58 = 10.13x and 16 / 8.1 = 1.975 bits are both real, they just answer different questions. The JSON labels each explicitly.

Protocol

• Model: Qwen/Qwen2.5-0.5B BitNet b1.58 continued-train (masked CE, deployed-forward QAT), exported as 168 ternary matrices.
• Context: 256 tokens · Val: 90-sample stratified held-out.
• Throughput: memory-bound decode on Apple M3 Max (M4 Pro is NOT folded in; different memory system).
• 70B scale check: 70e9 × 1.975 bits / 8 = 17.28 GB weights/token → ~22 tok/s at 384 GB/s. The 142.8 tok/s here is the 0.5B model.