Qwen3-14B-Base-QTEA (1.7-bit ternary)

Qwen3-14B-Base quantized to an effective 1.7 bits per weight with QTEA, a post-training method that ternarizes every linear weight to {-1, 0, +1} and repairs the resulting error with a sparse FP8 residual on the most salient columns. No retraining is involved.

QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization — EMNLP 2026 Main Conference.

⚠️ This checkpoint requires the QTEA codebase

This is not a standard Transformers checkpoint and cannot be loaded with AutoModelForCausalLM.from_pretrained. The file packed_qwen3_14b.pt contains only the quantized linear layers — packed ternary codes, FP8 residuals and scales. Embeddings, normalization layers and the LM head are read from the original Qwen/Qwen3-14B-Base, which is downloaded separately.

Use the reference implementation: https://github.com/Intelligent-Microsystems-Lab/QTEA

git clone https://github.com/Intelligent-Microsystems-Lab/QTEA
cd QTEA
pip install -r requirements.txt

# fetch this checkpoint
huggingface-cli download ims-lab/Qwen3-14B-base-QTEA packed_qwen3_14b.pt --local-dir packed/

# evaluate it (pulls Qwen/Qwen3-14B-Base for the non-quantized parts)
python eval/evaluate.py \
    --model Qwen/Qwen3-14B-Base \
    --checkpoint packed/packed_qwen3_14b.pt

Two backends are available. --backend dequant (the default) rebuilds dense FP16 weights and is the fastest way to reproduce the accuracy numbers below. --backend lut keeps the weights packed and runs the lookup-table CUDA kernel, which needs far less memory and is the path the paper's latency numbers come from; it requires nvcc on PATH.

Results

Perplexity and zero-shot accuracy for Qwen3-14B-Base, from Table 1 of the paper. All quantized models use group size 128.

Method	#W	WikiText2 ↓	C4 ↓	PiQA	ARC-e	ARC-c	Hella.	Wino.	OBQA	BoolQ	Avg ↑
FP16 (reference)	16	6.38	9.68	80.58	83.46	55.80	61.85	74.19	35.00	86.76	68.23
AWQ	2	2.68e7	2.18e7	53.00	24.60	23.00	25.30	50.70	20.00	46.20	34.69
GPTQ	2	37.90	74.50	56.31	34.64	20.65	33.30	52.72	17.20	46.33	37.31
Slim-LLM	2	22.85	68.38	61.83	52.54	29.35	31.52	52.04	20.40	61.20	44.13
PB-LLM	1.7	2.89e4	2.44e4	54.08	25.93	20.73	25.76	47.99	15.00	38.04	32.50
PT²-LLM	1.6	16.48	68.13	62.95	53.03	23.63	33.65	59.75	20.60	62.17	45.11
QTEA (this model)	1.7	11.78	26.14	68.93	67.42	34.39	41.17	64.72	26.00	65.93	52.65

Bold is the best sub-2-bit result, italic the second best; FP16 is the unquantized reference, not a competitor. Baseline numbers are taken from PT²-LLM (Yan et al., 2026).

Against PT²-LLM, the strongest sub-2-bit baseline, QTEA improves average zero-shot accuracy from 45.11 to 52.65 (a 16.7% relative gain) and lowers WikiText-2 perplexity by 1.40× (16.48 → 11.78) and C4 perplexity by 2.61× (68.13 → 26.14).

Zero-shot accuracy is the unnormalised average over the seven tasks. The paper's numbers were produced with the same harness generation as the published baselines, so individual task scores can move against a current lm_eval release even when the average lines up.

Efficiency

With the lookup-table CUDA kernel and CUDA Graphs, QTEA cuts per-token generation latency on Qwen3-14B from 23.75 to 6.56 ms — a 3.62× speedup over FP16 (13.3× against FP16 without CUDA Graphs). The sparse residual path is essentially free: latency matches a ternary-only kernel to within 0.01 ms/token. Speedup is stable at 3.07–3.20× for batch size 1 across 512–4096 tokens of context.

Quantization details

• Ternary base {-1, 0, +1} for every linear weight, group size 128.
• FP8 (E4M3) residuals on the top 5% most salient columns, 1:4 semi-sparse within those columns — 1.25% of all weights kept in FP8.
• Effective storage: 1.6 bpw for the ternary codes + 0.1 bpw for the residuals = 1.7 bpw.
• Calibrated on 256 sequences from the WikiText-2 training split.

Citation

@misc{guo2026qteaternaryllmssparse,
      title={QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization}, 
      author={Yipin Guo and Arun M George and Jie Fu and Tareq Mahmoud and Sixue Xing and Siddharth Joshi},
      year={2026},
      eprint={2609.00224},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2609.00224}, 
}

License

Released under the Apache 2.0 license, inherited from Qwen/Qwen3-14B-Base. The QTEA codebase is MIT-licensed.