RGOO-Qwen3-0.6B

Qwen3-0.6B compressed with RGOO, at 3 bits and at 2 bits per number in its linear layers.

RGOO lets every row of every weight table choose its own precision. A row's level minimizes the damage its rounding is predicted to do to the model's next-token distribution, plus the bits its codes cost after entropy coding. GPTQ then chooses the codes, and a short retuning of the row scales and normalization gains, with the codes frozen, makes the compressed model answer like the original. The files are real entropy-coded files; loading one decodes it into an ordinary transformers model. The method and its evaluation are in rgoo.pdf.

Files

File	Linear layers	Size	Agreement	Perplexity	KL
Qwen3-0.6B, original	16 bits	1.19 GB	100%	23.02	0
qwen3-0.6b-3bit.rgoo	3.01 bits	478 MB	82.9%	24.15	0.110
qwen3-0.6b-2bit.rgoo	2.03 bits	424 MB	66.3%	32.12	0.479

All 196 linear layers are compressed. The word table (311 MB) and the normalization gains stay at 16 bits, so a whole file averages 6.4 bits per number at 3 bits and 5.7 at 2 bits. Agreement is the share of 100,352 held-out positions at which the compressed model's most likely next token is the original's, measured on the files as loaded back. Perplexity is on the real next tokens, and KL is KL(original || compressed) per token. On Python source code, agreement is 85.6% at 3 bits and 65.1% at 2 bits (paper, Table 5).

In our quick chat tests, the 3-bit model answered simple questions correctly and wrote correct short code; the 2-bit model made noticeably more mistakes.

Usage

The rgoo code ships in this repository. It needs PyTorch with CUDA, transformers, NumPy and SciPy (requirements.txt).

Chat in a terminal:

hf download Yougori/RGOO-Qwen3-0.6B --local-dir RGOO-Qwen3-0.6B
cd RGOO-Qwen3-0.6B
python -m rgoo chat qwen3-0.6b-3bit.rgoo

In Python:

import sys

from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

path = snapshot_download("Yougori/RGOO-Qwen3-0.6B",
                         allow_patterns=["*.json", "*.txt", "rgoo/*", "qwen3-0.6b-3bit.rgoo"])
sys.path.insert(0, path)            # the rgoo code ships with the model
import rgoo

model = rgoo.load(f"{path}/qwen3-0.6b-3bit.rgoo")    # config and tokenizer come from this repository
tok = AutoTokenizer.from_pretrained(path)

messages = [{"role": "user", "content": "What is the capital of France?"}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
ids = tok(text, return_tensors="pt").input_ids.to(model.device)
out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.7, top_p=0.8, top_k=20)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

rgoo.load returns an ordinary transformers model in bfloat16, so generate, chat templates and thinking mode work as they do for Qwen3-0.6B.

How the files were made

python -m rgoo compress Qwen/Qwen3-0.6B --bits 3 --calib calib.txt --out qwen3-0.6b-3bit.rgoo
python -m rgoo compress Qwen/Qwen3-0.6B --bits 2 --calib calib.txt --out qwen3-0.6b-2bit.rgoo

The calibration text was 391,569 tokens of documents from a pretraining mixture of educational web text (70%), code (15%), mathematics (11%) and synthetic textbooks (4%). The agreement, perplexity and KL above are measured on that mixture's held-out validation stream. Each file took about 6 minutes to make on one NVIDIA GeForce RTX 5090 Laptop GPU.

Limitations

• A file is decoded to ordinary 16-bit weights when it is loaded. It saves storage and download size, not memory or time at inference.
• Loading needs the rgoo code in this repository; transformers cannot read .rgoo files by itself.
• The evaluation uses one calibration set, one held-out stream and one seed. See the paper's Limitations section.
• Qwen3-0.6B's own limitations apply. It is a small model and can be wrong, biased or repetitive. Do not use it for high-impact decisions.

Licenses

The compressed weights are derived from Qwen3-0.6B by the Qwen team, which is licensed under the Apache License 2.0 (LICENSE-Qwen3). They are distributed under the same license, as is the rgoo code (LICENSE). Changes from the original: the 196 linear layers were re-encoded with RGOO, and the normalization gains were retuned. The configuration and tokenizer files are Qwen's, unchanged. The paper, rgoo.pdf, is licensed under CC BY 4.0 (PAPER_LICENSE.txt).

Citation

@misc{yougori2026rgoo,
  title  = {RGOO: Fisher-Allocated, Entropy-Coded Weight Compression for Language Models},
  author = {Yougori},
  year   = {2026},
  url    = {https://huggingface.co/Yougori/RGOO-Qwen3-0.6B}
}

@article{yang2025qwen3,
  title   = {Qwen3 Technical Report},
  author  = {Yang, An and others},
  journal = {arXiv preprint arXiv:2505.09388},
  year    = {2025}
}