File size: 2,585 Bytes
1a0d2b6 55c28df 4c5636a 55c28df 4c5636a 55c28df 4c5636a 1a0d2b6 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a 9c41926 4c5636a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
library_name: transformers
pipeline_tag: text-generation
license: apache-2.0
base_model:
- google/gemma-4-E2B
tags:
- quantization
- sub-4-bit
- sub1quant
- int4
- int2
- gemma
- gemma4
- wikitext
datasets:
- wikitext
---
# sub1quant mixed-budget Gemma 4 E2B artifacts
This repository contains the mixed-budget sub-4-bit artifact from `sub1quant`.
The base model is not mirrored here; download `google/gemma-4-E2B` separately.
## Current artifact
| File | Method | Avg BPW | Size |
|------|--------|--------:|-----:|
| `quantized/gemma_mixed_budget_full_g128_target4p0.pt` | mixed budget, g128, target 4.0 BPW | 3.9990 | 948 MB |
The checkpoint contains 316 language-model weight tensors:
| Format | Count |
|--------|------:|
| Groupwise INT4 | 301 |
| INT2 + binary residual | 14 |
| INT2 + error-budget k4 side channel | 1 |
## Live Colab evaluation
Run date: 2026-06-29
Hardware/runtime: NVIDIA L4, CUDA, dense BF16 evaluation after applying the quantized weights.
| Run | Runtime dtype | WikiText tokens | Chunks | PPL |
|-----|---------------|----------------:|-------:|----:|
| Unquantized `google/gemma-4-E2B` base | BF16 | 292,282 | 571 | 108.4542 |
| Mixed budget full g128 target 4.0 | BF16 dense eval after applying quantized weights | 292,282 | 571 | 107.5656 |
This supports a narrow claim: BF16-baseline-equivalent perplexity on this exact Gemma4/WikiText/Colab runner at about 4.00 BPW. It is not an FP16 result, not an FP8 comparison, and not a throughput result. The current evaluator reconstructs/applies weights into a normal dense model for correctness.
Result files:
- `eval_results/mixed_budget_full_g128_target4p0_ppl_live.json`
- `eval_results/base_full_ppl_live.json`
- `eval_results/mixed_budget_live_colab_comparison.json`
- `eval_results/mixed_budget_scan_full_g128_target4p0.json`
## Reproduce
```bash
pip install "transformers>=5.5.0" torch accelerate safetensors huggingface_hub
python -c "from huggingface_hub import snapshot_download; snapshot_download('google/gemma-4-E2B', local_dir='./models/gemma-4-E2B')"
python scripts/limited_ppl_bench.py \
--label mixed_budget_full_g128_target4p0 \
--model-dir models/gemma-4-E2B \
--wikitext data/wiki.test.txt \
--quantized-pt quantized/gemma_mixed_budget_full_g128_target4p0.pt \
--tokens 1000000000 \
--max-length 512 \
--stride 512 \
--device cuda \
--output eval_results/mixed_budget_full_g128_target4p0_ppl_live.json
```
## License
The quantization code and metadata in this repository are Apache-2.0. The base model remains governed by Google's Gemma license.
|