| --- |
| library_name: transformers |
| pipeline_tag: text-generation |
| license: apache-2.0 |
| base_model: |
| - google/gemma-4-E2B |
| tags: |
| - quantization |
| - sub-4-bit |
| - sub1quant |
| - int4 |
| - int2 |
| - gemma |
| - gemma4 |
| - wikitext |
| datasets: |
| - wikitext |
| --- |
| |
| # sub1quant mixed-budget Gemma 4 E2B artifacts |
|
|
| This repository contains the mixed-budget sub-4-bit artifact from `sub1quant`. |
| The base model is not mirrored here; download `google/gemma-4-E2B` separately. |
|
|
| ## Current artifact |
|
|
| | File | Method | Avg BPW | Size | |
| |------|--------|--------:|-----:| |
| | `quantized/gemma_mixed_budget_full_g128_target4p0.pt` | mixed budget, g128, target 4.0 BPW | 3.9990 | 948 MB | |
|
|
| The checkpoint contains 316 language-model weight tensors: |
|
|
| | Format | Count | |
| |--------|------:| |
| | Groupwise INT4 | 301 | |
| | INT2 + binary residual | 14 | |
| | INT2 + error-budget k4 side channel | 1 | |
|
|
| ## Live Colab evaluation |
|
|
| Run date: 2026-06-29 |
|
|
| Hardware/runtime: NVIDIA L4, CUDA, dense BF16 evaluation after applying the quantized weights. |
|
|
| | Run | Runtime dtype | WikiText tokens | Chunks | PPL | |
| |-----|---------------|----------------:|-------:|----:| |
| | Unquantized `google/gemma-4-E2B` base | BF16 | 292,282 | 571 | 108.4542 | |
| | Mixed budget full g128 target 4.0 | BF16 dense eval after applying quantized weights | 292,282 | 571 | 107.5656 | |
|
|
| This supports a narrow claim: BF16-baseline-equivalent perplexity on this exact Gemma4/WikiText/Colab runner at about 4.00 BPW. It is not an FP16 result, not an FP8 comparison, and not a throughput result. The current evaluator reconstructs/applies weights into a normal dense model for correctness. |
|
|
| Result files: |
|
|
| - `eval_results/mixed_budget_full_g128_target4p0_ppl_live.json` |
| - `eval_results/base_full_ppl_live.json` |
| - `eval_results/mixed_budget_live_colab_comparison.json` |
| - `eval_results/mixed_budget_scan_full_g128_target4p0.json` |
|
|
| ## Reproduce |
|
|
| ```bash |
| pip install "transformers>=5.5.0" torch accelerate safetensors huggingface_hub |
| |
| python -c "from huggingface_hub import snapshot_download; snapshot_download('google/gemma-4-E2B', local_dir='./models/gemma-4-E2B')" |
| |
| python scripts/limited_ppl_bench.py \ |
| --label mixed_budget_full_g128_target4p0 \ |
| --model-dir models/gemma-4-E2B \ |
| --wikitext data/wiki.test.txt \ |
| --quantized-pt quantized/gemma_mixed_budget_full_g128_target4p0.pt \ |
| --tokens 1000000000 \ |
| --max-length 512 \ |
| --stride 512 \ |
| --device cuda \ |
| --output eval_results/mixed_budget_full_g128_target4p0_ppl_live.json |
| ``` |
|
|
| ## License |
|
|
| The quantization code and metadata in this repository are Apache-2.0. The base model remains governed by Google's Gemma license. |
|
|
