| --- |
| license: apache-2.0 |
| base_model: google/gemma-4-E2B-it-qat-q4_0-unquantized |
| pipeline_tag: text-generation |
| tags: |
| - gemma4 |
| - int8 |
| - cpu |
| --- |
| |
| # Gemma 4 E2B int8 for gemma4.c |
|
|
| This is the ready-to-run model file for [gemma4.c](https://github.com/ryanssenn/gemma4.c), a small educational implementation of Gemma 4 E2B inference on CPU. |
|
|
| The project's `exporter.py` converts the original [Google Gemma 4 E2B QAT checkpoint](https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-unquantized) into one 4.7 GiB file containing the model configuration, tokenizer, and weights. During export, matrix weights are quantized to int8 with FP16 scales. |
|
|
| ## Download |
|
|
| ```bash |
| hf download QmogAI/gemma4-e2b-int8 gemma4-E2B-int8.bin --local-dir . |
| ``` |
|
|
| ## Run |
|
|
| ```bash |
| git clone https://github.com/ryanssenn/gemma4.c |
| cd gemma4.c |
| make |
| ./run -m ./gemma4-E2B-int8.bin -t 1.0 -n 256 "Why is the sky blue?" |
| ``` |
|
|
| ## Export your own |
|
|
| If you would rather create the file from the original checkpoint: |
|
|
| ```bash |
| python3 -m pip install -r requirements.txt |
| python3 exporter.py /path/to/gemma-4-E2B-it-qat-q4_0-unquantized -o ./gemma4-E2B-int8.bin |
| ``` |
|
|
| The runtime is made for x86-64 CPUs with AVX2 and FMA. AVX-512 VNNI is used when available. It is intended for learning and experimentation. |
|
|
