| ---
|
| license: apache-2.0
|
| base_model: Qwen/Qwen3-4B
|
| base_model_relation: quantized
|
| library_name: epure-runtime
|
| pipeline_tag: text-generation
|
| language:
|
| - en
|
| tags:
|
| - exeaon
|
| - epure
|
| - compressed
|
| - quantized
|
| - edge
|
| - cpu
|
| ---
|
|
|
| # Exeaon-Dzo-4B
|
|
|
| Compressed with E-PURE. Runs with the free
|
| [`epure-runtime`](https://github.com/ExeaonLM/epure-runtime), and **stays
|
| compressed in memory** — the dense weight is never assembled.
|
|
|
| | | |
|
| |---|---|
|
| | Base model | [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) |
|
| | Size on disk | **2.040 GB** |
|
| | Compression | **3.67x** |
|
| | Bits per weight | 4.33 (measured index entropy) |
|
| | Format | `.ebin` |
|
|
|
| ## Quality
|
|
|
| Measured against the base model on the same hardware, same harness version,
|
| same `limit`. Not copied from anyone's README.
|
|
|
| | benchmark | Qwen3-4B | Exeaon-Dzo-4B | delta |
|
| |---|---|---|---|
|
| | ARC-Challenge | 51.67 | 49.33 | -2.34 |
|
| | ARC-Easy | 78.33 | 76.00 | -2.33 |
|
| | HellaSwag | 57.67 | 57.67 | +0.00 |
|
| | PIQA | 74.67 | 74.33 | -0.34 |
|
| | **mean** | **65.59** | **64.33** | **-1.25** |
|
|
|
| **Retention: 98.1%** of base mean accuracy.
|
|
|
| Note where the loss sits rather than reading only the mean: reasoning-heavy
|
| tasks degrade first, and a model chosen for retrieval or classification is
|
| affected differently from one chosen for reasoning.
|
|
|
| ## Speed and footprint
|
|
|
| | | Qwen3-4B | Exeaon-Dzo-4B |
|
| |---|---|---|
|
| | decode, batch 1 | 11.9 tok/s | 8.5 tok/s |
|
| | decode, batch 8 | 89.0 tok/s | 30.1 tok/s |
|
| | peak VRAM | 8.65 GB | **5.02 GB** |
|
|
|
| **Read this honestly: on GPU we are slower than dense fp16.** A vendor
|
| tensor-core GEMM is extremely well tuned and a large GPU has bandwidth to
|
| spare, so dequantization costs more cycles than compression buys back. The win
|
| on GPU is footprint — whether the model fits, and what is left for the KV
|
| cache. On CPU, where bandwidth is the binding constraint, the fused kernel wins
|
| on both footprint and speed.
|
|
|
| Measured on Tesla T4, torch 2.5.1+cu121,
|
| lm-eval 0.4.12, limit 300, kernel `rust+triton`.
|
|
|
| ## Usage
|
|
|
| ```bash
|
| pip install epure-runtime
|
| ```
|
|
|
| ```python
|
| from epure import load
|
|
|
| model, tok = load("Exeaon/Exeaon-Dzo-4B")
|
| ids = tok("Explain the memory wall.", return_tensors="pt").input_ids
|
| print(tok.decode(model.generate(ids, max_new_tokens=128)[0]))
|
| ```
|
|
|
| ```bash
|
| epure run Exeaon/Exeaon-Dzo-4B --prompt "Explain the memory wall."
|
| ```
|
|
|
| No Rust toolchain needed: the fused CPU kernel ships compiled inside the wheel.
|
|
|
| ## Fine-tuning without decompressing
|
|
|
| Quantization indices stay frozen; the codebook and per-group scales train —
|
| about 1% of the weight values. The dense weight is never materialized, so
|
| memory tracks activations rather than parameter count.
|
|
|
| ```python
|
| from epure import load, make_trainable, snapshot_indices, verify_frozen
|
|
|
| model, tok = load("Exeaon/Exeaon-Dzo-4B")
|
| params, n = make_trainable(model, mode="both")
|
| before = snapshot_indices(model)
|
| ... # an ordinary PyTorch loop over `params`
|
| verify_frozen(model, before) # raises if any index moved
|
| ```
|
|
|
| ## Limitations
|
|
|
| - Compression is lossy; the measured gap is in the table above.
|
| - The KV cache is not compressed by default and can exceed the weights at long
|
| context.
|
| - GPU decode is slower than dense fp16 (see above).
|
| - Inherits every limitation, bias and knowledge cutoff of the base model.
|
| - Not evaluated for safety-critical, medical, legal or financial use.
|
|
|
| ## Licence and attribution
|
|
|
| Derived from [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), released under Apache-2.0, which
|
| permits redistribution of modified versions. The original `LICENSE` and
|
| `NOTICE` files are included in this repository unmodified, as it requires.
|
|
|
| **Ours:** the compression method, calibration, packaging, runtime.
|
| **Not ours:** the pretrained knowledge, which comes from the base model
|
| authors. This model is not endorsed by or affiliated with them.
|
|
|
| ## Citation
|
|
|
| ```bibtex
|
| @misc{exeaon2026,
|
| title = {Exeaon: compressed language models that run, and train, without
|
| decompressing},
|
| author = {Akpalu, Elliot Elikplim},
|
| year = {2026},
|
| publisher = {Zenux Plimver Technologies LTD},
|
| url = {https://huggingface.co/Exeaon}
|
| }
|
| ```
|
|
|
| ---
|
|
|
| <sub>Zenux Plimver Technologies LTD, Ghana</sub>
|
|
|
