File size: 4,360 Bytes
fe40fb8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 | ---
license: apache-2.0
base_model: Qwen/Qwen3-4B
base_model_relation: quantized
library_name: epure-runtime
pipeline_tag: text-generation
language:
- en
tags:
- exeaon
- epure
- compressed
- quantized
- edge
- cpu
---
# Exeaon-Dzo-4B
Compressed with E-PURE. Runs with the free
[`epure-runtime`](https://github.com/ExeaonLM/epure-runtime), and **stays
compressed in memory** — the dense weight is never assembled.
| | |
|---|---|
| Base model | [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) |
| Size on disk | **2.040 GB** |
| Compression | **3.67x** |
| Bits per weight | 4.33 (measured index entropy) |
| Format | `.ebin` |
## Quality
Measured against the base model on the same hardware, same harness version,
same `limit`. Not copied from anyone's README.
| benchmark | Qwen3-4B | Exeaon-Dzo-4B | delta |
|---|---|---|---|
| ARC-Challenge | 51.67 | 49.33 | -2.34 |
| ARC-Easy | 78.33 | 76.00 | -2.33 |
| HellaSwag | 57.67 | 57.67 | +0.00 |
| PIQA | 74.67 | 74.33 | -0.34 |
| **mean** | **65.59** | **64.33** | **-1.25** |
**Retention: 98.1%** of base mean accuracy.
Note where the loss sits rather than reading only the mean: reasoning-heavy
tasks degrade first, and a model chosen for retrieval or classification is
affected differently from one chosen for reasoning.
## Speed and footprint
| | Qwen3-4B | Exeaon-Dzo-4B |
|---|---|---|
| decode, batch 1 | 11.9 tok/s | 8.5 tok/s |
| decode, batch 8 | 89.0 tok/s | 30.1 tok/s |
| peak VRAM | 8.65 GB | **5.02 GB** |
**Read this honestly: on GPU we are slower than dense fp16.** A vendor
tensor-core GEMM is extremely well tuned and a large GPU has bandwidth to
spare, so dequantization costs more cycles than compression buys back. The win
on GPU is footprint — whether the model fits, and what is left for the KV
cache. On CPU, where bandwidth is the binding constraint, the fused kernel wins
on both footprint and speed.
Measured on Tesla T4, torch 2.5.1+cu121,
lm-eval 0.4.12, limit 300, kernel `rust+triton`.
## Usage
```bash
pip install epure-runtime
```
```python
from epure import load
model, tok = load("Exeaon/Exeaon-Dzo-4B")
ids = tok("Explain the memory wall.", return_tensors="pt").input_ids
print(tok.decode(model.generate(ids, max_new_tokens=128)[0]))
```
```bash
epure run Exeaon/Exeaon-Dzo-4B --prompt "Explain the memory wall."
```
No Rust toolchain needed: the fused CPU kernel ships compiled inside the wheel.
## Fine-tuning without decompressing
Quantization indices stay frozen; the codebook and per-group scales train —
about 1% of the weight values. The dense weight is never materialized, so
memory tracks activations rather than parameter count.
```python
from epure import load, make_trainable, snapshot_indices, verify_frozen
model, tok = load("Exeaon/Exeaon-Dzo-4B")
params, n = make_trainable(model, mode="both")
before = snapshot_indices(model)
... # an ordinary PyTorch loop over `params`
verify_frozen(model, before) # raises if any index moved
```
## Limitations
- Compression is lossy; the measured gap is in the table above.
- The KV cache is not compressed by default and can exceed the weights at long
context.
- GPU decode is slower than dense fp16 (see above).
- Inherits every limitation, bias and knowledge cutoff of the base model.
- Not evaluated for safety-critical, medical, legal or financial use.
## Licence and attribution
Derived from [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), released under Apache-2.0, which
permits redistribution of modified versions. The original `LICENSE` and
`NOTICE` files are included in this repository unmodified, as it requires.
**Ours:** the compression method, calibration, packaging, runtime.
**Not ours:** the pretrained knowledge, which comes from the base model
authors. This model is not endorsed by or affiliated with them.
## Citation
```bibtex
@misc{exeaon2026,
title = {Exeaon: compressed language models that run, and train, without
decompressing},
author = {Akpalu, Elliot Elikplim},
year = {2026},
publisher = {Zenux Plimver Technologies LTD},
url = {https://huggingface.co/Exeaon}
}
```
---
<sub>Zenux Plimver Technologies LTD, Ghana</sub>
|