| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-14B |
| base_model_relation: quantized |
| library_name: epure-runtime |
| pipeline_tag: text-generation |
| language: |
| - en |
| tags: |
| - exeaon |
| - epure |
| - compressed |
| - quantized |
| - edge |
| --- |
| |
| # Exeaon1-Nunya-14B |
|
|
| Compressed with E-PURE. Runs with the free |
| [`epure-runtime`](https://github.com/ExeaonLM/epure-runtime), and **stays |
| compressed in memory** -- the dense weight is never assembled. |
|
|
| | | | |
| |---|---| |
| | Base model | [Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B) | |
| | Size on disk | **7.38 GB** | |
| | Compression | **3.73x** | |
| | Format | `.ebin` | |
|
|
| ## Quality |
|
|
| Measured against the base model on the same GPU, the same harness version and |
| the same `limit`. Not copied from anyone's README. |
|
|
| | benchmark | base | Exeaon1-Nunya-14B | delta | |
| |---|---|---|---| |
| | ARC-Challenge | 59.67 | 60.00 | +0.33 | |
| | ARC-Easy | 81.33 | 81.67 | +0.34 | |
| | HellaSwag | 66.67 | 67.33 | +0.66 | |
| | PIQA | 81.33 | 81.00 | -0.33 | |
| | **mean** | **72.25** | **72.50** | **+0.25** | |
|
|
| **Retention: 100.3%** of base mean accuracy. |
|
|
| The compressed model scores fractionally **above** the original on this sample. That is not a gain -- it is measurement noise at `limit=300`, where a single item is worth 0.33 points. What it does mean is that the loss from compression is smaller than this benchmark can resolve. |
|
|
| Scored at `limit=300`, so roughly 2-3 points of standard error per task. |
| Read the mean, not any single row. |
|
|
| ## Speed and footprint |
|
|
| | | base | Exeaon1-Nunya-14B | |
| |---|---|---| |
| | decode, batch 1 | 22.4 tok/s | 14.9 tok/s | |
| | decode, batch 8 | 172.1 tok/s | 29.7 tok/s | |
| | peak VRAM | 28.67 GB | **19.14 GB** | |
|
|
| **Read this honestly: on GPU we are slower than dense fp16.** A vendor |
| tensor-core GEMM is heavily tuned and a large GPU has bandwidth to spare, so |
| trading compute for memory loses there. The win is fitting in less memory -- |
| running where the dense model does not fit at all, on a smaller card, or |
| alongside more of them. |
|
|
| Measured on NVIDIA A100-SXM4-40GB. |
|
|
| ## Usage |
|
|
| ```python |
| pip install epure-runtime |
| ``` |
|
|
| ```python |
| import epure |
| |
| model, tok = epure.load("model.ebin", device="cuda") |
| print(tok.decode(model.generate(**tok("Hello", return_tensors="pt").to("cuda"), |
| max_new_tokens=50)[0])) |
| ``` |
|
|
| ## Licence |
|
|
| The runtime is Apache-2.0 and free. The compression method that produced this |
| container is proprietary to Zenux Plimver Technologies LTD. |
|
|
