Clarify which EPQ version (v1/v2) is active for Flint
Browse files
README.md
CHANGED
|
@@ -108,8 +108,22 @@ Throughput varied by about ±15 % between runs of the same tier; treat the t/s c
|
|
| 108 |
|
| 109 |
See [`results/test-status.md`](results/test-status.md) for the full system and validation record, including what was not tested.
|
| 110 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 111 |
## Attribution
|
| 112 |
|
| 113 |
-
The pager
|
| 114 |
|
| 115 |
The GGUF weights remain hosted by Unsloth. This repository contains source, patch, scripts, logs and test notes only.
|
|
|
|
| 108 |
|
| 109 |
See [`results/test-status.md`](results/test-status.md) for the full system and validation record, including what was not tested.
|
| 110 |
|
| 111 |
+
## Which EPQ version is ported
|
| 112 |
+
|
| 113 |
+
The source is [HelloSun's Nemotron EPQ](https://huggingface.co/HelloSun/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF_EPQ) with two patches applied in order: `0001-epq-nemotron-h-moe.patch` (EPQ v1) and `epqv2.patch` (EPQ v2 increment). The Flint port applies both, but only these parts are active for Flint:
|
| 114 |
+
|
| 115 |
+
| Feature | Source | Status for Flint |
|
| 116 |
+
|---|---|---|
|
| 117 |
+
| Expert paging, SSD→arena, per-expert custom op ("mode C") | v1 | Active; this is what all measurements use |
|
| 118 |
+
| Preload all experts into the arena | v2 | Code present; runs only when the arena holds all 12,032 experts (about 28.6 GiB at 2.44 MiB/slot). No measured tier reaches this, so it is not exercised here |
|
| 119 |
+
| Row-parallel MoE (16-thread row split) | v2 | Inactive: gated only for no-gate (Nemotron) experts; Flint has gate/up/down |
|
| 120 |
+
|
| 121 |
+
The Flint-specific hook routes gated SiLU experts through the pager. Nemotron's no-gate/ReLU-squared path is not used by Flint.
|
| 122 |
+
|
| 123 |
+
Flint's upstream `build_moe_ffn` also applies a power-of-two amax rescale of the down-projection input and undoes it on the output. This rescale is exact in F32 and Q8_K quantization is scale-invariant, so the paged path omits it. The layer-2–6 MoE comparison against native (which includes the rescale) confirms that the omission changes no results beyond float-summation noise.
|
| 124 |
+
|
| 125 |
## Attribution
|
| 126 |
|
| 127 |
+
The pager is adapted from HelloSun's Nemotron EPQ v1 and v2 (linked above).
|
| 128 |
|
| 129 |
The GGUF weights remain hosted by Unsloth. This repository contains source, patch, scripts, logs and test notes only.
|