File size: 11,322 Bytes
378a1c2 995bac2 378a1c2 1e2b8cd 378a1c2 1e2b8cd 88fa29f 378a1c2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 | ---
license: other
license_name: tencent-hy-license
license_link: https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE
base_model: tencent/Hy4-flash
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:
- strata
- hy4-flash
- moe
- quantization
- latent-moe
- engram
- dfly2
---
<p>
<a href="https://github.com/Niko1221/Strata"><img class="dark:hidden" src="./assets/masthead-light.svg" width="100%" alt="Strata on GitHub"><img class="hidden dark:block" src="./assets/masthead-dark.svg" width="100%" alt="Strata on GitHub"></a>
</p>
<h1 align="center">Hy4-Flash for Strata</h1>
<p align="center"><b>Run Tencent's 220-billion-parameter Hy4-Flash on your own computer.</b> It starts at 64 GiB of RAM with an 8 GiB graphics card, with packs for 64, 96 and 128 GiB. Free and open source. No account needed.</p>
<p align="center">
<img src="./assets/pagoda.gif" width="560" height="315" alt="5-step 1,300-voxel pagoda garden evolution">
</p>
<p align="center">
<a href="#pick-your-pack"><b>Pick your pack</b></a> · <a href="#get-started">Get started</a> · <a href="#how-close-is-it-to-the-original">Quality</a> · <a href="#performance-with-strata">Speed</a> · <a href="#support-strata">Support Strata</a>
</p>
<p>
<img class="dark:hidden" src="./assets/hero-light.svg" width="100%" alt="220B parameters, 12B active per token. Runs from 64 GiB of RAM with an 8 GiB graphics card (GPU). Up to 1M tokens of context, with text and images in.">
<img class="hidden dark:block" src="./assets/hero-dark.svg" width="100%" alt="220B parameters, 12B active per token. Runs from 64 GiB of RAM with an 8 GiB graphics card (GPU). Up to 1M tokens of context, with text and images in.">
</p>
**Hy4 flash** is Tencent's most cost-effective Hy model to date. It has 220B parameters in total (64B of them in its Engram lookup) and uses just 12B per token. It reads text and images, writes text, and supports up to 1M tokens of context (1,048,576).
## Pick your pack
**The number in the name is your RAM.** Find your computer's memory and take the pack with the same number.
<p>
<img class="dark:hidden" src="./assets/packs-light.svg" width="100%" alt="Three packs. 64 GiB of RAM: SQ-64, with an 8, 12 or 16 GiB graphics card, 130 GB on disk, the smallest download. 96 GiB: SQ-96, with an 8 or 12 GiB graphics card, 157 GB. 128 GiB: SQ-128, with an 8 or 12 GiB graphics card, 181 GB, the closest to the original.">
<img class="hidden dark:block" src="./assets/packs-dark.svg" width="100%" alt="Three packs. 64 GiB of RAM: SQ-64, with an 8, 12 or 16 GiB graphics card, 130 GB on disk, the smallest download. 96 GiB: SQ-96, with an 8 or 12 GiB graphics card, 157 GB. 128 GiB: SQ-128, with an 8 or 12 GiB graphics card, 181 GB, the closest to the original.">
</p>
| Your RAM | Graphics card | Pack file | Memory in use | Total on disk |
| ---: | --- | --- | ---: | ---: |
| **64 GiB** | 8, 12 or 16 GiB | `Hy4-Flash-SQ-64.strata` | 59.4 GiB | 129.61 GB |
| **96 GiB** | 8 or 12 GiB | `Hy4-Flash-SQ-96.strata` | 85.1 GiB | 157.16 GB |
| **128 GiB** | 8 or 12 GiB | `Hy4-Flash-SQ-128.strata` | 107.5 GiB | 181.23 GB |
"Graphics card" is the card's memory. "Memory in use" is RAM plus graphics memory, approximately. "Total on disk" is the pack plus the shared lookup file every pack needs, `Hy4-Flash-Engram-Q8.strata`, for text. [File sizes and checksums](#files-and-checksums) are below.
- **64 GiB of RAM and an 8 GiB graphics card are enough to start.** Take SQ-64.
- **128 GiB of unified memory?** SQ-128 is the closest match to the original. SQ-96 leaves about 22 GiB free for other work.
## Get started
1. **Install Strata.** [Download the app for Windows, Linux or Mac](https://github.com/Niko1221/Strata/releases/latest). It needs no account.
2. **Choose Hy4-Flash** and the pack that matches your RAM.
3. **Start chatting.**
<details>
<summary><b>Prefer the terminal?</b></summary>
Linux or Mac:
```sh
curl -fsSL --connect-timeout 15 --max-time 120 https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows PowerShell:
```powershell
irm -TimeoutSec 120 https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
On Windows, open a new terminal afterwards. Then run `strata launch` (or `strata launch --family hy4-flash --model SQ-64`) and follow the prompts.
</details>
## How close is it to the original?
**SQ-128 stays closest to the original model on both test sets.** Each smaller pack gives up some closeness to fit in less memory.
<p>
<img class="dark:hidden" src="./assets/closeness-light.svg" width="100%" alt="Drift from the original model, mean KL divergence, lower is better. Reasoning prompts: SQ-128 0.088, SQ-96 0.136, SQ-64 0.232. Tool-calling prompts: SQ-128 0.180, SQ-96 0.237, SQ-64 0.518. SQ-128 is the closest on both.">
<img class="hidden dark:block" src="./assets/closeness-dark.svg" width="100%" alt="Drift from the original model, mean KL divergence, lower is better. Reasoning prompts: SQ-128 0.088, SQ-96 0.136, SQ-64 0.232. Tool-calling prompts: SQ-128 0.180, SQ-96 0.237, SQ-64 0.518. SQ-128 is the closest on both.">
</p>
**How to read this.** Each pack's next-word predictions are compared with the full-precision model (BF16). The score is the average gap between the two, called KL divergence. Zero means identical, so a shorter bar is better.
| Pack | Reasoning prompts | Tool-calling prompts |
| --- | ---: | ---: |
| **SQ-128** | **0.08789** | **0.17969** |
| SQ-96 | 0.13591 | 0.23661 |
| SQ-64 | 0.23206 | 0.51828 |
<details>
<summary>How this was measured</summary>
Each set contains eight held-out prompts of 512 tokens. The evaluation scores all 4,096 prompt positions using the packed model and its required Engram file across all 42 layers. The tool-calling column measures token distributions on tool-related prompts. End-to-end tool success requires a separate evaluation.
</details>
## Performance with Strata
**Up to 240 tokens a second.** Speeds measured with Strata on eight machines, from two V100 cards to an RTX PRO 6000.
<p>
<img class="dark:hidden" src="./assets/speed-light.svg" width="100%" alt="Output speed with Strata, in tokens per second, with prompt speed beside it. RTX PRO 6000: 102.8 to 240, prompt 4,015. RTX 5090: 90 to 111, prompt 1,694 at 128K context. M5 Max with Metal 4: 69 to 110, prompt 1,215. R9700 or RX 9070 XT: 62, prompt 1,122. Strix Halo 128 GB: 51 to 60, prompt 728. RTX 5080 16 GB: 44, prompt 1,627. DGX Spark GB10: 43.9, prompt 924. Two V100 cards: 41.8, prompt 1,068.">
<img class="hidden dark:block" src="./assets/speed-dark.svg" width="100%" alt="Output speed with Strata, in tokens per second, with prompt speed beside it. RTX PRO 6000: 102.8 to 240, prompt 4,015. RTX 5090: 90 to 111, prompt 1,694 at 128K context. M5 Max with Metal 4: 69 to 110, prompt 1,215. R9700 or RX 9070 XT: 62, prompt 1,122. Strix Halo 128 GB: 51 to 60, prompt 728. RTX 5080 16 GB: 44, prompt 1,627. DGX Spark GB10: 43.9, prompt 924. Two V100 cards: 41.8, prompt 1,068.">
</p>
<details>
<summary>How speeds were measured</summary>
Measured with Strata 0.2.18 (`strata-hy4` 0.1.64, built-in `DFly2` speculative decoding on): `SQ-96` and `SQ-64` on RTX PRO 6000 (4K–104K context); `SQ-64` on RTX 5090 (128K context), Apple M5 Max 128 GB (Metal 4, 4K–32K context), Radeon AI PRO R9700 / RX 9070 XT (4K context), Ryzen AI Max+ 395 Strix Halo 128 GB (4K–128K context), RTX 5080 16 GB + 64 GB RAM (4K context), DGX Spark GB10 128 GB (4K context), and 2× Tesla V100 32 GB (4K context).
</details>
## Recommended settings
| Setting | Value |
| --- | --- |
| `temperature` | 0.9 |
| `top_p` | 1.0 |
| `top_k` | off |
Use the same values with thinking on (`reasoning_effort` high, the default) and for direct answers (`no_think`).
## Files and checksums
| File | Size |
| --- | ---: |
| `Hy4-Flash-SQ-64.strata` | 62.03 GB |
| `Hy4-Flash-SQ-96.strata` | 89.58 GB |
| `Hy4-Flash-SQ-128.strata` | 113.65 GB |
| `Hy4-Flash-Engram-Q8.strata`, needed by every pack | 67.58 GB |
| `Hy4-Flash-Vision.strata`, for reading images | 0.93 GB |
<details>
<summary>Pack file checksums (SHA-256)</summary>
```text
6e8ba2729465f38ed42b5bfa7db02c92beddc25ca0413b7f0c24ab9f1e8b51f4 Hy4-Flash-SQ-128.strata
3b1c5755a220c69892b3d2dbad506b8593a30be24b17a390ecbd1642c0c1e14c Hy4-Flash-SQ-96.strata
b2cc38fe4bded35df82a701f9e868ea09c6fcd9be52c8446289e394f804417b2 Hy4-Flash-SQ-64.strata
ab0cf9b3f9c64bd25f6212d4508e7443ab8b9bc133fbccc7bdfb83510a20c2c1 Hy4-Flash-Engram-Q8.strata
ef09bceff8f01b8d226d2dc56aef09cec719b21b45783ee9b5d64e7241b72a09 Hy4-Flash-Vision.strata
```
</details>
## Support Strata
<p align="center">
<b>Help keep local AI fast, independent, and open to everyone.</b><br>
<sub>Strata is built independently and released free under the MIT license. Sponsoring on GitHub or buying a coffee directly funds new GPU kernels, model quantizations, and hardware testing.</sub>
</p>
<p align="center">
<a href="https://github.com/sponsors/Niko1221"><img src="./assets/btn-sponsor.svg" alt="Sponsor Strata on GitHub" height="44"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="./assets/btn-coffee.svg" alt="Buy Strata a coffee" height="44"></a>
</p>
## Licence
- **The model:** Hy4-Flash is made by Tencent and released under the [Tencent Hy License](https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE). The files here are a quantized build of [tencent/Hy4-flash](https://huggingface.co/tencent/Hy4-flash).
- **The Strata engine:** [MIT](https://github.com/Niko1221/Strata/blob/main/LICENSE).
<!-- KL chart data
Edit the mean KL values below to update the chart.
| Pack | Reasoning mean KL | Tool-calling mean KL |
| --- | ---: | ---: |
| SQ-128 | 0.08789 | 0.17969 |
| SQ-96 | 0.13591 | 0.23661 |
| SQ-64 | 0.23206 | 0.51828 |
| BF16 reference | 0 | 0 |
| AngelSlim Q4_K_M | Unmeasured | Unmeasured |
Each split contains eight held-out prompts of 512 tokens. The evaluation scores all 4,096 prompt positions using the packed model and its required Engram file across all 42 layers. The tool-calling column measures token distributions on tool-related prompts. End-to-end tool success requires a separate evaluation.
| Pack | Reasoning perplexity | Tool-calling perplexity | Low-entropy mean KL |
| --- | ---: | ---: | ---: |
| BF16 reference | 5.263 | 15.990 | 0 |
| SQ-128 | 5.204 | 15.062 | 0.0477 |
| SQ-96 | 5.309 | 15.478 | 0.0660 |
| SQ-64 | 3.854 | 7.614 | 0.1164* |
\* The SQ-64 low-entropy result predates its final tuning pass. BF16 perplexity on that sequence was 3.06.
-->
<!-- Speed chart data
Edit the figures below, then run tools/make_charts.py to update the speed chart.
| Machine | Prompt speed | Output speed |
| --- | ---: | ---: |
| RTX PRO 6000 | 4,015 | 102.8–240 |
| RTX 5090 | 1,694 (at 128K context) | 90–111 |
| M5 Max (Metal 4) | 1,215 | 69–110 |
| R9700 / RX 9070 XT | 1,122 | 62 |
| Strix Halo 128 GB | 728 | 51–60 |
| RTX 5080 16 GB | 1,627 | 44 |
| DGX Spark (GB10) | 924 | 43.9 |
| 2x V100 | 1,068 | 41.8 |
-->
|