File size: 11,322 Bytes
378a1c2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
995bac2
378a1c2
 
 
1e2b8cd
378a1c2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1e2b8cd
88fa29f
 
 
 
 
 
 
 
 
 
 
378a1c2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
---

license: other
license_name: tencent-hy-license
license_link: https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE
base_model: tencent/Hy4-flash
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:
  - strata
  - hy4-flash
  - moe
  - quantization
  - latent-moe
  - engram
  - dfly2
---


<p>
<a href="https://github.com/Niko1221/Strata"><img class="dark:hidden" src="./assets/masthead-light.svg" width="100%" alt="Strata on GitHub"><img class="hidden dark:block" src="./assets/masthead-dark.svg" width="100%" alt="Strata on GitHub"></a>
</p>

<h1 align="center">Hy4-Flash for Strata</h1>

<p align="center"><b>Run Tencent's 220-billion-parameter Hy4-Flash on your own computer.</b> It starts at 64 GiB of RAM with an 8 GiB graphics card, with packs for 64, 96 and 128 GiB. Free and open source. No account needed.</p>

<p align="center">
  <img src="./assets/pagoda.gif" width="560" height="315" alt="5-step 1,300-voxel pagoda garden evolution">
</p>

<p align="center">
  <a href="#pick-your-pack"><b>Pick your pack</b></a> · <a href="#get-started">Get started</a> · <a href="#how-close-is-it-to-the-original">Quality</a> · <a href="#performance-with-strata">Speed</a> · <a href="#support-strata">Support Strata</a>
</p>

<p>
<img class="dark:hidden" src="./assets/hero-light.svg" width="100%" alt="220B parameters, 12B active per token. Runs from 64 GiB of RAM with an 8 GiB graphics card (GPU). Up to 1M tokens of context, with text and images in.">
<img class="hidden dark:block" src="./assets/hero-dark.svg" width="100%" alt="220B parameters, 12B active per token. Runs from 64 GiB of RAM with an 8 GiB graphics card (GPU). Up to 1M tokens of context, with text and images in.">
</p>

**Hy4 flash** is Tencent's most cost-effective Hy model to date. It has 220B parameters in total (64B of them in its Engram lookup) and uses just 12B per token. It reads text and images, writes text, and supports up to 1M tokens of context (1,048,576).

## Pick your pack

**The number in the name is your RAM.** Find your computer's memory and take the pack with the same number.

<p>
<img class="dark:hidden" src="./assets/packs-light.svg" width="100%" alt="Three packs. 64 GiB of RAM: SQ-64, with an 8, 12 or 16 GiB graphics card, 130 GB on disk, the smallest download. 96 GiB: SQ-96, with an 8 or 12 GiB graphics card, 157 GB. 128 GiB: SQ-128, with an 8 or 12 GiB graphics card, 181 GB, the closest to the original.">
<img class="hidden dark:block" src="./assets/packs-dark.svg" width="100%" alt="Three packs. 64 GiB of RAM: SQ-64, with an 8, 12 or 16 GiB graphics card, 130 GB on disk, the smallest download. 96 GiB: SQ-96, with an 8 or 12 GiB graphics card, 157 GB. 128 GiB: SQ-128, with an 8 or 12 GiB graphics card, 181 GB, the closest to the original.">
</p>

| Your RAM | Graphics card | Pack file | Memory in use | Total on disk |
| ---: | --- | --- | ---: | ---: |
| **64 GiB** | 8, 12 or 16 GiB | `Hy4-Flash-SQ-64.strata` | 59.4 GiB | 129.61 GB |
| **96 GiB** | 8 or 12 GiB | `Hy4-Flash-SQ-96.strata` | 85.1 GiB | 157.16 GB |
| **128 GiB** | 8 or 12 GiB | `Hy4-Flash-SQ-128.strata` | 107.5 GiB | 181.23 GB |

"Graphics card" is the card's memory. "Memory in use" is RAM plus graphics memory, approximately. "Total on disk" is the pack plus the shared lookup file every pack needs, `Hy4-Flash-Engram-Q8.strata`, for text. [File sizes and checksums](#files-and-checksums) are below.

- **64 GiB of RAM and an 8 GiB graphics card are enough to start.** Take SQ-64.
- **128 GiB of unified memory?** SQ-128 is the closest match to the original. SQ-96 leaves about 22 GiB free for other work.

## Get started

1. **Install Strata.** [Download the app for Windows, Linux or Mac](https://github.com/Niko1221/Strata/releases/latest). It needs no account.
2. **Choose Hy4-Flash** and the pack that matches your RAM.
3. **Start chatting.**

<details>
<summary><b>Prefer the terminal?</b></summary>

Linux or Mac:

```sh

curl -fsSL --connect-timeout 15 --max-time 120 https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh

```

Windows PowerShell:

```powershell

irm -TimeoutSec 120 https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex

```

On Windows, open a new terminal afterwards. Then run `strata launch` (or `strata launch --family hy4-flash --model SQ-64`) and follow the prompts.

</details>

## How close is it to the original?

**SQ-128 stays closest to the original model on both test sets.** Each smaller pack gives up some closeness to fit in less memory.

<p>
<img class="dark:hidden" src="./assets/closeness-light.svg" width="100%" alt="Drift from the original model, mean KL divergence, lower is better. Reasoning prompts: SQ-128 0.088, SQ-96 0.136, SQ-64 0.232. Tool-calling prompts: SQ-128 0.180, SQ-96 0.237, SQ-64 0.518. SQ-128 is the closest on both.">
<img class="hidden dark:block" src="./assets/closeness-dark.svg" width="100%" alt="Drift from the original model, mean KL divergence, lower is better. Reasoning prompts: SQ-128 0.088, SQ-96 0.136, SQ-64 0.232. Tool-calling prompts: SQ-128 0.180, SQ-96 0.237, SQ-64 0.518. SQ-128 is the closest on both.">
</p>

**How to read this.** Each pack's next-word predictions are compared with the full-precision model (BF16). The score is the average gap between the two, called KL divergence. Zero means identical, so a shorter bar is better.

| Pack | Reasoning prompts | Tool-calling prompts |
| --- | ---: | ---: |
| **SQ-128** | **0.08789** | **0.17969** |
| SQ-96 | 0.13591 | 0.23661 |
| SQ-64 | 0.23206 | 0.51828 |

<details>
<summary>How this was measured</summary>

Each set contains eight held-out prompts of 512 tokens. The evaluation scores all 4,096 prompt positions using the packed model and its required Engram file across all 42 layers. The tool-calling column measures token distributions on tool-related prompts. End-to-end tool success requires a separate evaluation.

</details>

## Performance with Strata

**Up to 240 tokens a second.** Speeds measured with Strata on eight machines, from two V100 cards to an RTX PRO 6000.

<p>
<img class="dark:hidden" src="./assets/speed-light.svg" width="100%" alt="Output speed with Strata, in tokens per second, with prompt speed beside it. RTX PRO 6000: 102.8 to 240, prompt 4,015. RTX 5090: 90 to 111, prompt 1,694 at 128K context. M5 Max with Metal 4: 69 to 110, prompt 1,215. R9700 or RX 9070 XT: 62, prompt 1,122. Strix Halo 128 GB: 51 to 60, prompt 728. RTX 5080 16 GB: 44, prompt 1,627. DGX Spark GB10: 43.9, prompt 924. Two V100 cards: 41.8, prompt 1,068.">
<img class="hidden dark:block" src="./assets/speed-dark.svg" width="100%" alt="Output speed with Strata, in tokens per second, with prompt speed beside it. RTX PRO 6000: 102.8 to 240, prompt 4,015. RTX 5090: 90 to 111, prompt 1,694 at 128K context. M5 Max with Metal 4: 69 to 110, prompt 1,215. R9700 or RX 9070 XT: 62, prompt 1,122. Strix Halo 128 GB: 51 to 60, prompt 728. RTX 5080 16 GB: 44, prompt 1,627. DGX Spark GB10: 43.9, prompt 924. Two V100 cards: 41.8, prompt 1,068.">
</p>

<details>
<summary>How speeds were measured</summary>

Measured with Strata 0.2.18 (`strata-hy4` 0.1.64, built-in `DFly2` speculative decoding on): `SQ-96` and `SQ-64` on RTX PRO 6000 (4K–104K context); `SQ-64` on RTX 5090 (128K context), Apple M5 Max 128 GB (Metal 4, 4K–32K context), Radeon AI PRO R9700 / RX 9070 XT (4K context), Ryzen AI Max+ 395 Strix Halo 128 GB (4K–128K context), RTX 5080 16 GB + 64 GB RAM (4K context), DGX Spark GB10 128 GB (4K context), and 2× Tesla V100 32 GB (4K context).

</details>

## Recommended settings

| Setting | Value |
| --- | --- |
| `temperature` | 0.9 |
| `top_p` | 1.0 |
| `top_k` | off |

Use the same values with thinking on (`reasoning_effort` high, the default) and for direct answers (`no_think`).

## Files and checksums

| File | Size |
| --- | ---: |
| `Hy4-Flash-SQ-64.strata` | 62.03 GB |
| `Hy4-Flash-SQ-96.strata` | 89.58 GB |
| `Hy4-Flash-SQ-128.strata` | 113.65 GB |
| `Hy4-Flash-Engram-Q8.strata`, needed by every pack | 67.58 GB |
| `Hy4-Flash-Vision.strata`, for reading images | 0.93 GB |

<details>
<summary>Pack file checksums (SHA-256)</summary>

```text

6e8ba2729465f38ed42b5bfa7db02c92beddc25ca0413b7f0c24ab9f1e8b51f4  Hy4-Flash-SQ-128.strata

3b1c5755a220c69892b3d2dbad506b8593a30be24b17a390ecbd1642c0c1e14c  Hy4-Flash-SQ-96.strata

b2cc38fe4bded35df82a701f9e868ea09c6fcd9be52c8446289e394f804417b2  Hy4-Flash-SQ-64.strata

ab0cf9b3f9c64bd25f6212d4508e7443ab8b9bc133fbccc7bdfb83510a20c2c1  Hy4-Flash-Engram-Q8.strata

ef09bceff8f01b8d226d2dc56aef09cec719b21b45783ee9b5d64e7241b72a09  Hy4-Flash-Vision.strata

```

</details>

## Support Strata

<p align="center">
  <b>Help keep local AI fast, independent, and open to everyone.</b><br>
  <sub>Strata is built independently and released free under the MIT license. Sponsoring on GitHub or buying a coffee directly funds new GPU kernels, model quantizations, and hardware testing.</sub>
</p>

<p align="center">
  <a href="https://github.com/sponsors/Niko1221"><img src="./assets/btn-sponsor.svg" alt="Sponsor Strata on GitHub" height="44"></a>&nbsp;&nbsp;
  <a href="https://buymeacoffee.com/strataengine"><img src="./assets/btn-coffee.svg" alt="Buy Strata a coffee" height="44"></a>
</p>

## Licence

- **The model:** Hy4-Flash is made by Tencent and released under the [Tencent Hy License](https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE). The files here are a quantized build of [tencent/Hy4-flash](https://huggingface.co/tencent/Hy4-flash).
- **The Strata engine:** [MIT](https://github.com/Niko1221/Strata/blob/main/LICENSE).

<!-- KL chart data
Edit the mean KL values below to update the chart.

| Pack | Reasoning mean KL | Tool-calling mean KL |
| --- | ---: | ---: |
| SQ-128 | 0.08789 | 0.17969 |
| SQ-96 | 0.13591 | 0.23661 |
| SQ-64 | 0.23206 | 0.51828 |
| BF16 reference | 0 | 0 |
| AngelSlim Q4_K_M | Unmeasured | Unmeasured |

Each split contains eight held-out prompts of 512 tokens. The evaluation scores all 4,096 prompt positions using the packed model and its required Engram file across all 42 layers. The tool-calling column measures token distributions on tool-related prompts. End-to-end tool success requires a separate evaluation.

| Pack | Reasoning perplexity | Tool-calling perplexity | Low-entropy mean KL |
| --- | ---: | ---: | ---: |
| BF16 reference | 5.263 | 15.990 | 0 |
| SQ-128 | 5.204 | 15.062 | 0.0477 |
| SQ-96 | 5.309 | 15.478 | 0.0660 |
| SQ-64 | 3.854 | 7.614 | 0.1164* |

\* The SQ-64 low-entropy result predates its final tuning pass. BF16 perplexity on that sequence was 3.06.

-->

<!-- Speed chart data
Edit the figures below, then run tools/make_charts.py to update the speed chart.



| Machine | Prompt speed | Output speed |

| --- | ---: | ---: |

| RTX PRO 6000 | 4,015 | 102.8–240 |

| RTX 5090 | 1,694 (at 128K context) | 90–111 |

| M5 Max (Metal 4) | 1,215 | 69–110 |

| R9700 / RX 9070 XT | 1,122 | 62 |

| Strix Halo 128 GB | 728 | 51–60 |

| RTX 5080 16 GB | 1,627 | 44 |

| DGX Spark (GB10) | 924 | 43.9 |

| 2x V100 | 1,068 | 41.8 |



-->