|
Download README.md from strata-engine/Hy4-Flash-Strata: direct link, hf CLI and curl.
- Browser
- Download file 11.3 kB
-
https://huggingface.co/strata-engine/Hy4-Flash-Strata/resolve/main/README.md
- Command line
-
hf download hf://strata-engine/Hy4-Flash-Strata/README.md
-
curl -L -o README.md https://huggingface.co/strata-engine/Hy4-Flash-Strata/resolve/main/README.md
11.3 kB
| license: other | |
| license_name: tencent-hy-license | |
| license_link: https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE | |
| base_model: tencent/Hy4-flash | |
| base_model_relation: quantized | |
| pipeline_tag: image-text-to-text | |
| tags: | |
| - strata | |
| - hy4-flash | |
| - moe | |
| - quantization | |
| - latent-moe | |
| - engram | |
| - dfly2 | |
| <p> | |
| <a href="https://github.com/Niko1221/Strata"><img class="dark:hidden" src="./assets/masthead-light.svg" width="100%" alt="Strata on GitHub"><img class="hidden dark:block" src="./assets/masthead-dark.svg" width="100%" alt="Strata on GitHub"></a> | |
| </p> | |
| <h1 align="center">Hy4-Flash for Strata</h1> | |
| <p align="center"><b>Run Tencent's 220-billion-parameter Hy4-Flash on your own computer.</b> It starts at 64 GiB of RAM with an 8 GiB graphics card, with packs for 64, 96 and 128 GiB. Free and open source. No account needed.</p> | |
| <p align="center"> | |
| <img src="./assets/pagoda.gif" width="560" height="315" alt="5-step 1,300-voxel pagoda garden evolution"> | |
| </p> | |
| <p align="center"> | |
| <a href="#pick-your-pack"><b>Pick your pack</b></a> · <a href="#get-started">Get started</a> · <a href="#how-close-is-it-to-the-original">Quality</a> · <a href="#performance-with-strata">Speed</a> · <a href="#support-strata">Support Strata</a> | |
| </p> | |
| <p> | |
| <img class="dark:hidden" src="./assets/hero-light.svg" width="100%" alt="220B parameters, 12B active per token. Runs from 64 GiB of RAM with an 8 GiB graphics card (GPU). Up to 1M tokens of context, with text and images in."> | |
| <img class="hidden dark:block" src="./assets/hero-dark.svg" width="100%" alt="220B parameters, 12B active per token. Runs from 64 GiB of RAM with an 8 GiB graphics card (GPU). Up to 1M tokens of context, with text and images in."> | |
| </p> | |
| **Hy4 flash** is Tencent's most cost-effective Hy model to date. It has 220B parameters in total (64B of them in its Engram lookup) and uses just 12B per token. It reads text and images, writes text, and supports up to 1M tokens of context (1,048,576). | |
| ## Pick your pack | |
| **The number in the name is your RAM.** Find your computer's memory and take the pack with the same number. | |
| <p> | |
| <img class="dark:hidden" src="./assets/packs-light.svg" width="100%" alt="Three packs. 64 GiB of RAM: SQ-64, with an 8, 12 or 16 GiB graphics card, 130 GB on disk, the smallest download. 96 GiB: SQ-96, with an 8 or 12 GiB graphics card, 157 GB. 128 GiB: SQ-128, with an 8 or 12 GiB graphics card, 181 GB, the closest to the original."> | |
| <img class="hidden dark:block" src="./assets/packs-dark.svg" width="100%" alt="Three packs. 64 GiB of RAM: SQ-64, with an 8, 12 or 16 GiB graphics card, 130 GB on disk, the smallest download. 96 GiB: SQ-96, with an 8 or 12 GiB graphics card, 157 GB. 128 GiB: SQ-128, with an 8 or 12 GiB graphics card, 181 GB, the closest to the original."> | |
| </p> | |
| | Your RAM | Graphics card | Pack file | Memory in use | Total on disk | | |
| | ---: | --- | --- | ---: | ---: | | |
| | **64 GiB** | 8, 12 or 16 GiB | `Hy4-Flash-SQ-64.strata` | 59.4 GiB | 129.61 GB | | |
| | **96 GiB** | 8 or 12 GiB | `Hy4-Flash-SQ-96.strata` | 85.1 GiB | 157.16 GB | | |
| | **128 GiB** | 8 or 12 GiB | `Hy4-Flash-SQ-128.strata` | 107.5 GiB | 181.23 GB | | |
| "Graphics card" is the card's memory. "Memory in use" is RAM plus graphics memory, approximately. "Total on disk" is the pack plus the shared lookup file every pack needs, `Hy4-Flash-Engram-Q8.strata`, for text. [File sizes and checksums](#files-and-checksums) are below. | |
| - **64 GiB of RAM and an 8 GiB graphics card are enough to start.** Take SQ-64. | |
| - **128 GiB of unified memory?** SQ-128 is the closest match to the original. SQ-96 leaves about 22 GiB free for other work. | |
| ## Get started | |
| 1. **Install Strata.** [Download the app for Windows, Linux or Mac](https://github.com/Niko1221/Strata/releases/latest). It needs no account. | |
| 2. **Choose Hy4-Flash** and the pack that matches your RAM. | |
| 3. **Start chatting.** | |
| <details> | |
| <summary><b>Prefer the terminal?</b></summary> | |
| Linux or Mac: | |
| ```sh | |
| curl -fsSL --connect-timeout 15 --max-time 120 https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh | |
| ``` | |
| Windows PowerShell: | |
| ```powershell | |
| irm -TimeoutSec 120 https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex | |
| ``` | |
| On Windows, open a new terminal afterwards. Then run `strata launch` (or `strata launch --family hy4-flash --model SQ-64`) and follow the prompts. | |
| </details> | |
| ## How close is it to the original? | |
| **SQ-128 stays closest to the original model on both test sets.** Each smaller pack gives up some closeness to fit in less memory. | |
| <p> | |
| <img class="dark:hidden" src="./assets/closeness-light.svg" width="100%" alt="Drift from the original model, mean KL divergence, lower is better. Reasoning prompts: SQ-128 0.088, SQ-96 0.136, SQ-64 0.232. Tool-calling prompts: SQ-128 0.180, SQ-96 0.237, SQ-64 0.518. SQ-128 is the closest on both."> | |
| <img class="hidden dark:block" src="./assets/closeness-dark.svg" width="100%" alt="Drift from the original model, mean KL divergence, lower is better. Reasoning prompts: SQ-128 0.088, SQ-96 0.136, SQ-64 0.232. Tool-calling prompts: SQ-128 0.180, SQ-96 0.237, SQ-64 0.518. SQ-128 is the closest on both."> | |
| </p> | |
| **How to read this.** Each pack's next-word predictions are compared with the full-precision model (BF16). The score is the average gap between the two, called KL divergence. Zero means identical, so a shorter bar is better. | |
| | Pack | Reasoning prompts | Tool-calling prompts | | |
| | --- | ---: | ---: | | |
| | **SQ-128** | **0.08789** | **0.17969** | | |
| | SQ-96 | 0.13591 | 0.23661 | | |
| | SQ-64 | 0.23206 | 0.51828 | | |
| <details> | |
| <summary>How this was measured</summary> | |
| Each set contains eight held-out prompts of 512 tokens. The evaluation scores all 4,096 prompt positions using the packed model and its required Engram file across all 42 layers. The tool-calling column measures token distributions on tool-related prompts. End-to-end tool success requires a separate evaluation. | |
| </details> | |
| ## Performance with Strata | |
| **Up to 240 tokens a second.** Speeds measured with Strata on eight machines, from two V100 cards to an RTX PRO 6000. | |
| <p> | |
| <img class="dark:hidden" src="./assets/speed-light.svg" width="100%" alt="Output speed with Strata, in tokens per second, with prompt speed beside it. RTX PRO 6000: 102.8 to 240, prompt 4,015. RTX 5090: 90 to 111, prompt 1,694 at 128K context. M5 Max with Metal 4: 69 to 110, prompt 1,215. R9700 or RX 9070 XT: 62, prompt 1,122. Strix Halo 128 GB: 51 to 60, prompt 728. RTX 5080 16 GB: 44, prompt 1,627. DGX Spark GB10: 43.9, prompt 924. Two V100 cards: 41.8, prompt 1,068."> | |
| <img class="hidden dark:block" src="./assets/speed-dark.svg" width="100%" alt="Output speed with Strata, in tokens per second, with prompt speed beside it. RTX PRO 6000: 102.8 to 240, prompt 4,015. RTX 5090: 90 to 111, prompt 1,694 at 128K context. M5 Max with Metal 4: 69 to 110, prompt 1,215. R9700 or RX 9070 XT: 62, prompt 1,122. Strix Halo 128 GB: 51 to 60, prompt 728. RTX 5080 16 GB: 44, prompt 1,627. DGX Spark GB10: 43.9, prompt 924. Two V100 cards: 41.8, prompt 1,068."> | |
| </p> | |
| <details> | |
| <summary>How speeds were measured</summary> | |
| Measured with Strata 0.2.18 (`strata-hy4` 0.1.64, built-in `DFly2` speculative decoding on): `SQ-96` and `SQ-64` on RTX PRO 6000 (4K–104K context); `SQ-64` on RTX 5090 (128K context), Apple M5 Max 128 GB (Metal 4, 4K–32K context), Radeon AI PRO R9700 / RX 9070 XT (4K context), Ryzen AI Max+ 395 Strix Halo 128 GB (4K–128K context), RTX 5080 16 GB + 64 GB RAM (4K context), DGX Spark GB10 128 GB (4K context), and 2× Tesla V100 32 GB (4K context). | |
| </details> | |
| ## Recommended settings | |
| | Setting | Value | | |
| | --- | --- | | |
| | `temperature` | 0.9 | | |
| | `top_p` | 1.0 | | |
| | `top_k` | off | | |
| Use the same values with thinking on (`reasoning_effort` high, the default) and for direct answers (`no_think`). | |
| ## Files and checksums | |
| | File | Size | | |
| | --- | ---: | | |
| | `Hy4-Flash-SQ-64.strata` | 62.03 GB | | |
| | `Hy4-Flash-SQ-96.strata` | 89.58 GB | | |
| | `Hy4-Flash-SQ-128.strata` | 113.65 GB | | |
| | `Hy4-Flash-Engram-Q8.strata`, needed by every pack | 67.58 GB | | |
| | `Hy4-Flash-Vision.strata`, for reading images | 0.93 GB | | |
| <details> | |
| <summary>Pack file checksums (SHA-256)</summary> | |
| ```text | |
| 6e8ba2729465f38ed42b5bfa7db02c92beddc25ca0413b7f0c24ab9f1e8b51f4 Hy4-Flash-SQ-128.strata | |
| 3b1c5755a220c69892b3d2dbad506b8593a30be24b17a390ecbd1642c0c1e14c Hy4-Flash-SQ-96.strata | |
| b2cc38fe4bded35df82a701f9e868ea09c6fcd9be52c8446289e394f804417b2 Hy4-Flash-SQ-64.strata | |
| ab0cf9b3f9c64bd25f6212d4508e7443ab8b9bc133fbccc7bdfb83510a20c2c1 Hy4-Flash-Engram-Q8.strata | |
| ef09bceff8f01b8d226d2dc56aef09cec719b21b45783ee9b5d64e7241b72a09 Hy4-Flash-Vision.strata | |
| ``` | |
| </details> | |
| ## Support Strata | |
| <p align="center"> | |
| <b>Help keep local AI fast, independent, and open to everyone.</b><br> | |
| <sub>Strata is built independently and released free under the MIT license. Sponsoring on GitHub or buying a coffee directly funds new GPU kernels, model quantizations, and hardware testing.</sub> | |
| </p> | |
| <p align="center"> | |
| <a href="https://github.com/sponsors/Niko1221"><img src="./assets/btn-sponsor.svg" alt="Sponsor Strata on GitHub" height="44"></a> | |
| <a href="https://buymeacoffee.com/strataengine"><img src="./assets/btn-coffee.svg" alt="Buy Strata a coffee" height="44"></a> | |
| </p> | |
| ## Licence | |
| - **The model:** Hy4-Flash is made by Tencent and released under the [Tencent Hy License](https://huggingface.co/tencent/Hy4-flash/blob/main/LICENSE). The files here are a quantized build of [tencent/Hy4-flash](https://huggingface.co/tencent/Hy4-flash). | |
| - **The Strata engine:** [MIT](https://github.com/Niko1221/Strata/blob/main/LICENSE). | |
| <!-- KL chart data | |
| Edit the mean KL values below to update the chart. | |
| | Pack | Reasoning mean KL | Tool-calling mean KL | | |
| | --- | ---: | ---: | | |
| | SQ-128 | 0.08789 | 0.17969 | | |
| | SQ-96 | 0.13591 | 0.23661 | | |
| | SQ-64 | 0.23206 | 0.51828 | | |
| | BF16 reference | 0 | 0 | | |
| | AngelSlim Q4_K_M | Unmeasured | Unmeasured | | |
| Each split contains eight held-out prompts of 512 tokens. The evaluation scores all 4,096 prompt positions using the packed model and its required Engram file across all 42 layers. The tool-calling column measures token distributions on tool-related prompts. End-to-end tool success requires a separate evaluation. | |
| | Pack | Reasoning perplexity | Tool-calling perplexity | Low-entropy mean KL | | |
| | --- | ---: | ---: | ---: | | |
| | BF16 reference | 5.263 | 15.990 | 0 | | |
| | SQ-128 | 5.204 | 15.062 | 0.0477 | | |
| | SQ-96 | 5.309 | 15.478 | 0.0660 | | |
| | SQ-64 | 3.854 | 7.614 | 0.1164* | | |
| \* The SQ-64 low-entropy result predates its final tuning pass. BF16 perplexity on that sequence was 3.06. | |
| --> | |
| <!-- Speed chart data | |
| Edit the figures below, then run tools/make_charts.py to update the speed chart. | |
| | Machine | Prompt speed | Output speed | | |
| | --- | ---: | ---: | | |
| | RTX PRO 6000 | 4,015 | 102.8–240 | | |
| | RTX 5090 | 1,694 (at 128K context) | 90–111 | | |
| | M5 Max (Metal 4) | 1,215 | 69–110 | | |
| | R9700 / RX 9070 XT | 1,122 | 62 | | |
| | Strix Halo 128 GB | 728 | 51–60 | | |
| | RTX 5080 16 GB | 1,627 | 44 | | |
| | DGX Spark (GB10) | 924 | 43.9 | | |
| | 2x V100 | 1,068 | 41.8 | | |
| --> | |