ashxhart's picture
Clarify runtime status and checkpoint sizes
dd375b3 verified
|
Raw History Blame Contribute Delete
2.98 kB
---
license: other
license_name: tencent-hy-license
license_link: https://huggingface.co/tencent/Hy4-flash/blob/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959/LICENSE
base_model: tencent/Hy4-flash
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:
- mlx
- mtp
- dfly2
- speculative-decoding
- hunyuan
- hy4-flash
- mixture-of-experts
- vision-language
- quantized
- apple-silicon
- 4-bit
---
# Hy4-flash MLX 4-bit with DFly2
> **Preview checkpoint.** Weight conversion checks passed. TensorFold 1.2.9 rejects this architecture at startup. Generation, DFly2 parity, and runtime memory requirements are pending validation.
This is TensorFold’s MLX conversion of [Tencent Hy4-flash](https://huggingface.co/tencent/Hy4-flash/tree/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959), a multimodal model that takes text and images. Both variants include Tencent’s DFly2 drafter at source precision.
<p align="center">
<img src="assets/logo-en.png" alt="Tencent Hy" width="280" />
</p>
## Choose a checkpoint
| Checkpoint | Weight files |
| --- | ---: |
| **This repo · MLX 4-bit** | 129.86 GB |
| [MLX 8-bit](https://huggingface.co/TensorFold/Hy4-flash-MLX-8bit-MTP) | 239.30 GB |
These sizes are for stored weights. Runtime memory depends on the engine, context length, cache settings, and concurrent requests. Check the [TensorFold releases](https://github.com/ashhart/TensorFold/releases) for runtime support before downloading.
## What’s included
| Component | Format |
| --- | --- |
| Target weights | MLX affine 4-bit safetensors |
| Quantization group size | 64 for most weights and 32 for Engram embeddings |
| DFly2 drafter | Upstream weights and configuration at source precision |
| Vision tower and MoE router gates | Source precision |
The repo’s MTP suffix identifies the variant that bundles the DFly2 drafter. DFly2 requires the model’s sparse-attention index and KV cache.
## Architecture
Tencent describes Hy4-flash as a multimodal mixture-of-experts model with 220 billion total parameters and about 12 billion active parameters per token. The language stack combines sliding-window attention, sparse attention and Engram embeddings. The upstream DFly2 drafter has five layers that reuse the target model's KV cache and sparse-attention positions.
## Conversion and validation
The conversion uses Tencent's pinned BF16 checkpoint and retains the tokenizer, processor, chat template and DFly2 configuration. The 148 output shards match the tensor index, sampled packed weights match MLX affine quantization, and all 87 DFly2 tensors match the upstream bytes. Generation and drafted-versus-plain parity are pending runtime support.
## Licence and attribution
The original model, DFly2 drafter and evaluations are the work of Tencent and the upstream contributors. TensorFold prepares the MLX conversion. The [Tencent Hy License](https://huggingface.co/tencent/Hy4-flash/blob/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959/LICENSE) applies.