ashxhart commited on
Commit
587d3b4
·
verified ·
1 Parent(s): 4d989ef

Draft Hy4-flash model card

Browse files
Files changed (1) hide show
  1. README.md +66 -0
README.md ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: tencent-hy-license
4
+ license_link: https://huggingface.co/tencent/Hy4-flash/blob/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959/LICENSE
5
+ base_model: tencent/Hy4-flash
6
+ base_model_relation: quantized
7
+ pipeline_tag: image-text-to-text
8
+ tags:
9
+ - mlx
10
+ - mtp
11
+ - dfly2
12
+ - speculative-decoding
13
+ - hunyuan
14
+ - hy4-flash
15
+ - mixture-of-experts
16
+ - vision-language
17
+ - quantized
18
+ - apple-silicon
19
+ - 4-bit
20
+ ---
21
+
22
+ # Hy4-flash MLX 4-bit + DFly2
23
+
24
+ > Draft model card. Checkpoint conversion is verified and TensorFold runtime support is pending.
25
+
26
+ This is TensorFold's MLX conversion of [Tencent Hy4-flash](https://huggingface.co/tencent/Hy4-flash/tree/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959), with the upstream DFly2 drafter packaged alongside the target weights.
27
+
28
+ ## About this conversion
29
+
30
+ | Item | Configuration |
31
+ | --- | --- |
32
+ | Target weight format | MLX affine 4-bit Safetensors |
33
+ | Group size | 64 for most weights, 32 for Engram embeddings |
34
+ | Draft weights | Upstream DFly2 weights and configuration at source precision |
35
+ | Vision tower and MoE router gates | Source precision |
36
+ | Verified tensor payload | 129.86 GB |
37
+
38
+ The [MLX 8-bit companion](https://huggingface.co/TensorFold/Hy4-flash-MLX-8bit-MTP) uses the same upstream revision and drafter.
39
+
40
+ ## Download and use
41
+
42
+ Download and quick-start instructions are pending runtime validation.
43
+
44
+ ## Apple Silicon performance
45
+
46
+ Decode speed and peak memory are pending measurement on Apple Silicon.
47
+
48
+ ## Runtime compatibility
49
+
50
+ TensorFold 1.2.9 rejects this checkpoint at startup because it has no `hy_v4_flash` backend. DFly2 additionally depends on the target's sparse-attention index and KV cache. The `MTP` suffix identifies the TensorFold variant that packages these draft weights.
51
+
52
+ ## Architecture
53
+
54
+ Tencent describes Hy4-flash as a multimodal mixture-of-experts model with 220 billion total parameters and about 12 billion active parameters per token. The language stack combines sliding-window attention, sparse attention and Engram embeddings. The upstream DFly2 drafter has five layers that reuse the target model's KV cache and sparse-attention positions.
55
+
56
+ ## Conversion and validation
57
+
58
+ The conversion uses Tencent's pinned BF16 checkpoint and retains the tokenizer, processor, chat template and DFly2 configuration. The 148 output shards match the tensor index, sampled packed weights match MLX affine quantization, and all 87 DFly2 tensors match the upstream bytes. Generation and drafted-versus-plain parity are pending runtime support.
59
+
60
+ ## Limitations
61
+
62
+ The file size estimates cover checkpoint weights. Runtime memory also depends on context length, cache settings and concurrent requests. No Apple Silicon memory tier has been measured for this conversion.
63
+
64
+ ## Licence and attribution
65
+
66
+ The original model, DFly2 drafter and evaluations are the work of Tencent and the upstream contributors. TensorFold prepares the MLX conversion. The [Tencent Hy License](https://huggingface.co/tencent/Hy4-flash/blob/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959/LICENSE) applies.