Image-Text-to-Text
MLX
Safetensors
hy_v4_flash
mtp
dfly2
speculative-decoding
hunyuan
hy4-flash
mixture-of-experts
vision-language
quantized
apple-silicon
4-bit precision
conversational
Instructions to use TensorFold/Hy4-flash-MLX-4bit-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use TensorFold/Hy4-flash-MLX-4bit-MTP with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("TensorFold/Hy4-flash-MLX-4bit-MTP") config = load_config("TensorFold/Hy4-flash-MLX-4bit-MTP") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use TensorFold/Hy4-flash-MLX-4bit-MTP with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Hy4-flash-MLX-4bit-MTP"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "TensorFold/Hy4-flash-MLX-4bit-MTP" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use TensorFold/Hy4-flash-MLX-4bit-MTP with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Hy4-flash-MLX-4bit-MTP"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default TensorFold/Hy4-flash-MLX-4bit-MTP
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use TensorFold/Hy4-flash-MLX-4bit-MTP with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Hy4-flash-MLX-4bit-MTP"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "TensorFold/Hy4-flash-MLX-4bit-MTP" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Draft Hy4-flash model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: tencent-hy-license
|
| 4 |
+
license_link: https://huggingface.co/tencent/Hy4-flash/blob/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959/LICENSE
|
| 5 |
+
base_model: tencent/Hy4-flash
|
| 6 |
+
base_model_relation: quantized
|
| 7 |
+
pipeline_tag: image-text-to-text
|
| 8 |
+
tags:
|
| 9 |
+
- mlx
|
| 10 |
+
- mtp
|
| 11 |
+
- dfly2
|
| 12 |
+
- speculative-decoding
|
| 13 |
+
- hunyuan
|
| 14 |
+
- hy4-flash
|
| 15 |
+
- mixture-of-experts
|
| 16 |
+
- vision-language
|
| 17 |
+
- quantized
|
| 18 |
+
- apple-silicon
|
| 19 |
+
- 4-bit
|
| 20 |
+
---
|
| 21 |
+
|
| 22 |
+
# Hy4-flash MLX 4-bit + DFly2
|
| 23 |
+
|
| 24 |
+
> Draft model card. Checkpoint conversion is verified and TensorFold runtime support is pending.
|
| 25 |
+
|
| 26 |
+
This is TensorFold's MLX conversion of [Tencent Hy4-flash](https://huggingface.co/tencent/Hy4-flash/tree/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959), with the upstream DFly2 drafter packaged alongside the target weights.
|
| 27 |
+
|
| 28 |
+
## About this conversion
|
| 29 |
+
|
| 30 |
+
| Item | Configuration |
|
| 31 |
+
| --- | --- |
|
| 32 |
+
| Target weight format | MLX affine 4-bit Safetensors |
|
| 33 |
+
| Group size | 64 for most weights, 32 for Engram embeddings |
|
| 34 |
+
| Draft weights | Upstream DFly2 weights and configuration at source precision |
|
| 35 |
+
| Vision tower and MoE router gates | Source precision |
|
| 36 |
+
| Verified tensor payload | 129.86 GB |
|
| 37 |
+
|
| 38 |
+
The [MLX 8-bit companion](https://huggingface.co/TensorFold/Hy4-flash-MLX-8bit-MTP) uses the same upstream revision and drafter.
|
| 39 |
+
|
| 40 |
+
## Download and use
|
| 41 |
+
|
| 42 |
+
Download and quick-start instructions are pending runtime validation.
|
| 43 |
+
|
| 44 |
+
## Apple Silicon performance
|
| 45 |
+
|
| 46 |
+
Decode speed and peak memory are pending measurement on Apple Silicon.
|
| 47 |
+
|
| 48 |
+
## Runtime compatibility
|
| 49 |
+
|
| 50 |
+
TensorFold 1.2.9 rejects this checkpoint at startup because it has no `hy_v4_flash` backend. DFly2 additionally depends on the target's sparse-attention index and KV cache. The `MTP` suffix identifies the TensorFold variant that packages these draft weights.
|
| 51 |
+
|
| 52 |
+
## Architecture
|
| 53 |
+
|
| 54 |
+
Tencent describes Hy4-flash as a multimodal mixture-of-experts model with 220 billion total parameters and about 12 billion active parameters per token. The language stack combines sliding-window attention, sparse attention and Engram embeddings. The upstream DFly2 drafter has five layers that reuse the target model's KV cache and sparse-attention positions.
|
| 55 |
+
|
| 56 |
+
## Conversion and validation
|
| 57 |
+
|
| 58 |
+
The conversion uses Tencent's pinned BF16 checkpoint and retains the tokenizer, processor, chat template and DFly2 configuration. The 148 output shards match the tensor index, sampled packed weights match MLX affine quantization, and all 87 DFly2 tensors match the upstream bytes. Generation and drafted-versus-plain parity are pending runtime support.
|
| 59 |
+
|
| 60 |
+
## Limitations
|
| 61 |
+
|
| 62 |
+
The file size estimates cover checkpoint weights. Runtime memory also depends on context length, cache settings and concurrent requests. No Apple Silicon memory tier has been measured for this conversion.
|
| 63 |
+
|
| 64 |
+
## Licence and attribution
|
| 65 |
+
|
| 66 |
+
The original model, DFly2 drafter and evaluations are the work of Tencent and the upstream contributors. TensorFold prepares the MLX conversion. The [Tencent Hy License](https://huggingface.co/tencent/Hy4-flash/blob/d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959/LICENSE) applies.
|