Instructions to use TensorFold/Hy4-flash-MLX-8bit-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use TensorFold/Hy4-flash-MLX-8bit-MTP with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("TensorFold/Hy4-flash-MLX-8bit-MTP") config = load_config("TensorFold/Hy4-flash-MLX-8bit-MTP") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use TensorFold/Hy4-flash-MLX-8bit-MTP with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Hy4-flash-MLX-8bit-MTP"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "TensorFold/Hy4-flash-MLX-8bit-MTP" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use TensorFold/Hy4-flash-MLX-8bit-MTP with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Hy4-flash-MLX-8bit-MTP"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default TensorFold/Hy4-flash-MLX-8bit-MTP
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use TensorFold/Hy4-flash-MLX-8bit-MTP with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "TensorFold/Hy4-flash-MLX-8bit-MTP"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "TensorFold/Hy4-flash-MLX-8bit-MTP" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Hy4-flash MLX 8-bit with DFly2
Preview checkpoint. Weight conversion checks passed. TensorFold 1.2.9 rejects this architecture at startup. Generation, DFly2 parity, and runtime memory requirements are pending validation.
This is TensorFold’s MLX conversion of Tencent Hy4-flash, a multimodal model that takes text and images. Both variants include Tencent’s DFly2 drafter at source precision.
Choose a checkpoint
| Checkpoint | Weight files |
|---|---|
| MLX 4-bit | 129.86 GB |
| This repo · MLX 8-bit | 239.30 GB |
These sizes are for stored weights. Runtime memory depends on the engine, context length, cache settings, and concurrent requests. Check the TensorFold releases for runtime support before downloading.
What’s included
| Component | Format |
|---|---|
| Target weights | MLX affine 8-bit safetensors |
| Quantization group size | 64 for most weights and 32 for Engram embeddings |
| DFly2 drafter | Upstream weights and configuration at source precision |
| Vision tower and MoE router gates | Source precision |
The repo’s MTP suffix identifies the variant that bundles the DFly2 drafter. DFly2 requires the model’s sparse-attention index and KV cache.
Architecture
Tencent describes Hy4-flash as a multimodal mixture-of-experts model with 220 billion total parameters and about 12 billion active parameters per token. The language stack combines sliding-window attention, sparse attention and Engram embeddings. The upstream DFly2 drafter has five layers that reuse the target model's KV cache and sparse-attention positions.
Conversion and validation
The conversion uses Tencent's pinned BF16 checkpoint and retains the tokenizer, processor, chat template and DFly2 configuration. The 148 output shards match the tensor index, sampled packed weights match MLX affine quantization, and all 87 DFly2 tensors match the upstream bytes. Generation and drafted-versus-plain parity are pending runtime support.
Licence and attribution
The original model, DFly2 drafter and evaluations are the work of Tencent and the upstream contributors. TensorFold prepares the MLX conversion. The Tencent Hy License applies.
- Downloads last month
- -
8-bit
Model tree for TensorFold/Hy4-flash-MLX-8bit-MTP
Base model
tencent/Hy4-flash