Instructions to use nace-ai/Hy4-MLX-4.6bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use nace-ai/Hy4-MLX-4.6bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("nace-ai/Hy4-MLX-4.6bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use nace-ai/Hy4-MLX-4.6bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nace-ai/Hy4-MLX-4.6bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nace-ai/Hy4-MLX-4.6bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use nace-ai/Hy4-MLX-4.6bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "nace-ai/Hy4-MLX-4.6bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "nace-ai/Hy4-MLX-4.6bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nace-ai/Hy4-MLX-4.6bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use nace-ai/Hy4-MLX-4.6bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nace-ai/Hy4-MLX-4.6bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default nace-ai/Hy4-MLX-4.6bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use nace-ai/Hy4-MLX-4.6bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nace-ai/Hy4-MLX-4.6bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "nace-ai/Hy4-MLX-4.6bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
We introduce Hy4 MLX from Nace.AI, Tencent's Hy4 in 447.3 GB of text weights at 4.6 bits per weight. Generate text with the included Linux CUDA runner.
Results
Selected public quantizations, scored against native BF16 on the same 11 contexts and 5,621 positions. Download sizes include vision weights, with the shared BF16 projector added to every GGUF file. Bit labels show average text-model storage precision, excluding vision. MLX and GGUF use different inference backends, so measured divergence can include backend effects.
Overview
The full model retains every routed expert, with precision assigned by weight class.
- 78 decoder layers, with all 256 routed experts retained in every MoE layer.
- 456.3 GB complete weight download, including the retained nontext weights.
Nace Weight Space Quantization
Four-bit experts, higher precision for the rest.
Routed expert weights use 4-bit storage. Eligible trunk matrices use 8-bit storage, while routers, KV expansion and sensitive parameters keep higher precision. The text model averages 4.6 bits per weight.
Run it
Use Linux with CUDA 13, as tested on an NVIDIA A40.
python3 -m venv .venv-hy4
. .venv-hy4/bin/activate
pip install huggingface_hub
hf download nace-ai/Hy4-MLX-4.6bit --local-dir ./Hy4-MLX-4.6bit
pip install -r ./Hy4-MLX-4.6bit/requirements-cuda13.txt
CUDA_VISIBLE_DEVICES=0 python ./Hy4-MLX-4.6bit/run.py \
--model ./Hy4-MLX-4.6bit --output ./hy4-answer \
--device gpu --prompt 'What is the capital of France? Answer one short sentence.' \
--max-tokens 16
Choose a new output directory for each prompt.
License
Tencent Hy License, Version 1.0, including its Section 10 commercial terms for large Model-as-a-Service operators. See NOTICE and modifications.
- Downloads last month
- 55
8-bit
Model tree for nace-ai/Hy4-MLX-4.6bit
Base model
tencent/Hy4


