vimalnakrani/HY-Embodied-0.5-4bit-mlx

HY-Embodied-0.5 — 4-bit MLX

4-bit affine quantization (group size 64) of tencent/HY-Embodied-0.5, a 3.79B mixture-of-transformers embodied VLM, running natively on Apple Silicon. These weights require the from-scratch MLX implementation of the hunyuan_vl_mot architecture published alongside them: hy-embodied-mlx. No other public runtime supports this architecture; the implementation validates token-for-token against the reference before quantization.

What was modified

The original BF16 safetensors were converted to MLX layout, and the 448 decoder linear layers (both the text path and the _v vision-routed path) plus the tied embedding table were quantized to 4-bit, group size 64. The vision tower, merger, and all norms remain BF16. The quantization_manifest.json file in this repo records the decision for every module. Nothing else was changed.

Measured

	weights	decode tok/s	Where2Place no-think	Where2Place think
this repo (4-bit)	2.59 GiB	157.6	0.662 [0.565, 0.747]	0.631 [0.534, 0.720]
bf16 reference	7.05 GiB	66.8	0.696 [0.600, 0.778]	0.690 [0.593, 0.772]

Text-only probe (50 scripted prompts): 0.660 [0.522, 0.776] no-think, 0.860 [0.738, 0.930] think (bf16: 0.700 [0.562, 0.809] / 0.900 [0.786, 0.957]).

Brackets are Wilson 95% intervals (n=100 pointing, n=50 probe). Directionally lower than bf16 on pointing in both modes, though not statistically significant at these sample sizes; if memory allows, prefer the 5-bit variant or above. Measured on an M3 Max (36 GB), greedy decoding; per-question CSVs, statistical addendum, and one-command reproduction live in the implementation repo. All comparisons are within-harness: Tencent's published Where2Place 68.0 comes from an unpublished harness and is cited for range only.

All variants and the runtime are collected at https://huggingface.co/collections/vimalnakrani/hy-embodied-05-mlx-6a550eb39f59d2adf90c0355.

The "Use this model" snippet Hugging Face auto-generates for MLX repos (mlx-vlm) does not support this architecture; the Usage section below is the working path.

Usage

from PIL import Image
from transformers import AutoTokenizer
from hy_embodied_mlx.model import load, generate
from hy_embodied_mlx.pointing import FORMAT_INSTRUCTION
from hy_embodied_mlx.processor import Processor

model_dir = "HY-Embodied-0.5-4bit-mlx"
tok = AutoTokenizer.from_pretrained(model_dir)
model = load(model_dir)
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": f"Point to the red mug in the image. {FORMAT_INSTRUCTION}"},
]}]
inputs = Processor(tok)(messages, images=[Image.open("desk.jpg")])
print(tok.decode(generate(model, inputs, max_tokens=128)))

Pointing needs the format instruction shown — a bare "point to X" gets a prose location description. Emitted coordinates are integers in 0-1000, normalized to the preprocessed canvas (Tencent's documentation does not specify the frame; for images whose dimensions are multiples of 32 and within the 2048x2048 pixel budget, the canvas is pixel-identical to the input image). Thinking mode is controlled with enable_thinking=True/False on the chat template.

License

These weights are a Model Derivative of Tencent HY, distributed under the Tencent HY Community License (full text in the LICENSE file; NOTICE included). This is not an open-source license. The obligations and restrictions pass through to you:

• Territory: the license does not grant rights in the European Union, the United Kingdom, or South Korea.
• The Section 5(a) acceptable-use restrictions and the Section 5(b) restriction — including not using this model or its outputs to improve any other AI model — apply to these weights and anything you build on them.
• If you redistribute these weights or derivatives of them, include a copy of the license agreement, the NOTICE file, and a prominent statement of what you modified.

These Model Derivatives are distributed by the Hugging Face account vimalnakrani. This repository is an independent quantization and is not affiliated with, sponsored, or endorsed by Tencent.