Hy3-JANG_2K

Quantized tencent/Hy3 for Apple Silicon MLX / JANG runtimes — a 295B-total / 21B-active text MoE, packed to ~94 GiB. This is the clean non-MTP JANG_2K bundle (smallest 2K pack). For the variant that keeps Hy3's native Multi-Token-Prediction head, see Hy3-JANG_2K-MTP.

Source	tencent/Hy3
License	other — inherits the upstream Tencent Hunyuan Community License
Architecture	hy_v3 (HYV3ForCausalLM), text-only
Parameters	295B total / 21B active per token
Format	JANG_2K (mixed-affine), routed experts avg 2.33-bit
Bundle size	101.40 GB (94.44 GiB), 22 shards, 2,876 tensor keys
MTP	none (num_nextn_predict_layers = 0) — MTP head not included
Context	262,144 tokens

What this is

Hy3-JANG_2K is a JANG mixed-affine quantization of Tencent's Hy3 dense-MoE, targeting Apple Silicon runtimes (MLX / vMLX). The 2K profile spends an extra bit on the routed down_proj (3-bit vs the 2-bit gate/up), which cleans up the sampling tail relative to a uniform 2-bit pack. This bundle drops the native MTP layer for the smallest footprint; use Hy3-JANG_2K-MTP if you want speculative decoding.

Quantization (JANG_2K)

Tensor family	Policy
Routed expert gate_proj / up_proj	affine 2-bit, group size 128
Routed expert down_proj	affine 3-bit, group size 128
Attention q/k/v/o	affine 8-bit
Shared expert	affine 8-bit
Dense layer-0 MLP	affine 8-bit
embed_tokens	affine 6-bit
lm_head	affine 8-bit
RMSNorms, router gate, expert bias	16-bit passthrough

Routed-expert effective average: 2.33 bit. AWQ scaling is disabled for this bundle (measured negligible on Hy3).

Architecture

Hy3 is a text-only dense-causal-GQA MoE — not MLA, not SSM, not sliding-window, not a VLM.

• 80 decoder layers, hidden_size 4096
• GQA: 64 attention heads / 8 KV heads, head_dim 128, QK-norm
• RoPE default, rope_theta 11,158,840, max_position_embeddings 262,144
• MoE: 192 routed experts, top-8, sigmoid router + expert-correction bias, route_norm, router_scaling_factor 2.826, 1 shared expert, first_k_dense_replace 1
• No MTP layer in this bundle (num_nextn_predict_layers = 0)
• vocab_size 120,832

Reasoning & tool use

• Reasoning: <think>…</think> tags, reasoning_effort (no_think / low / high).
• Tool calling: Hunyuan / Tencent XML-style tags (<tool_calls>, <tool_call>, <arg_key>, <arg_value>).
• Hy3's tokenizer uses a :opensource special-token dialect (e.g. <｜hy_eos:opensource｜>, <think:opensource>); the bundled chat_template.jinja is the upstream template. A compatible runtime must resolve these variant-suffixed tokens at the token→text boundary.

Runtime support

• Converted and structurally verified (index complete, 2,876 tensors / 22 shards, no MTP tensors).
• Runs on the vMLX Python engine with Hy3 support: JANG affine loader, GQA KV cache, <think> reasoning stream, and Hunyuan tool-call parsing.

Requires a Hy3-aware MLX/JANG runtime. Stock mlx-lm / transformers will not load the JANG mixed-affine layout as-is.

Known limitations

• No published quality benchmark yet for this specific pack.
• Very loose sampling (top_p 1.0 + temperature 0.9) exposes more of the routed-expert tail; a mild top_p ≤ 0.9 or min_p floor is recommended for long-form generation.

소개 (Korean)

이 번들은 Tencent의 Hy3 (295B 총 파라미터 / 21B 활성 MoE, 텍스트 전용)를 Apple Silicon MLX / JANG 런타임용으로 양자화한 모델입니다. JANG_2K 프로파일은 라우팅 전문가의 down_proj를 3-bit로, gate/up을 2-bit로 양자화합니다(평균 2.33-bit). 이 번들은 MTP 헤드를 포함하지 않는 가장 작은 2K 팩이며, 스펙티브 디코딩이 필요하면 Hy3-JANG_2K-MTP를 사용하세요. Hy3의 GQA 어텐션과 MoE 라우팅, :opensource 특수 토큰 방식을 정확히 구현한 런타임에서만 사용해야 합니다.