Instructions to use OpenMOSE/Hy3-REAP-200B-21B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenMOSE/Hy3-REAP-200B-21B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenMOSE/Hy3-REAP-200B-21B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenMOSE/Hy3-REAP-200B-21B")
model = AutoModelForCausalLM.from_pretrained("OpenMOSE/Hy3-REAP-200B-21B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenMOSE/Hy3-REAP-200B-21B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenMOSE/Hy3-REAP-200B-21B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenMOSE/Hy3-REAP-200B-21B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenMOSE/Hy3-REAP-200B-21B

SGLang

How to use OpenMOSE/Hy3-REAP-200B-21B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenMOSE/Hy3-REAP-200B-21B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenMOSE/Hy3-REAP-200B-21B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenMOSE/Hy3-REAP-200B-21B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenMOSE/Hy3-REAP-200B-21B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenMOSE/Hy3-REAP-200B-21B with Docker Model Runner:
```
docker model run hf.co/OpenMOSE/Hy3-REAP-200B-21B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Hy3-REAP-200B-21B

A 33%-expert-pruned version of Tencent Hunyuan Hy3 (295B / A21B), compressed with REAP (Router-weighted Expert Activation Pruning) — pruning only, no expert merging.

This checkpoint reduces the number of routed experts in tencent/Hy3 from 192 → 128 per MoE layer (−33%) using the REAP saliency criterion, as implemented in the SamsungSAILMontreal/ream codebase. The result is a smaller, drop-in Hunyuan V3 model (~200B total parameters) that keeps the same 21B active-parameter path and the full 256K context window.

Because the router is left untouched and no experts are averaged together, this is a pure REAP checkpoint — the pruned model retains the original router's independent, input-dependent control over the surviving experts.

Model Overview

Property	Value
Base model	`tencent/Hy3` (Hunyuan V3, 295B / A21B)
Architecture	`HYV3ForCausalLM` (`model_type: hy_v3`)
Compression method	REAP (saliency = `reap`, merging = `none`)
Pruning ratio	~33% of routed experts (192 → 128 per layer)
Total parameters	~200B (down from ~295B)
Active parameters	~21B (unchanged)
Routed experts / layer	128 (was 192), top-8 routing
Shared experts / layer	1 (unchanged)
Layers	80 (1 dense + 79 sparse)
Context length	256K (`max_position_embeddings = 262144`)
MTP layers	1 (`num_nextn_predict_layers = 1`)
Precision	BF16
Calibration data	`OpenMOSE/reap-calib-mix`

The active-parameter path is unchanged because top-8 routing and the single shared expert are preserved; pruning removes only rarely / weakly used routed experts, so per-token FLOPs stay the same while the total memory footprint drops by roughly one third.

Method: REAP (pruning, not merging)

REAP (Router-weighted Expert Activation Pruning) scores every expert on a calibration set using a saliency signal that combines router gate-values (how strongly / often the router selects an expert) and expert activation norms (the magnitude of each expert's output contribution), normalized by the number of tokens routed to that expert. Experts with the lowest saliency contribute least to each layer's output and are removed.

REAP is a one-shot, post-training procedure: no gradient updates or fine-tuning are required after pruning. Crucially, unlike expert-merging approaches, pruning does not collapse experts into shared weights, so the router keeps its fine-grained, input-dependent modulation of the remaining experts. The REAP authors show this matters most on generative tasks, where merging introduces an irreducible reconstruction error.

Why REAP rather than REAM on Hy3

The ream codebase supports both REAP (pruning) and REAM (Router-weighted Expert Activation Merging). REAM often preserves multiple-choice accuracy better by merging grouped experts instead of dropping them.

For Hy3 specifically, we found that REAP gave the better trade-off between quality loss and achievable pruning depth than REAM. At the compression ratio used here, the pruning-only path stayed closer to the baseline on generative behavior while allowing a cleaner one-third reduction, so this release ships as a pure REAP checkpoint (merging: none).

Calibration data

Off-the-shelf calibration corpora left the expert-saliency estimates unbalanced across domains. To improve pruning accuracy, calibration used OpenMOSE/reap-calib-mix — the same mixed dataset used in the OpenMOSE model-distillation pipeline — which balances the domain distribution seen during saliency accumulation and produced a more even expert-importance estimate.

Pruning configuration

The run used the following key arguments (from config.json → merge_args):

Argument	Value
`saliency`	`reap`
`merging`	`none` (pure pruning)
`grouping`	`ream`
`group_size`	16
`merge_size`	128 (target experts per layer)
`dataset`	`reapmix_pack` (`OpenMOSE/reap-calib-mix`)
`calib_size`	3072
`calib_seq_len`	2048
`batch_size`	8
`mix_ratio`	`1.0`
`no_gated_sim`	true
`no_sequential`	true

Architecture: before vs. after

Field	Base Hy3	Hy3-REAP-200B-21B
`num_experts`	192	128
`num_experts_per_tok`	8	8
`num_shared_experts`	1	1
`num_hidden_layers`	80	80
`hidden_size`	4096	4096
`moe_intermediate_size`	1536	1536
`expert_hidden_dim`	1536	1536
`intermediate_size` (dense)	13312	13312
`first_k_dense_replace`	1	1
`num_attention_heads`	64	64
`num_key_value_heads`	8	8
`head_dim`	128	128
`qk_norm`	true	true
`moe_router_use_sigmoid`	true	true
`router_scaling_factor`	2.826	2.826
`rope_theta`	11158840.0	11158840.0
`max_position_embeddings`	262144	262144
`vocab_size`	120832	120832
`tie_word_embeddings`	false	false

Everything except the routed-expert count is preserved, including the router, attention, shared expert, dense layer, and RoPE configuration.

Evaluation

Benchmark	Base Hy3	Hy3-REAP (33%)	Δ
MMLU	85.65%	82.37%	−3.28 pt (−3.8% rel.)
GSM8K	94.54%	94.62%	+0.08 pt (≈ noise)

At 33% expert pruning, MMLU drops by only a few points while GSM8K is effectively unchanged (within run-to-run noise). Grade-school math reasoning survives the compression almost perfectly, and broad knowledge (MMLU) degrades modestly.

A note on Hy3's pruning sensitivity

In our experiments, Hy3 is considerably less pruning-tolerant than other open MoE models such as Qwen3 and GLM. Those families can typically absorb deeper expert pruning (often up to 50%) with small quality loss, whereas Hy3 leaves very little headroom: quality degrades sharply beyond the ratio used here. In practice, **33% is close to the useful pruning ceiling for Hy3** — this checkpoint sits near that ceiling deliberately, and we do not recommend pushing the expert count much lower without recovery training. The careful calibration mix (reap-calib-mix) was necessary precisely because of this low tolerance.

Benchmark figures are self-reported and intended for relative comparison against the base model under an identical evaluation harness; absolute numbers may differ from other setups.

Usage

The model uses the standard Hunyuan V3 (hy_v3) architecture and is a drop-in replacement for tencent/Hy3 at reduced size. Serve it the same way you would serve Hy3.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "OpenMOSE/Hy3-REAP-200B-21B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Explain mixture-of-experts pruning in one paragraph."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Hy3's hybrid fast/slow reasoning modes (no_think / think_low / think_high) and tool-calling behavior are inherited from the base model.

Limitations

One-shot compression, no recovery training. This checkpoint is pruned only; no post-pruning distillation or fine-tuning was applied. Optional expert-wise distillation could recover some of the MMLU gap if needed.
Near the pruning ceiling. Hy3 tolerates pruning poorly (see above). Do not expect deeper pruning of this architecture to remain lossless.
Benchmark coverage is narrow. Only MMLU and GSM8K were measured here. Long-context retrieval, coding, agentic tool use, and multilingual behavior were not re-evaluated after pruning and may be affected differently.
Inherited behavior. All safety, bias, and factuality characteristics of the base Hy3 model carry over and may interact with pruning in ways not fully characterized.

License

This is a derivative of tencent/Hy3, which is released under the Apache License 2.0 (the full Hy3 release removed the regional restrictions present in the earlier Hy3-preview). This checkpoint is distributed under the same terms. Users should confirm they are complying with the base model's license before deployment.

Acknowledgements & Citation

Pruning was performed with the SamsungSAILMontreal/ream codebase, which implements both REAP and REAM. The base model is Tencent Hunyuan Hy3.

If you use this checkpoint, please cite the REAP paper (pruning criterion), the REAM paper (codebase / merging counterpart), and the base Hunyuan Hy3 model.

@article{reap2025,
  title   = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression},
  author  = {Cerebras Research},
  journal = {arXiv preprint arXiv:2510.13999},
  year    = {2025}
}

@article{ream2026,
  title   = {REAM: Merging Improves Pruning of Experts in LLMs},
  author  = {Samsung SAIL Montreal},
  journal = {arXiv preprint arXiv:2604.04356},
  year    = {2026}
}

Pruned and released by OpenMOSE.