---
license: apache-2.0
license_link: https://huggingface.co/tencent/Hy3/blob/main/LICENSE
base_model:
- tencent/Hy3
base_model_relation: quantized
quantized_by: AtomicChat
pipeline_tag: text-generation
library_name: gguf
tags:
- atomic-chat
- hunyuan
- tencent
- hy3
- moe
- gguf
- imatrix
- quantized
- llama.cpp
---
**Hy3** (Tencent Hunyuan Hy3), self-quantized to GGUF by [Atomic Chat](https://atomic.chat). Built straight from Tencent's original weights with a per-tensor importance matrix. Runs fully offline, including a 1-bit build that squeezes this 295B model onto a single multi-GPU box.
## Highlights
- **295B-parameter MoE with 21B active** (192 experts, top-8) plus a 3.8B MTP layer, so it delivers flagship-scale capability at a fraction of the active compute.
- **Rivals open-source flagships with 2-5x its parameters**, per Tencent, and edges out GLM-5.1 in blind expert evaluation (2.67/4).
- **Strong agentic and coding scores** (Tencent-reported): SWE-Bench Verified 78, GPQA Diamond 90.4, SWE-Bench Pro 57.9.
- **Production-grade reliability**: stable tool calls, hallucination rate down to 5.4%, commonsense errors 12.7%, multi-turn context issues 7.9%.
- **256K context** with grouped-query attention (8 KV heads, head dim 128) and an MTP layer for speculative decoding.
- **Adjustable reasoning** via `reasoning_effort` (`no_think` / `low` / `high`) for chain-of-thought when a task needs it.
- **Full imatrix quantization** over [`calibration_datav3`](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8), including a 1-bit `IQ1_M`.
> [!NOTE]
> These GGUFs are **self-quantized from the original weights**, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.
> [!IMPORTANT]
> Always pass `--jinja` so the **Hy3 chat template** is applied. Without it the model can emit malformed turns.
## Model Overview
| Property | Value |
|---|---|
| Base model | `tencent/Hy3` |
| Total / active parameters | 295B total / 21B active (plus a 3.8B MTP layer) |
| Layers | 80 (plus 1 MTP layer) |
| Experts | 192 experts, top-8 activated |
| Context length | 256K |
| Architecture | Mixture-of-Experts, GQA (8 KV heads, head dim 128), MTP speculative-decoding layer, reasoning modes |
| This repo | GGUF quants (imatrix): `Q4_K_M` for near-reference quality, and a 1-bit `IQ1_M` (≈92GB) that makes this 295B model run locally. |
Scores are Tencent's published results for the base `tencent/Hy3`. Quantization preserves the large majority of this; `Q4_K_M` and up sit within a point or two of full precision.
## Choosing a quant
| Quant | Size | Notes |
|---|---|---|
| **`IQ1_M`** | 91.8 GB | **Smallest. 1-bit imatrix build that makes the 295B model run locally (≈92GB, e.g. a 4-GPU box). Expect quality tradeoffs; reasoning still works.** |
| **`Q4_K_M`** | 184.7 GB | **Recommended for quality. Near-reference; needs roughly 185GB of combined VRAM and RAM.** |
## Get started
Run Hy3 locally with:
- **[Atomic Chat](https://atomic.chat):** the easiest path. Open the app, search `AtomicChat/Hy3-GGUF`, pick a quant, hit **Use this model**.
- **llama.cpp:** `llama-server -hf AtomicChat/Hy3-GGUF:Q4_K_M --jinja -c 8192`
- **Ollama:** `ollama run hf.co/AtomicChat/Hy3-GGUF:Q4_K_M`
- **LM Studio / Jan:** search the repo id, download any quant.
## Best practices
| Parameter | Value |
|---|---|
| temperature | 0.9 |
| top_p | 1.0 |
**Reasoning is recommended for Hy3.** Pass `reasoning_effort` (`high` for full chain-of-thought, `low`, or `no_think` for direct answers). On hard tasks, running without reasoning can produce truncated or malformed output.
## Run in llama.cpp
```bash
git clone https://github.com/ggerganov/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server
```
```bash
./llama.cpp/build/bin/llama-server \
-hf AtomicChat/Hy3-GGUF:IQ1_M \
--jinja -ngl 99 -c 8192 -fa on
```
## How these were made
1. Download `tencent/Hy3` (original weights).
2. Convert to GGUF with a [llama.cpp](https://github.com/ggerganov/llama.cpp) build that supports the Hy3 (`hy_v3`) architecture and its MTP layer.
3. Build an importance matrix over `calibration_datav3`.
4. Quantize with `--imatrix`: `Q4_K_M` for quality and `IQ1_M` for the smallest footprint that keeps this 295B model coherent.
## License
Released by Tencent under the Apache 2.0 license. Quantized by Atomic Chat.
