Upload README.md with huggingface_hub

README.md

@@ -1,10 +1,12 @@
 ---
 license: apache-2.0
 base_model: tencent/Hy3
+base_model_relation: quantized
 pipeline_tag: text-generation
 library_name: vllm
 tags:
   - hunyuan
+  - hy_v3
   - mixture-of-experts
   - moe
   - 4-bit
@@ -17,7 +19,7 @@ tags:
   - yarn
 ---
 
-# Hy3 1M context 4-bit (INT4) quantization of tencent/Hy3
+# Hy3-W4A16 — 4-bit (INT4) quantization of tencent/Hy3
 
 A **4-bit weight-only (W4A16)** quantization of **[tencent/Hy3](https://huggingface.co/tencent/Hy3)**
 (`HYV3ForCausalLM`, `hy_v3`) — a 295B-parameter / 21B-active Mixture-of-Experts model.
@@ -33,16 +35,7 @@ Packaged in the **`compressed-tensors`** `pack-quantized` format so it loads dir
   Marlin INT4 MoE + Linear kernels. Fast tensor-core prefill.
 - **Long context via YaRN.** With YaRN RoPE scaling the context extends from the native
   **262,144** up to **1,048,576 (1M)** tokens (configurable). Dense needle-in-a-haystack
-  retrieval is **verified past native (1M, PASS)** on a single GPU; see *Long context* below.
-
-  ## Verified results (single B300, this checkpoint)
-
-| Test | Result |
-|---|---|
-| **HumanEval pass@1 (greedy)** | ✅ **150/164 = 91.5%** — coding ability well-preserved at 4-bit |
-| Needle-in-a-haystack 4K / 16K / 64K / 137K  | ✅ all PASS |
-| **Needle-in-a-haystack 320k/1024K dense (YaRN ×4, fp8 KV)** | ✅ **PASS**  |
-
+  retrieval is **verified past native (318K, PASS)** on a single GPU; see *Long context* below.
 
 ## Quantization details
 
@@ -64,8 +57,8 @@ Requires a vLLM build new enough to include the `hy_v3` architecture (vLLM `main
 writing). Example on a single GPU:
 
 ```bash
-vllm serve /path/to/autotrust/Hy3-1M \
-  --max-model-len 1048576 \
+vllm serve /path/to/Hy3-W4A16 \
+  --max-model-len 262144 \
   --gpu-memory-utilization 0.9 \
   --trust-remote-code
 ```
@@ -75,10 +68,10 @@ runtime JIT could not compile for `compute_103a` with the bundled CUDA toolkit,
 default sampler/attention. Work around it with the Triton attention backend + native sampler:
 
 ```bash
-VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve /path/autotrust/Hy3-1M \
+VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve /path/to/Hy3-W4A16 \
   --attention-backend TRITON_ATTN \
   --kv-cache-dtype fp8 \
-  --max-model-len 1048576 \
+  --max-model-len 262144 \
   --gpu-memory-utilization 0.9 \
   --trust-remote-code --enforce-eager
 ```
@@ -116,9 +109,11 @@ Memory on a single ~275 GB GPU (146 GB weights):
 | Test | Result |
 |---|---|
 | **HumanEval pass@1 (greedy)** | ✅ **150/164 = 91.5%** — coding ability well-preserved at 4-bit |
-| Needle-in-a-haystack 4K / 16K / 64K / 137K  | ✅ all PASS |
-| **Needle-in-a-haystack 320k/1024K dense (YaRN ×4, fp8 KV)** | ✅ **PASS**  |
-
+| **GSM8K (0-shot CoT, greedy)** | ✅ **1265/1319 = 95.9%** — math reasoning preserved at 4-bit |
+| Chat sanity (`11+22+33` → 66; capital of France → Paris; first 5 primes) | ✅ correct |
+| Needle-in-a-haystack 4K / 16K / 64K / 137K (in-range) | ✅ all PASS |
+| **Needle-in-a-haystack 318K dense (YaRN ×4, fp8 KV)** | ✅ **PASS** — retrieval works past the native 262,144 |
+| 1M-*distance* retrieval (sparse position probe, YaRN ×4) | ✅ retrieves where raw RoPE fails |
 
 <details>
 <summary><b>How HumanEval was measured</b> (for reproducibility)</summary>
@@ -138,12 +133,19 @@ Memory on a single ~275 GB GPU (146 GB weights):
   raw-completion protocol), so a few of the 14 misses may be extraction artifacts — treat 91.5%
   as a conservative figure.
 
+**GSM8K:** full 1319-problem test set, **0-shot chain-of-thought**, greedy (`temperature=0`,
+`max_tokens=512`), prompt *"Solve step by step… on the last line write 'The answer is <number>'"*;
+the final number is compared to the gold answer (after `####`). Result: **1265/1319 = 95.9%**.
+
 </details>
 
 ## Caveats & honesty
 
 - **4-bit RTN is lossy.** Chat/reasoning quality is well-preserved in our checks, but expect small
   degradation vs the BF16 original, especially on exact-match/coding tasks.
+- **`>262144` is extrapolation.** The base model was trained to 262,144; YaRN is applied at inference.
+  Dense retrieval is verified to **318K**; full dense-1M quality across depths is **not** exhaustively
+  validated here — evaluate for your use case.
 - This is a **community derivative**, not affiliated with or endorsed by Tencent.
 
 ## License