# Reproduction Guide — HiF8 QAT for OpenPangu-Embedded-1B

IEEE ICME 2026 Low Bit-width Large Model Quantization Challenge submission.

## Environment Setup

### Training environment (pangu1b_hif8)
```bash
conda env create -f pangu1b_hif8.yml
conda activate pangu1b_hif8
```

### Evaluation environment (pangu1b_eval)
```bash
conda env create -f pangu1b_eval.yml
conda activate pangu1b_eval
pip install lm-eval==0.4.11 ray==2.55.1 "antlr4-python3-runtime==4.11" sympy math_verify
```

## Quantized Model

The submitted quantized checkpoint corresponds to **max_quant_lr1e5** (HiF8 W8A8 QAT, lr=1e-5).

Place the checkpoint at:
```
checkpoints/max_quant_lr1e5/final/
```

## Reproduce Training

### Step 1 — BF16 baseline (lr=1e-5)
```bash
bash pangu_hif8_pretrain/run_train_bf16_lr1e5.sh
```
Output: `checkpoints/bf16_lr1e5/final/`

### Step 2 — HiF8 QAT (max_quant_lr1e5, lr=1e-5)
```bash
bash pangu_hif8_pretrain/run_train_max_quant_lr1e5.sh
```
Output: `checkpoints/max_quant_lr1e5/final/`

Key quantization settings:
| Parameter | Value |
|-----------|-------|
| Quantization | W8A8 HiF8 |
| amax algorithm | `max` (over history window) |
| amax history length | 64 steps |
| BF16 warmup steps | 500 |
| HiF8 max value (fwd/bwd) | 15 |
| Learning rate | 1e-5 |
| Global batch size | 1024 |
| Max steps | 10000 |
| High-precision layers | 5 |

Training takes approximately **21 hours** on 8× NVIDIA H100 80GB.

## Reproduce Evaluation

```bash
cd evaluate_benchmarks

# Evaluate a single model
bash run_eval.sh max_quant_lr1e5

# Generate comparison table (baseline: bf16_lr1e5)
python compare_results.py
```

Benchmarks: MMLU (5-shot), GSM8K (5-shot), MATH500/minerva_math (4-shot),
HellaSwag (10-shot), ARC-Easy (25-shot), ARC-Challenge (25-shot).

## Key Results (max_quant_lr1e5 vs bf16_lr1e5 baseline)

| Task | BF16 (lr=1e-5) | HiF8 QAT (lr=1e-5) | Drop |
|------|---------------|---------------------|------|
| MMLU (5-shot) | 43.36% | 43.17% | 0.43% |
| GSM8K (5-shot) | 1.59% | 1.29% | — (noise) |
| MATH500 (4-shot) | 0.50% | 0.46% | — (noise) |
| HellaSwag (10-shot) | 41.10% | 40.86% | 0.58% |
| ARC-Easy (25-shot) | 51.56% | 51.01% | 1.06% |
| ARC-Challenge (25-shot) | 36.77% | 36.69% | 0.22% |