metadata
license: apache-2.0
base_model:
- baidu/ERNIE-4.5-21B-A3B-Base-PT
We performed a 20% REAP on the base model this is preparation for CPT and SFT/DPO. Further releases will be under IRIS 18B.
| Benchmark | Score | Notes |
|---|---|---|
| ARC-Easy | 79.59% | acc_norm |
| ARC-Challenge | 50.60% | acc_norm |
| HellaSwag | 70.50% | acc_norm |
| Winogrande | 67.17% | acc |
| GSM8K | 79.00% | exact_match (flexible-extract) |
| MMLU | 65.82% | acc (average across all subjects) |