openPangu-2.0-Pro
中文 | English
1. Introduction
openPangu-2.0-Pro is an MoE model trained on Ascend. The model has 505B total parameters and 18B activated parameters. Its context length is 512k. The total pretraining data contains 34T tokens. During Post-training, openPangu-2.0-Pro is trained through unified SFT with slow and fast thinking capability, multiple specialist RL traning, on-policy distillation combining multiple RL specialists. More details, please refer to openPangu-2.0 Tech Report.
2. Architecture
openPangu-2.0-Pro brings several major architectural improvements:
- Efficient attention: The model retains MLA for efficient inference and combines DSA and SWA in a 1:2 layer ratio. SWA layers handle local-window modeling, while DSA layers capture sparse global context. This design lowers compute, memory footprint, and memory access costs for long-context inference while preserving accuracy.
- Residual topology: The conventional residual path is replaced with a 4-stream mHC design, improving representation diversity and generalization.
- Multi-token prediction (MTP): The model uses three MTP heads to draft 3 additional tokens per step, enabling faster inference through self-speculative decoding.
- Optimizer: Training uses the Muon optimizer for faster convergence.
3. Evaluation Results
| Benchmark | Metric | openPangu-2.0-Pro-Thinking | openPangu-2.0-Pro-Non-Thinking |
|---|---|---|---|
| General | |||
| CL-Bench | Acc | 19.7 | 17.8 |
| IFEval | Prompt Strict | 94.5 | 90.4 |
| IFBench | Prompt Strict | 79.9 | 59.5 |
| AgentIF | (CSR+ISR)/2 | 44.4 | 42.2 |
| SysBench | ISR | 90.8 | 82.6 |
| Multichallenge | Acc | 62.9 | 52.2 |
| SimpleQA Verified | Acc | 34.9 | 27.1 |
| Chinese-SimpleQA | Acc | 74.2 | 75.7 |
| Reasoning | |||
| HMMT Feb 2026 | Avg@16 | 86.2 | 63.3 |
| - w/ Python | Avg@16 | 93.2 | - |
| AIME 2026 | Avg@16 | 95.4 | 87.3 |
| - w/ Python | Avg@16 | 97.9 | - |
| Apex | Avg@16 | 17.7 | 2.1 |
| IMO-AnswerBench | Acc | 84.3 | 60.8 |
| BBEH | Harmonic Mean | 69.7 | 55.0 |
| GPQA-Diamond | Avg@4 | 87.9 | 81.1 |
| HLE | Acc | 27.1 | 9.0 |
| Agent | |||
| TAU2-Bench | Avg@3 | 81.1 | 71.6 |
| MCP-Atlas | Acc | 61.3 | 48.8 |
| SkillsBench | Avg@5 | 48.9 | 47.5 |
| PinchBench | Avg@3 | 84.0 | 88.9 |
| WildClawBench | Avg@3 | 46.5 | 43.1 |
| Claw-Eval | Pass^3 | 61.7 | 49.4 |
| GDPVal | Acc | 74.9 | 51.2 |
| BrowseComp | Acc | 65.7 | - |
| Coding | |||
| LiveCodeBench V6 | Avg@3 | 85.7 | 74.5 |
| FeatBench | Avg@3 | 47.1 | 48.4 |
| DeepCodeBench | Avg@3 | 74.2 | 72.6 |
| SWE-bench Verified | Avg@3 | 68.5 | 66.8 |
4. Deployment
- omni-infer：Please refer to [openPangu-2.0-Pro deploy guide]
- source code repo:[openPangu-2.0-Infer]
5. Model License
Unless otherwise noted, the openPangu-2.0-Pro model is licensed under the terms and conditions of OPENPANGU MODEL LICENSE AGREEMENT VERSION 2.0, which is intended to be used permissively and enable the further development of artificial intelligence technologies. Please refer to the LICENSE file located in the root directory of the model repository for details.
6. Disclaimer
Due to the technical limitations inherent in the technology on which the openPangu-2.0-Pro model (“Model”) relies and the fact that the artificial intelligence generated content is automatically produced by Model, Huawei cannot make any guarantees regarding the following matters:
- The output of this Model is automatically generated via AI algorithms, it does not rule out the possibility that some of the information may be flawed, unreasonable, or cause discomfort, and the generated content does not represent Huawei's attitude or standpoint;
- There is no guarantee that this Model is 100% accurate, reliable, functional, timely, secure and safety, error-free, uninterrupted, continuously stable, or free of any faults;
- The output of this Model does not constitute any advices or decisions for you, and it does not guarantee the authenticity, completeness, accuracy, timeliness, legality, functionality, or practicality of the generated content. The generated content cannot replace professionals in medical, legal, and other fields in answering your questions. The generated content is for your reference only and does not represent any attitude, standpoint, or position of Huawei. You need to make independent judgments based on your actual situation, and Huawei does not assume any responsibilities.
7. Contact
If you have any question, please raise an issue or contact us at openPangu@huawei.com.