Sync model (SHA: af340d0547381f9a366057bd467cd1ea313d889c)

.gitattributes

@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+special_tokens_map.json filter=lfs diff=lfs merge=lfs -text
+*.cms filter=lfs diff=lfs merge=lfs -text
+*.hwp7s filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.modelenc filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,37 @@
+OpenPangu Model License Agreement Version 2.0
+
+This OpenPangu Model License Agreement Version 2.0 (the "Agreement") is a legal agreement between You and Huawei Technologies Co., Ltd. ("Huawei", "We" or "Us"), and it governs Your reproducing, use, modification, and distribution of openPangu as made available by Huawei under this Agreement. 
+
+By using, reproducing, modifying, distributing, performing or displaying any portion or element of openPangu, or otherwise accepting the terms of this Agreement, You agree to be bound by this Agreement.
+
+1.	Definitions. 
+1.1.	“openPangu” or “Model” means openPangu large language models and software, including trained model weights, parameters (including optimizer states), accompanying source code and scripts released under this Agreement. 
+1.2.	“Derivative Model” means all (1) modifications to the Model, (2) works based on the Model, and (3) any other derivative works of the Model. For clarity, information or content results from operating or otherwise using the Model is not a Derivative Model. 
+1.3.	“You” or “Your” means an individual or Legal Entity exercising permissions granted by this Agreement and/or using the Model for any purpose.
+1.4.	“Third Party” or “Third Parties” means individuals or legal entities that are not under common control with Us or You.
+
+2.	License Grant. Subject to Your full compliance with the terms and conditions of this Agreement, We hereby grant to You a perpetual, worldwide, non-exclusive, non-transferable, no-charge, royalty-free license (except as stated in Section 3) to use, reproduce, modify, and distribute the Model. 
+
+3.	Conditions for License Grant. 
+3.1.	You represent and warrant that You will not, access, download, install, run, deploy, integrate, modify, or otherwise use the Model, directly or indirectly, within the European Union.
+3.2.	If You initiate any copyright or patent litigation against any entity (including filing a cross-claim or counterclaim in such litigation), alleging that this Model or Derivative Models constitute direct or indirect copyright or patent infringement, then any license granted to You under this Agreement for the Model or Derivative Models shall terminate as of the date the aforementioned litigation is filed.  
+
+4.	Redistribution. 
+4.1.	If You distribute the Model or Derivative Model, You shall retain in Your distribution (1) a copy of this agreement, and (2) all copyright notices and other notices of origin included in the Model that are applicable to Your distribution.
+4.2.	Further, if You distribute or make available to Third Parties a product or service (including another AI model) based on the Model, You are required to (1) display the acknowledgement “Powered by openPangu” and (2) include a trademark notice “openPangu is a trademark of Huawei Technologies Co., Ltd.” on related webpages, user manuals, product documentations or other advertising materials mentioning features of the Model.
+4.3.	You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for Derivative Model made by You as a whole, provided Your use, reproduction, and distribution of the Model otherwise complies with the terms and conditions of this Agreement.
+
+5.	Ownership. We do not claim ownership to any information or content generated using the Model or Derivative Model that are made by You. You are solely responsible for evaluating the accuracy and appropriateness of such information or content for Your use case.
+
+6.	Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of Huawei, except as required for complying with Section 4.2.
+
+7.	Indemnity. You will indemnify and hold harmless Huawei from and against any claim by any third party arising out of or related to Your use or distribution of the Model or Derivative Model made by You (e.g. a violation against Section 3). For avoidance of doubt, “third party” in this clause include supervisory authorities.
+
+8.	THE MODEL IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+9.	IN NO EVENT SHALL WE BE LIABLE TO YOU FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO ANY DIRECT, OR INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING FROM YOUR USE OR INABILITY TO USE THE MODEL, IN WHOLE OR IN PART, NO MATTER HOW IT’S CAUSED OR THE LEGAL THEORY IT IS BASED ON, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+10.	HUAWEI MAY UPDATE THIS AGREEMENT AT ANY TIME TO COMPLY WITH LEGAL AND REGULATORY REQUIREMENTS, AND YOU AGREE TO ABIDE BY ANY UPDATED LICENSE OR TO CEASE COPYING, USING, MODIFYING AND DISTRIBUTING THIS MODEL AND ANY DERIVATIVE MODELS. 
+
+
+END OF THE TERMS AND CONDITIONS

OPEN SOURCE SOFTWARE NOTICE

@@ -0,0 +1,218 @@
+OPEN SOURCE SOFTWARE NOTICE
+
+Please note we provide an open source software notice along with this product and/or this product firmware (in the following just “this product”). The open source software licenses are granted by the respective right holders. And the open source licenses prevail all other license information with regard to the respective open source software contained in the product, including but not limited to End User Software Licensing Agreement. This notice is provided on behalf of Huawei Technologies Co. Ltd. and any of its local subsidiaries which may have provided this product to you in your local country. 
+
+Warranty Disclaimer    
+THE OPEN SOURCE SOFTWARE IN THIS PRODUCT IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS.
+
+Copyright Notice and License Texts 
+
+Software: transformers 5.0.0
+Copyright notice: 
+Copyright 2022 EleutherAI and the HuggingFace Inc. team. All rights reserved.
+
+License Text:
+----------------------------------------
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+

README.md

@@ -1,3 +1,30 @@
----
-license: mit
----
+# openPangu-2.0-Pro-Int8
+中文 | [English](README_EN.md)
+
+## 1. 简介
+openPangu-2.0-Pro 是基于昇腾 NPU 训练的大规模混合专家（MoE）语言模型，参数规模约 505B，每 token 激活参数规模约 18B，模型支持 512k 上下文长度，训练数据总量约34T tokens。后训练阶段完成快慢合一微调（SFT）、多专项强化学习（RL），并通过在线蒸馏（OPD）完成能力合一。
+openPangu-2.0-Pro-Int8 是 openPangu-2.0-Pro 的量化版本。
+
+## 2. 模型架构
+openPangu-2.0-Pro-Int8 在模型架构上实现了全面的升级：
+
+- Attention架构：沿用高效 MLA，并采用 DSA+SWA 独立分层混合架构，层配比为 1:2；SWA 层负责局部窗口建模，DSA 层负责稀疏全局聚合，在保持精度的同时显著降低长序列推理的计算、显存与访存开销。
+- 拓扑架构：将传统残差连接升级为 4 支流 mHC 架构，提升表征多样性与泛化能力。
+- 自投机模块：采用 3 头 MTP 架构，一次额外预测 3 个 token，显著提升模型的推理速度。
+- Muon优化器：训练中采用Muon优化器，获得更快的收敛速度。
+
+## 3. 部署和使用
+- 使用omni-infer推理框架，部署方式参考[[openPangu-2.0-Pro部署说明](https://gitcode.com/ascend-tribe/openPangu-2.0-Infer/blob/main/README.md)]
+- 源码仓:[[openPangu-2.0-Infer]](https://gitcode.com/ascend-tribe/openPangu-2.0-Infer)
+
+## 4. 模型许可证
+除文件中对开源许可证另有约定外，openPangu-2.0-Pro-Int8 模型根据 OPENPANGU MODEL LICENSE AGREEMENT VERSION 2.0 授权，旨在允许使用并促进人工智能技术的进一步发展。有关详细信息，请参阅模型存储库根目录中的 [LICENSE](LICENSE) 文件。
+
+## 5. 免责声明
+由于 openPangu-2.0-Pro-Int8 （“模型”）所依赖的技术固有的限制，以及人工智能生成的内容是由模型自动生成的，华为无法对以下事项做出任何保证：
+- 该模型的输出通过AI算法自动生成，不能排除某些信息可能存在缺陷、不合理或引起不适的可能性，生成的内容不代表华为的态度或立场； 
+- 无法保证该模型100%准确、可靠、功能齐全、及时、安全、无错误、不间断、持续稳定或无任何故障； 
+- 该模型的输出内容不构成任何建议或决策，也不保证生成的内容的真实性、完整性、准确性、及时性、合法性、功能性或实用性。生成的内容不能替代医疗、法律等领域的专业人士回答您的问题。生成的内容仅供参考，不代表华为的任何态度、立场或观点。您需要根据实际情况做出独立判断，华为不承担任何责任。
+
+## 6. 反馈
+如果有任何意见和建议，请提交issue或联系[openPangu@huawei.com](url)。

README_EN.md

@@ -0,0 +1,34 @@
+# openPangu-2.0-Pro-Int8
+[中文](README.md) | English
+
+## 1. Introduction
+openPangu-2.0-Pro is an MoE model trained on Ascend. The model has 505B total parameters and 18B activated parameters. Its context length is 512k. The total pretraining data contains 34T tokens. During Post-training, openPangu-2.0-Pro is trained through unified SFT with slow and fast thinking capability, multiple specialist RL traning, on-policy distillation combining multiple RL specialists. 
+openPangu-2.0-Pro-Int8 is a quantized version of openPangu-2.0-Pro.
+
+## 2. Architecture
+openPangu-2.0-Pro-Int8 brings several major architectural improvements:
+
+- Efficient attention: The model retains MLA for efficient inference and combines DSA and SWA in a 1:2 layer ratio. SWA layers handle local-window modeling, while DSA layers capture sparse global context. This design lowers compute, memory footprint, and memory access costs for long-context inference while preserving accuracy.
+- Residual topology: The conventional residual path is replaced with a 4-stream mHC design, improving representation diversity and generalization.
+- Multi-token prediction (MTP): The model uses three MTP heads to draft 3 additional tokens per step, enabling faster inference through self-speculative decoding.
+- Optimizer: Training uses the Muon optimizer for faster convergence.
+
+## 3. Deployment
+- omni-infer：Please refer to [[openPangu-2.0-Pro deploy guide](https://gitcode.com/ascend-tribe/openPangu-2.0-Infer/blob/main/README_EN.md)]
+- source code repo:[[openPangu-2.0-Infer]](https://gitcode.com/ascend-tribe/openPangu-2.0-Infer)
+
+## 4. Model License
+Unless otherwise noted, the openPangu-2.0-Pro-Int8 model is licensed under the terms and conditions of OPENPANGU MODEL LICENSE AGREEMENT VERSION 2.0, which is intended to be used permissively and enable the further development of artificial intelligence technologies. Please refer to the [LICENSE](LICENSE) file located in the root directory of the model repository for details.
+
+## 5. Disclaimer
+Due to the technical limitations inherent in the technology on which the openPangu-2.0-Pro-Int8 model (“Model”) relies and the fact that the artificial intelligence generated content is automatically produced by Model, Huawei cannot make any guarantees regarding the following matters:
+
+- The output of this Model is automatically generated via AI algorithms, it does not rule out the possibility that some of the information may be flawed, unreasonable, or cause discomfort, and the generated content does not represent Huawei's attitude or standpoint; 
+- There is no guarantee that this Model is 100% accurate, reliable, functional, timely, secure and safety, error-free, uninterrupted, continuously stable, or free of any faults; 
+- The output of this Model does not constitute any advices or decisions for you, and it does not guarantee the authenticity, completeness, accuracy, timeliness, legality, functionality, or practicality of the generated content. The generated content cannot replace professionals in medical, legal, and other fields in answering your questions. The generated content is for your reference only and does not represent any attitude, standpoint, or position of Huawei. You need to make independent judgments based on your actual situation, and Huawei does not assume any responsibilities.
+
+## 6. Contact
+If you have any question, please raise an issue or contact us at [openPangu@huawei.com](url).
+
+
+

config.json

@@ -0,0 +1,623 @@
+{
+  "architectures": ["OpenPanguV2ForCausalLM"],
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_openpangu_v2.OpenPanguV2Config"
+  },
+  "_attn_implementation":"eager",
+  "block_post_layernorm_idx": [0, 4, 9, 14, 19, 24, 29, 34, 39, 44],
+  "bos_token_id": 148899,
+  "dsa_layers": [0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 49],
+  "dtype": "bfloat16",
+  "eos_token_id": 148902,
+  "first_k_dense_replace": 3,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "index_head_dim": 128,
+  "index_n_heads": 32,
+  "index_topk": 2048,
+  "intermediate_size": 16128,
+  "kv_lora_rank": 512,
+  "max_position_embeddings": 524288,
+  "mhc_num_stream": 4,
+  "mhc_recur_norm": 20,
+  "mhc_use_gamma": true,
+  "model_type": "openpangu_v2",
+  "moe_intermediate_size": 1792,
+  "n_routed_experts": 384,
+  "n_shared_experts": 1,
+  "norm_topk_prob": true,
+  "num_attention_heads": 64,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 50,
+  "num_nextn_predict_layers": 3,
+  "pad_token_id": 148902,
+  "param_sink_number": 128,
+  "q_lora_rank": 1536,
+  "qk_nope_head_dim": 128,
+  "qk_rope_head_dim": 64,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 6400000,
+  "rope_interleave": false,
+  "router_enable_expert_bias": true,
+  "routed_scaling_factor": 2.5,
+  "router_sliding_window": 3,
+  "sandwich_norm": true,
+  "sliding_window": 512,
+  "sliding_window_list": [512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 2048, 2048, 2048],
+  "swa_layers": [1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19, 20, 22, 23, 25, 26, 28, 29, 31, 32, 34, 35, 37, 38, 40, 41, 43, 44, 46, 47, 50, 51, 52],
+  "tie_word_embeddings": false,
+  "transformers_version": "5.0.0",
+  "use_cache": true,
+  "use_mhc": true,
+  "use_mome": true,
+  "v_head_dim": 128,
+  "vocab_size": 151552,
+  "quantization_config": {
+    "quant_method": "compressed-tensors",
+    "quantize": "w8a8_dynamic",
+    "format": "int-quantized",
+    "quantization_status": "compressed",
+    "config_groups": {
+      "group_0": {
+        "format": "int-quantized",
+        "targets": [
+          "Linear"
+        ],
+        "weights": {
+          "type": "int",
+          "num_bits": 8,
+          "symmetric": true,
+          "strategy": "channel",
+          "dynamic": false,
+          "observer": "minmax",
+          "actorder": null,
+          "group_size": null,
+          "block_structure": null,
+          "observer_kwargs": {}
+        },
+        "input_activations": {
+          "type": "int",
+          "num_bits": 8,
+          "symmetric": true,
+          "strategy": "token",
+          "dynamic": true,
+          "actorder": null,
+          "group_size": null,
+          "block_structure": null,
+          "observer": null,
+          "observer_kwargs": {}
+        },
+        "output_activations": null
+      }
+    },
+    "ignore": [
+      "lm_head",
+      "model.embed_tokens",
+      "model.layers.0.attn_mhc_module.phi",
+      "model.layers.0.mlp_mhc_module.phi",
+      "model.layers.0.self_attn.indexer.weights_proj",
+      "model.layers.0.self_attn.indexer.wk",
+      "model.layers.0.self_attn.indexer.wq_b",
+      "model.layers.0.self_attn.kv_a_proj_with_mqa",
+      "model.layers.0.self_attn.kv_b_proj",
+      "model.layers.0.self_attn.q_a_proj",
+      "model.layers.1.attn_mhc_module.phi",
+      "model.layers.1.mlp_mhc_module.phi",
+      "model.layers.1.self_attn.kv_a_proj_with_mqa",
+      "model.layers.1.self_attn.kv_b_proj",
+      "model.layers.1.self_attn.q_a_proj",
+      "model.layers.10.attn_mhc_module.phi",
+      "model.layers.10.mlp.gate",
+      "model.layers.10.mlp.shared_experts.down_proj",
+      "model.layers.10.mlp.shared_experts.gate_proj",
+      "model.layers.10.mlp.shared_experts.up_proj",
+      "model.layers.10.mlp_mhc_module.phi",
+      "model.layers.10.self_attn.kv_a_proj_with_mqa",
+      "model.layers.10.self_attn.kv_b_proj",
+      "model.layers.10.self_attn.q_a_proj",
+      "model.layers.11.attn_mhc_module.phi",
+      "model.layers.11.mlp.gate",
+      "model.layers.11.mlp.shared_experts.down_proj",
+      "model.layers.11.mlp.shared_experts.gate_proj",
+      "model.layers.11.mlp.shared_experts.up_proj",
+      "model.layers.11.mlp_mhc_module.phi",
+      "model.layers.11.self_attn.kv_a_proj_with_mqa",
+      "model.layers.11.self_attn.kv_b_proj",
+      "model.layers.11.self_attn.q_a_proj",
+      "model.layers.12.attn_mhc_module.phi",
+      "model.layers.12.mlp.gate",
+      "model.layers.12.mlp.shared_experts.down_proj",
+      "model.layers.12.mlp.shared_experts.gate_proj",
+      "model.layers.12.mlp.shared_experts.up_proj",
+      "model.layers.12.mlp_mhc_module.phi",
+      "model.layers.12.self_attn.indexer.weights_proj",
+      "model.layers.12.self_attn.indexer.wk",
+      "model.layers.12.self_attn.indexer.wq_b",
+      "model.layers.12.self_attn.kv_a_proj_with_mqa",
+      "model.layers.12.self_attn.kv_b_proj",
+      "model.layers.12.self_attn.q_a_proj",
+      "model.layers.13.attn_mhc_module.phi",
+      "model.layers.13.mlp.gate",
+      "model.layers.13.mlp.shared_experts.down_proj",
+      "model.layers.13.mlp.shared_experts.gate_proj",
+      "model.layers.13.mlp.shared_experts.up_proj",
+      "model.layers.13.mlp_mhc_module.phi",
+      "model.layers.13.self_attn.kv_a_proj_with_mqa",
+      "model.layers.13.self_attn.kv_b_proj",
+      "model.layers.13.self_attn.q_a_proj",
+      "model.layers.14.attn_mhc_module.phi",
+      "model.layers.14.mlp.gate",
+      "model.layers.14.mlp.shared_experts.down_proj",
+      "model.layers.14.mlp.shared_experts.gate_proj",
+      "model.layers.14.mlp.shared_experts.up_proj",
+      "model.layers.14.mlp_mhc_module.phi",
+      "model.layers.14.self_attn.kv_a_proj_with_mqa",
+      "model.layers.14.self_attn.kv_b_proj",
+      "model.layers.14.self_attn.q_a_proj",
+      "model.layers.15.attn_mhc_module.phi",
+      "model.layers.15.mlp.gate",
+      "model.layers.15.mlp.shared_experts.down_proj",
+      "model.layers.15.mlp.shared_experts.gate_proj",
+      "model.layers.15.mlp.shared_experts.up_proj",
+      "model.layers.15.mlp_mhc_module.phi",
+      "model.layers.15.self_attn.indexer.weights_proj",
+      "model.layers.15.self_attn.indexer.wk",
+      "model.layers.15.self_attn.indexer.wq_b",
+      "model.layers.15.self_attn.kv_a_proj_with_mqa",
+      "model.layers.15.self_attn.kv_b_proj",
+      "model.layers.15.self_attn.q_a_proj",
+      "model.layers.16.attn_mhc_module.phi",
+      "model.layers.16.mlp.gate",
+      "model.layers.16.mlp.shared_experts.down_proj",
+      "model.layers.16.mlp.shared_experts.gate_proj",
+      "model.layers.16.mlp.shared_experts.up_proj",
+      "model.layers.16.mlp_mhc_module.phi",
+      "model.layers.16.self_attn.kv_a_proj_with_mqa",
+      "model.layers.16.self_attn.kv_b_proj",
+      "model.layers.16.self_attn.q_a_proj",
+      "model.layers.17.attn_mhc_module.phi",
+      "model.layers.17.mlp.gate",
+      "model.layers.17.mlp.shared_experts.down_proj",
+      "model.layers.17.mlp.shared_experts.gate_proj",
+      "model.layers.17.mlp.shared_experts.up_proj",
+      "model.layers.17.mlp_mhc_module.phi",
+      "model.layers.17.self_attn.kv_a_proj_with_mqa",
+      "model.layers.17.self_attn.kv_b_proj",
+      "model.layers.17.self_attn.q_a_proj",
+      "model.layers.18.attn_mhc_module.phi",
+      "model.layers.18.mlp.gate",
+      "model.layers.18.mlp.shared_experts.down_proj",
+      "model.layers.18.mlp.shared_experts.gate_proj",
+      "model.layers.18.mlp.shared_experts.up_proj",
+      "model.layers.18.mlp_mhc_module.phi",
+      "model.layers.18.self_attn.indexer.weights_proj",
+      "model.layers.18.self_attn.indexer.wk",
+      "model.layers.18.self_attn.indexer.wq_b",
+      "model.layers.18.self_attn.kv_a_proj_with_mqa",
+      "model.layers.18.self_attn.kv_b_proj",
+      "model.layers.18.self_attn.q_a_proj",
+      "model.layers.19.attn_mhc_module.phi",
+      "model.layers.19.mlp.gate",
+      "model.layers.19.mlp.shared_experts.down_proj",
+      "model.layers.19.mlp.shared_experts.gate_proj",
+      "model.layers.19.mlp.shared_experts.up_proj",
+      "model.layers.19.mlp_mhc_module.phi",
+      "model.layers.19.self_attn.kv_a_proj_with_mqa",
+      "model.layers.19.self_attn.kv_b_proj",
+      "model.layers.19.self_attn.q_a_proj",
+      "model.layers.2.attn_mhc_module.phi",
+      "model.layers.2.mlp_mhc_module.phi",
+      "model.layers.2.self_attn.kv_a_proj_with_mqa",
+      "model.layers.2.self_attn.kv_b_proj",
+      "model.layers.2.self_attn.q_a_proj",
+      "model.layers.20.attn_mhc_module.phi",
+      "model.layers.20.mlp.gate",
+      "model.layers.20.mlp.shared_experts.down_proj",
+      "model.layers.20.mlp.shared_experts.gate_proj",
+      "model.layers.20.mlp.shared_experts.up_proj",
+      "model.layers.20.mlp_mhc_module.phi",
+      "model.layers.20.self_attn.kv_a_proj_with_mqa",
+      "model.layers.20.self_attn.kv_b_proj",
+      "model.layers.20.self_attn.q_a_proj",
+      "model.layers.21.attn_mhc_module.phi",
+      "model.layers.21.mlp.gate",
+      "model.layers.21.mlp.shared_experts.down_proj",
+      "model.layers.21.mlp.shared_experts.gate_proj",
+      "model.layers.21.mlp.shared_experts.up_proj",
+      "model.layers.21.mlp_mhc_module.phi",
+      "model.layers.21.self_attn.indexer.weights_proj",
+      "model.layers.21.self_attn.indexer.wk",
+      "model.layers.21.self_attn.indexer.wq_b",
+      "model.layers.21.self_attn.kv_a_proj_with_mqa",
+      "model.layers.21.self_attn.kv_b_proj",
+      "model.layers.21.self_attn.q_a_proj",
+      "model.layers.22.attn_mhc_module.phi",
+      "model.layers.22.mlp.gate",
+      "model.layers.22.mlp.shared_experts.down_proj",
+      "model.layers.22.mlp.shared_experts.gate_proj",
+      "model.layers.22.mlp.shared_experts.up_proj",
+      "model.layers.22.mlp_mhc_module.phi",
+      "model.layers.22.self_attn.kv_a_proj_with_mqa",
+      "model.layers.22.self_attn.kv_b_proj",
+      "model.layers.22.self_attn.q_a_proj",
+      "model.layers.23.attn_mhc_module.phi",
+      "model.layers.23.mlp.gate",
+      "model.layers.23.mlp.shared_experts.down_proj",
+      "model.layers.23.mlp.shared_experts.gate_proj",
+      "model.layers.23.mlp.shared_experts.up_proj",
+      "model.layers.23.mlp_mhc_module.phi",
+      "model.layers.23.self_attn.kv_a_proj_with_mqa",
+      "model.layers.23.self_attn.kv_b_proj",
+      "model.layers.23.self_attn.q_a_proj",
+      "model.layers.24.attn_mhc_module.phi",
+      "model.layers.24.mlp.gate",
+      "model.layers.24.mlp.shared_experts.down_proj",
+      "model.layers.24.mlp.shared_experts.gate_proj",
+      "model.layers.24.mlp.shared_experts.up_proj",
+      "model.layers.24.mlp_mhc_module.phi",
+      "model.layers.24.self_attn.indexer.weights_proj",
+      "model.layers.24.self_attn.indexer.wk",
+      "model.layers.24.self_attn.indexer.wq_b",
+      "model.layers.24.self_attn.kv_a_proj_with_mqa",
+      "model.layers.24.self_attn.kv_b_proj",
+      "model.layers.24.self_attn.q_a_proj",
+      "model.layers.25.attn_mhc_module.phi",
+      "model.layers.25.mlp.gate",
+      "model.layers.25.mlp.shared_experts.down_proj",
+      "model.layers.25.mlp.shared_experts.gate_proj",
+      "model.layers.25.mlp.shared_experts.up_proj",
+      "model.layers.25.mlp_mhc_module.phi",
+      "model.layers.25.self_attn.kv_a_proj_with_mqa",
+      "model.layers.25.self_attn.kv_b_proj",
+      "model.layers.25.self_attn.q_a_proj",
+      "model.layers.26.attn_mhc_module.phi",
+      "model.layers.26.mlp.gate",
+      "model.layers.26.mlp.shared_experts.down_proj",
+      "model.layers.26.mlp.shared_experts.gate_proj",
+      "model.layers.26.mlp.shared_experts.up_proj",
+      "model.layers.26.mlp_mhc_module.phi",
+      "model.layers.26.self_attn.kv_a_proj_with_mqa",
+      "model.layers.26.self_attn.kv_b_proj",
+      "model.layers.26.self_attn.q_a_proj",
+      "model.layers.27.attn_mhc_module.phi",
+      "model.layers.27.mlp.gate",
+      "model.layers.27.mlp.shared_experts.down_proj",
+      "model.layers.27.mlp.shared_experts.gate_proj",
+      "model.layers.27.mlp.shared_experts.up_proj",
+      "model.layers.27.mlp_mhc_module.phi",
+      "model.layers.27.self_attn.indexer.weights_proj",
+      "model.layers.27.self_attn.indexer.wk",
+      "model.layers.27.self_attn.indexer.wq_b",
+      "model.layers.27.self_attn.kv_a_proj_with_mqa",
+      "model.layers.27.self_attn.kv_b_proj",
+      "model.layers.27.self_attn.q_a_proj",
+      "model.layers.28.attn_mhc_module.phi",
+      "model.layers.28.mlp.gate",
+      "model.layers.28.mlp.shared_experts.down_proj",
+      "model.layers.28.mlp.shared_experts.gate_proj",
+      "model.layers.28.mlp.shared_experts.up_proj",
+      "model.layers.28.mlp_mhc_module.phi",
+      "model.layers.28.self_attn.kv_a_proj_with_mqa",
+      "model.layers.28.self_attn.kv_b_proj",
+      "model.layers.28.self_attn.q_a_proj",
+      "model.layers.29.attn_mhc_module.phi",
+      "model.layers.29.mlp.gate",
+      "model.layers.29.mlp.shared_experts.down_proj",
+      "model.layers.29.mlp.shared_experts.gate_proj",
+      "model.layers.29.mlp.shared_experts.up_proj",
+      "model.layers.29.mlp_mhc_module.phi",
+      "model.layers.29.self_attn.kv_a_proj_with_mqa",
+      "model.layers.29.self_attn.kv_b_proj",
+      "model.layers.29.self_attn.q_a_proj",
+      "model.layers.3.attn_mhc_module.phi",
+      "model.layers.3.mlp.gate",
+      "model.layers.3.mlp.shared_experts.down_proj",
+      "model.layers.3.mlp.shared_experts.gate_proj",
+      "model.layers.3.mlp.shared_experts.up_proj",
+      "model.layers.3.mlp_mhc_module.phi",
+      "model.layers.3.self_attn.indexer.weights_proj",
+      "model.layers.3.self_attn.indexer.wk",
+      "model.layers.3.self_attn.indexer.wq_b",
+      "model.layers.3.self_attn.kv_a_proj_with_mqa",
+      "model.layers.3.self_attn.kv_b_proj",
+      "model.layers.3.self_attn.q_a_proj",
+      "model.layers.30.attn_mhc_module.phi",
+      "model.layers.30.mlp.gate",
+      "model.layers.30.mlp.shared_experts.down_proj",
+      "model.layers.30.mlp.shared_experts.gate_proj",
+      "model.layers.30.mlp.shared_experts.up_proj",
+      "model.layers.30.mlp_mhc_module.phi",
+      "model.layers.30.self_attn.indexer.weights_proj",
+      "model.layers.30.self_attn.indexer.wk",
+      "model.layers.30.self_attn.indexer.wq_b",
+      "model.layers.30.self_attn.kv_a_proj_with_mqa",
+      "model.layers.30.self_attn.kv_b_proj",
+      "model.layers.30.self_attn.q_a_proj",
+      "model.layers.31.attn_mhc_module.phi",
+      "model.layers.31.mlp.gate",
+      "model.layers.31.mlp.shared_experts.down_proj",
+      "model.layers.31.mlp.shared_experts.gate_proj",
+      "model.layers.31.mlp.shared_experts.up_proj",
+      "model.layers.31.mlp_mhc_module.phi",
+      "model.layers.31.self_attn.kv_a_proj_with_mqa",
+      "model.layers.31.self_attn.kv_b_proj",
+      "model.layers.31.self_attn.q_a_proj",
+      "model.layers.32.attn_mhc_module.phi",
+      "model.layers.32.mlp.gate",
+      "model.layers.32.mlp.shared_experts.down_proj",
+      "model.layers.32.mlp.shared_experts.gate_proj",
+      "model.layers.32.mlp.shared_experts.up_proj",
+      "model.layers.32.mlp_mhc_module.phi",
+      "model.layers.32.self_attn.kv_a_proj_with_mqa",
+      "model.layers.32.self_attn.kv_b_proj",
+      "model.layers.32.self_attn.q_a_proj",
+      "model.layers.33.attn_mhc_module.phi",
+      "model.layers.33.mlp.gate",
+      "model.layers.33.mlp.shared_experts.down_proj",
+      "model.layers.33.mlp.shared_experts.gate_proj",
+      "model.layers.33.mlp.shared_experts.up_proj",
+      "model.layers.33.mlp_mhc_module.phi",
+      "model.layers.33.self_attn.indexer.weights_proj",
+      "model.layers.33.self_attn.indexer.wk",
+      "model.layers.33.self_attn.indexer.wq_b",
+      "model.layers.33.self_attn.kv_a_proj_with_mqa",
+      "model.layers.33.self_attn.kv_b_proj",
+      "model.layers.33.self_attn.q_a_proj",
+      "model.layers.34.attn_mhc_module.phi",
+      "model.layers.34.mlp.gate",
+      "model.layers.34.mlp.shared_experts.down_proj",
+      "model.layers.34.mlp.shared_experts.gate_proj",
+      "model.layers.34.mlp.shared_experts.up_proj",
+      "model.layers.34.mlp_mhc_module.phi",
+      "model.layers.34.self_attn.kv_a_proj_with_mqa",
+      "model.layers.34.self_attn.kv_b_proj",
+      "model.layers.34.self_attn.q_a_proj",
+      "model.layers.35.attn_mhc_module.phi",
+      "model.layers.35.mlp.gate",
+      "model.layers.35.mlp.shared_experts.down_proj",
+      "model.layers.35.mlp.shared_experts.gate_proj",
+      "model.layers.35.mlp.shared_experts.up_proj",
+      "model.layers.35.mlp_mhc_module.phi",
+      "model.layers.35.self_attn.kv_a_proj_with_mqa",
+      "model.layers.35.self_attn.kv_b_proj",
+      "model.layers.35.self_attn.q_a_proj",
+      "model.layers.36.attn_mhc_module.phi",
+      "model.layers.36.mlp.gate",
+      "model.layers.36.mlp.shared_experts.down_proj",
+      "model.layers.36.mlp.shared_experts.gate_proj",
+      "model.layers.36.mlp.shared_experts.up_proj",
+      "model.layers.36.mlp_mhc_module.phi",
+      "model.layers.36.self_attn.indexer.weights_proj",
+      "model.layers.36.self_attn.indexer.wk",
+      "model.layers.36.self_attn.indexer.wq_b",
+      "model.layers.36.self_attn.kv_a_proj_with_mqa",
+      "model.layers.36.self_attn.kv_b_proj",
+      "model.layers.36.self_attn.q_a_proj",
+      "model.layers.37.attn_mhc_module.phi",
+      "model.layers.37.mlp.gate",
+      "model.layers.37.mlp.shared_experts.down_proj",
+      "model.layers.37.mlp.shared_experts.gate_proj",
+      "model.layers.37.mlp.shared_experts.up_proj",
+      "model.layers.37.mlp_mhc_module.phi",
+      "model.layers.37.self_attn.kv_a_proj_with_mqa",
+      "model.layers.37.self_attn.kv_b_proj",
+      "model.layers.37.self_attn.q_a_proj",
+      "model.layers.38.attn_mhc_module.phi",
+      "model.layers.38.mlp.gate",
+      "model.layers.38.mlp.shared_experts.down_proj",
+      "model.layers.38.mlp.shared_experts.gate_proj",
+      "model.layers.38.mlp.shared_experts.up_proj",
+      "model.layers.38.mlp_mhc_module.phi",
+      "model.layers.38.self_attn.kv_a_proj_with_mqa",
+      "model.layers.38.self_attn.kv_b_proj",
+      "model.layers.38.self_attn.q_a_proj",
+      "model.layers.39.attn_mhc_module.phi",
+      "model.layers.39.mlp.gate",
+      "model.layers.39.mlp.shared_experts.down_proj",
+      "model.layers.39.mlp.shared_experts.gate_proj",
+      "model.layers.39.mlp.shared_experts.up_proj",
+      "model.layers.39.mlp_mhc_module.phi",
+      "model.layers.39.self_attn.indexer.weights_proj",
+      "model.layers.39.self_attn.indexer.wk",
+      "model.layers.39.self_attn.indexer.wq_b",
+      "model.layers.39.self_attn.kv_a_proj_with_mqa",
+      "model.layers.39.self_attn.kv_b_proj",
+      "model.layers.39.self_attn.q_a_proj",
+      "model.layers.4.attn_mhc_module.phi",
+      "model.layers.4.mlp.gate",
+      "model.layers.4.mlp.shared_experts.down_proj",
+      "model.layers.4.mlp.shared_experts.gate_proj",
+      "model.layers.4.mlp.shared_experts.up_proj",
+      "model.layers.4.mlp_mhc_module.phi",
+      "model.layers.4.self_attn.kv_a_proj_with_mqa",
+      "model.layers.4.self_attn.kv_b_proj",
+      "model.layers.4.self_attn.q_a_proj",
+      "model.layers.40.attn_mhc_module.phi",
+      "model.layers.40.mlp.gate",
+      "model.layers.40.mlp.shared_experts.down_proj",
+      "model.layers.40.mlp.shared_experts.gate_proj",
+      "model.layers.40.mlp.shared_experts.up_proj",
+      "model.layers.40.mlp_mhc_module.phi",
+      "model.layers.40.self_attn.kv_a_proj_with_mqa",
+      "model.layers.40.self_attn.kv_b_proj",
+      "model.layers.40.self_attn.q_a_proj",
+      "model.layers.41.attn_mhc_module.phi",
+      "model.layers.41.mlp.gate",
+      "model.layers.41.mlp.shared_experts.down_proj",
+      "model.layers.41.mlp.shared_experts.gate_proj",
+      "model.layers.41.mlp.shared_experts.up_proj",
+      "model.layers.41.mlp_mhc_module.phi",
+      "model.layers.41.self_attn.kv_a_proj_with_mqa",
+      "model.layers.41.self_attn.kv_b_proj",
+      "model.layers.41.self_attn.q_a_proj",
+      "model.layers.42.attn_mhc_module.phi",
+      "model.layers.42.mlp.gate",
+      "model.layers.42.mlp.shared_experts.down_proj",
+      "model.layers.42.mlp.shared_experts.gate_proj",
+      "model.layers.42.mlp.shared_experts.up_proj",
+      "model.layers.42.mlp_mhc_module.phi",
+      "model.layers.42.self_attn.indexer.weights_proj",
+      "model.layers.42.self_attn.indexer.wk",
+      "model.layers.42.self_attn.indexer.wq_b",
+      "model.layers.42.self_attn.kv_a_proj_with_mqa",
+      "model.layers.42.self_attn.kv_b_proj",
+      "model.layers.42.self_attn.q_a_proj",
+      "model.layers.43.attn_mhc_module.phi",
+      "model.layers.43.mlp.gate",
+      "model.layers.43.mlp.shared_experts.down_proj",
+      "model.layers.43.mlp.shared_experts.gate_proj",
+      "model.layers.43.mlp.shared_experts.up_proj",
+      "model.layers.43.mlp_mhc_module.phi",
+      "model.layers.43.self_attn.kv_a_proj_with_mqa",
+      "model.layers.43.self_attn.kv_b_proj",
+      "model.layers.43.self_attn.q_a_proj",
+      "model.layers.44.attn_mhc_module.phi",
+      "model.layers.44.mlp.gate",
+      "model.layers.44.mlp.shared_experts.down_proj",
+      "model.layers.44.mlp.shared_experts.gate_proj",
+      "model.layers.44.mlp.shared_experts.up_proj",
+      "model.layers.44.mlp_mhc_module.phi",
+      "model.layers.44.self_attn.kv_a_proj_with_mqa",
+      "model.layers.44.self_attn.kv_b_proj",
+      "model.layers.44.self_attn.q_a_proj",
+      "model.layers.45.attn_mhc_module.phi",
+      "model.layers.45.mlp.gate",
+      "model.layers.45.mlp.shared_experts.down_proj",
+      "model.layers.45.mlp.shared_experts.gate_proj",
+      "model.layers.45.mlp.shared_experts.up_proj",
+      "model.layers.45.mlp_mhc_module.phi",
+      "model.layers.45.self_attn.indexer.weights_proj",
+      "model.layers.45.self_attn.indexer.wk",
+      "model.layers.45.self_attn.indexer.wq_b",
+      "model.layers.45.self_attn.kv_a_proj_with_mqa",
+      "model.layers.45.self_attn.kv_b_proj",
+      "model.layers.45.self_attn.q_a_proj",
+      "model.layers.46.attn_mhc_module.phi",
+      "model.layers.46.mlp.gate",
+      "model.layers.46.mlp.shared_experts.down_proj",
+      "model.layers.46.mlp.shared_experts.gate_proj",
+      "model.layers.46.mlp.shared_experts.up_proj",
+      "model.layers.46.mlp_mhc_module.phi",
+      "model.layers.46.self_attn.kv_a_proj_with_mqa",
+      "model.layers.46.self_attn.kv_b_proj",
+      "model.layers.46.self_attn.q_a_proj",
+      "model.layers.47.attn_mhc_module.phi",
+      "model.layers.47.mlp.gate",
+      "model.layers.47.mlp.shared_experts.down_proj",
+      "model.layers.47.mlp.shared_experts.gate_proj",
+      "model.layers.47.mlp.shared_experts.up_proj",
+      "model.layers.47.mlp_mhc_module.phi",
+      "model.layers.47.self_attn.kv_a_proj_with_mqa",
+      "model.layers.47.self_attn.kv_b_proj",
+      "model.layers.47.self_attn.q_a_proj",
+      "model.layers.48.attn_mhc_module.phi",
+      "model.layers.48.mlp.gate",
+      "model.layers.48.mlp.shared_experts.down_proj",
+      "model.layers.48.mlp.shared_experts.gate_proj",
+      "model.layers.48.mlp.shared_experts.up_proj",
+      "model.layers.48.mlp_mhc_module.phi",
+      "model.layers.48.self_attn.indexer.weights_proj",
+      "model.layers.48.self_attn.indexer.wk",
+      "model.layers.48.self_attn.indexer.wq_b",
+      "model.layers.48.self_attn.kv_a_proj_with_mqa",
+      "model.layers.48.self_attn.kv_b_proj",
+      "model.layers.48.self_attn.q_a_proj",
+      "model.layers.49.attn_mhc_module.phi",
+      "model.layers.49.mlp.gate",
+      "model.layers.49.mlp.shared_experts.down_proj",
+      "model.layers.49.mlp.shared_experts.gate_proj",
+      "model.layers.49.mlp.shared_experts.up_proj",
+      "model.layers.49.mlp_mhc_module.phi",
+      "model.layers.49.self_attn.indexer.weights_proj",
+      "model.layers.49.self_attn.indexer.wk",
+      "model.layers.49.self_attn.indexer.wq_b",
+      "model.layers.49.self_attn.kv_a_proj_with_mqa",
+      "model.layers.49.self_attn.kv_b_proj",
+      "model.layers.49.self_attn.q_a_proj",
+      "model.layers.5.attn_mhc_module.phi",
+      "model.layers.5.mlp.gate",
+      "model.layers.5.mlp.shared_experts.down_proj",
+      "model.layers.5.mlp.shared_experts.gate_proj",
+      "model.layers.5.mlp.shared_experts.up_proj",
+      "model.layers.5.mlp_mhc_module.phi",
+      "model.layers.5.self_attn.kv_a_proj_with_mqa",
+      "model.layers.5.self_attn.kv_b_proj",
+      "model.layers.5.self_attn.q_a_proj",
+      "model.layers.50.embed_tokens",
+      "model.layers.50.mlp.gate",
+      "model.layers.50.mlp.shared_experts.down_proj",
+      "model.layers.50.mlp.shared_experts.gate_proj",
+      "model.layers.50.mlp.shared_experts.up_proj",
+      "model.layers.50.self_attn.kv_a_proj_with_mqa",
+      "model.layers.50.self_attn.kv_b_proj",
+      "model.layers.50.self_attn.q_a_proj",
+      "model.layers.50.shared_head.head",
+      "model.layers.51.embed_tokens",
+      "model.layers.51.mlp.gate",
+      "model.layers.51.mlp.shared_experts.down_proj",
+      "model.layers.51.mlp.shared_experts.gate_proj",
+      "model.layers.51.mlp.shared_experts.up_proj",
+      "model.layers.51.self_attn.kv_a_proj_with_mqa",
+      "model.layers.51.self_attn.kv_b_proj",
+      "model.layers.51.self_attn.q_a_proj",
+      "model.layers.51.shared_head.head",
+      "model.layers.52.embed_tokens",
+      "model.layers.52.mlp.gate",
+      "model.layers.52.mlp.shared_experts.down_proj",
+      "model.layers.52.mlp.shared_experts.gate_proj",
+      "model.layers.52.mlp.shared_experts.up_proj",
+      "model.layers.52.self_attn.kv_a_proj_with_mqa",
+      "model.layers.52.self_attn.kv_b_proj",
+      "model.layers.52.self_attn.q_a_proj",
+      "model.layers.52.shared_head.head",
+      "model.layers.6.attn_mhc_module.phi",
+      "model.layers.6.mlp.gate",
+      "model.layers.6.mlp.shared_experts.down_proj",
+      "model.layers.6.mlp.shared_experts.gate_proj",
+      "model.layers.6.mlp.shared_experts.up_proj",
+      "model.layers.6.mlp_mhc_module.phi",
+      "model.layers.6.self_attn.indexer.weights_proj",
+      "model.layers.6.self_attn.indexer.wk",
+      "model.layers.6.self_attn.indexer.wq_b",
+      "model.layers.6.self_attn.kv_a_proj_with_mqa",
+      "model.layers.6.self_attn.kv_b_proj",
+      "model.layers.6.self_attn.q_a_proj",
+      "model.layers.7.attn_mhc_module.phi",
+      "model.layers.7.mlp.gate",
+      "model.layers.7.mlp.shared_experts.down_proj",
+      "model.layers.7.mlp.shared_experts.gate_proj",
+      "model.layers.7.mlp.shared_experts.up_proj",
+      "model.layers.7.mlp_mhc_module.phi",
+      "model.layers.7.self_attn.kv_a_proj_with_mqa",
+      "model.layers.7.self_attn.kv_b_proj",
+      "model.layers.7.self_attn.q_a_proj",
+      "model.layers.8.attn_mhc_module.phi",
+      "model.layers.8.mlp.gate",
+      "model.layers.8.mlp.shared_experts.down_proj",
+      "model.layers.8.mlp.shared_experts.gate_proj",
+      "model.layers.8.mlp.shared_experts.up_proj",
+      "model.layers.8.mlp_mhc_module.phi",
+      "model.layers.8.self_attn.kv_a_proj_with_mqa",
+      "model.layers.8.self_attn.kv_b_proj",
+      "model.layers.8.self_attn.q_a_proj",
+      "model.layers.9.attn_mhc_module.phi",
+      "model.layers.9.mlp.gate",
+      "model.layers.9.mlp.shared_experts.down_proj",
+      "model.layers.9.mlp.shared_experts.gate_proj",
+      "model.layers.9.mlp.shared_experts.up_proj",
+      "model.layers.9.mlp_mhc_module.phi",
+      "model.layers.9.self_attn.indexer.weights_proj",
+      "model.layers.9.self_attn.indexer.wk",
+      "model.layers.9.self_attn.indexer.wq_b",
+      "model.layers.9.self_attn.kv_a_proj_with_mqa",
+      "model.layers.9.self_attn.kv_b_proj",
+      "model.layers.9.self_attn.q_a_proj",
+      "model.merge_mhc_module.phi"
+    ],
+    "sparsity_config": {},
+    "transform_config": {},
+    "global_compression_ratio": 1.9661,
+    "kv_cache_scheme": null
+  }
+}

configuration_openpangu_v2.py

@@ -0,0 +1,191 @@
+# coding=utf-8
+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+""" OpenPangu_v2 model configuration"""
+
+from transformers.configuration_utils import PretrainedConfig
+from transformers.utils import logging
+
+try:
+    # transformers >= 5.0 exports RopeParameters as a TypedDict.
+    from transformers.modeling_rope_utils import RopeParameters
+except ImportError:
+    # transformers < 5.0 has no RopeParameters. Provide a TypedDict with the
+    # same fields so the `rope_parameters` annotation still resolves. It is
+    # only used for typing -- `rope_parameters` defaults to None and is not
+    # exercised by config.json, so this fallback never affects behavior.
+    from typing import TypedDict
+
+    class RopeParameters(TypedDict, total=False):
+        rope_theta: float
+        rope_type: str | None
+        partial_rotary_factor: float | None
+        factor: float | None
+        original_max_position_embeddings: int | None
+        attention_factor: float | None
+        beta_fast: float | None
+        beta_slow: float | None
+        short_factor: list[float] | None
+        long_factor: list[float] | None
+        low_freq_factor: float | None
+        high_freq_factor: float | None
+
+logger = logging.get_logger(__name__)
+
+
+class OpenPanguV2Config(PretrainedConfig):
+    model_type = "openpangu_v2"
+    keys_to_ignore_at_inference = ["past_key_values"]
+
+    # Default tensor parallel plan for base model `OpenPangu_v2`
+    base_model_tp_plan = {
+        "layers.*.self_attn.q_proj": "colwise",
+        "layers.*.self_attn.k_proj": "colwise",
+        "layers.*.self_attn.v_proj": "colwise",
+        "layers.*.self_attn.o_proj": "rowwise",
+        "layers.*.mlp.gate_proj": "colwise",
+        "layers.*.mlp.up_proj": "colwise",
+        "layers.*.mlp.down_proj": "rowwise",
+        "layers.*.mlp.experts.gate_up_proj": "rowwise",
+        "layers.*.mlp.experts.down_proj": "rowwise",
+        "layers.*.mlp.shared_experts.gate_proj": "colwise",
+        "layers.*.mlp.shared_experts.up_proj": "colwise",
+        "layers.*.mlp.shared_experts.down_proj": "rowwise",
+    }
+    base_model_pp_plan = {
+        "embed_tokens": (["input_ids"], ["inputs_embeds"]),
+        "layers": (["hidden_states", "attention_mask"], ["hidden_states"]),
+        "norm": (["hidden_states"], ["hidden_states"]),
+    }
+
+    def __init__(
+        self,
+        attention_dropout: float | None = 0.0,
+        block_post_layernorm_idx: list[int] | None = None,
+        dsa_layers: list[str] | None = None,
+        first_k_dense_replace: int | None = 0,
+        head_dim: int | None = None,
+        hidden_act: str | None = "silu",
+        hidden_size: int | None = None,
+        index_head_dim: int | None = None,
+        index_n_heads: int | None = None,
+        index_topk: int | None = None,
+        intermediate_size: int | None = None,
+        kv_lora_rank: int | None = None,
+        layer_types: list[str] | None = None,
+        max_position_embeddings: int | None = None,
+        mhc_num_stream: int | None = None,
+        mhc_recur_norm: int | None = None,
+        mhc_use_gamma: bool | None = None,
+        moe_intermediate_size: int | None = None,
+        n_routed_experts: int | None = None,
+        n_shared_experts: int | None = None,
+        norm_topk_prob: bool | None = None,
+        num_attention_heads: int | None = None,
+        num_experts_per_tok: int | None = None,
+        num_hidden_layers: int | None = 0,
+        num_key_value_heads: int | None = None,
+        param_sink_number: int | None = 0,
+        q_lora_rank: int | None = None,
+        qk_nope_head_dim: int | None = None,
+        qk_rope_head_dim: int | None = None,
+        rms_norm_eps: float | None = 1e-5,
+        rope_interleave: bool | None = False,
+        rope_parameters: RopeParameters | dict[str, RopeParameters] | None = None,
+        routed_scaling_factor: float | None = None,
+        router_sliding_window: int | None = 0,
+        sandwich_norm: bool | None = False,
+        sliding_window: int | list[int] | None = None,
+        swa_layers: list[str] | None = None,
+        use_cache: bool | None = True,
+        use_mhc: bool | None = False,
+        v_head_dim: int | None = None,
+        vocab_size: int | None = None,
+        **kwargs,
+    ):
+        self.vocab_size = vocab_size
+        self.max_position_embeddings = max_position_embeddings
+        self.hidden_size = hidden_size
+        self.intermediate_size = intermediate_size
+        self.moe_intermediate_size = moe_intermediate_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        # for backward compatibility
+        if num_key_value_heads is None:
+            num_key_value_heads = num_attention_heads
+        self.num_key_value_heads = num_key_value_heads
+        self.head_dim = head_dim
+        self.v_head_dim = v_head_dim
+        self.hidden_act = hidden_act
+        self.rms_norm_eps = rms_norm_eps
+        self.use_cache = use_cache
+        self.rope_parameters = rope_parameters
+        self.attention_dropout = attention_dropout
+        self.layer_types = layer_types
+
+        self.n_shared_experts = n_shared_experts
+        self.n_routed_experts = n_routed_experts
+        self.routed_scaling_factor = routed_scaling_factor
+        self.kv_lora_rank = kv_lora_rank
+        self.q_lora_rank = q_lora_rank
+        self.qk_rope_head_dim = qk_rope_head_dim
+        self.qk_nope_head_dim = qk_nope_head_dim
+        if qk_rope_head_dim is not None and qk_nope_head_dim is not None:
+            self.head_dim = qk_rope_head_dim
+            self.qk_head_dim = qk_nope_head_dim + qk_rope_head_dim
+        self.num_experts_per_tok = num_experts_per_tok
+        self.first_k_dense_replace = first_k_dense_replace
+        self.norm_topk_prob = norm_topk_prob
+        self.rope_interleave = rope_interleave
+
+        self.sliding_window = sliding_window
+        self.swa_layers = swa_layers
+
+        self.param_sink_number = param_sink_number
+        self.router_sliding_window = router_sliding_window
+        self.sandwich_norm = sandwich_norm
+        self.block_post_layernorm_idx = block_post_layernorm_idx
+        self.use_mhc = use_mhc
+        self.mhc_use_gamma = mhc_use_gamma
+        self.mhc_recur_norm = mhc_recur_norm
+        self.mhc_num_stream = mhc_num_stream
+        
+        # Indexer (DSA) parameters
+        self.dsa_layers = dsa_layers
+        self.index_topk = index_topk
+        self.index_head_dim = index_head_dim
+        self.index_n_heads = index_n_heads
+        
+        if self.layer_types is None:
+            if self.swa_layers is not None:
+                self.layer_types = [
+                    "sliding_attention"
+                    if i in self.swa_layers
+                    else "full_attention"
+                    for i in range(self.num_hidden_layers)
+                ]
+            else:
+                self.layer_types = ["full_attention" for _ in range(self.num_hidden_layers)]
+        
+        if num_hidden_layers is not None and self.layer_types is not None and len(self.layer_types) != num_hidden_layers:
+            raise ValueError(
+                f"`num_hidden_layers` ({num_hidden_layers}) must be equal to the number of layer types "
+                f"({len(layer_types)})"
+            )
+
+        super().__init__(**kwargs)
+
+
+__all__ = ["OpenPanguV2Config"]

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "eos_token_id": [148902],
+  "temperature": 1.0,
+  "top_p": 0.8,
+  "top_k": 151552,
+  "seed": 1234
+}

model-00001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b39f4f900cf256290295a26ae8248a00ced1664cab78d839698176aa869d290
+size 4655677784

model-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edb85a65dfb2d117eef54ff165a41b1759266e9a36e80767d7401dde470af282
+size 2878764124

model-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:732544017fa9a804b5dd966b7c6a25475ffb13670f53b22813fafadea8cb399a
+size 2000992336

model-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:825bbb44cc878fae49847cf6f0557a2ceea0887b032fd709970e0921dd8ad398
+size 2000992712

model-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df3daebe8ea4fdf81f92f7aedaf239ee8317beb62ea8a89ee04ffaa7969eedc4
+size 2000992712

model-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:434abbe3be7f3e04008148fac570758c2373369fb2568c5d22fcefa05d854e9f
+size 2001791608

model-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35094cc5bd919cbf0342c0d29ae61a85b5266e95b3f5737bd7311ff7037018a5
+size 2002444440

model-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:befccbc9c3f533acaa69d670052912e448121759da3445dd28fc5f03229175dd
+size 2051698892

model-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a608f1c39b0a3a6b94f66ab2dc0a1437713cbd31692f88b8be76d1e2d6b1620c
+size 2000992632

model-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df779fc9e3e6a64a0ae8f6346e9b4e8960f212ea1bcf71d1ce29bfca9844c657
+size 2000992712

model-00011.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33a7ca72ff3b77945457244d30a5cdd0a4c55118668b2c739bebc661c6130ee1
+size 2001298928

model-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f38140dcd39663c6ebcf2bc1576d7974350ead7fd806e9c5d0bcb83db47391b
+size 2002444336

model-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24ad38a7ec9965ca98473d281cfda20032dc3a6e75f9bd729ffc44831f1bbfd8
+size 2052232652

model-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9b0d6311eaf21a9192ee0fba69a4f0ef5e0c28a53770361fe5022d9531d5985
+size 2000992488

model-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8aad9ec61444bed8552640d62852860c88c8e54a82c23084d218975dabc4023
+size 2000992712

model-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17fd92f09889bd1fabe2142e03252a601acc01ba1a477b4a372ba17291d9e9b1
+size 2000992712

model-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6100532b699274753f97b111762f15a02d1c3070870d8080f74686b24916cb3
+size 2002257800

model-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32eca7bf18aeb0cdce8e89af67300d1263e03e0c63ddaa4f2bf5aa23b01b10e2
+size 2066906348

model-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7718a8437d71accf96d995ff245e2652d280bc7941e25003a2b402abb4e90053
+size 2000992320

model-00020.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:031fe379e46f4f5ecb0269040d53044dcaaf38096bf8590eabc680135776dfed
+size 2000992712

model-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47b29e197e5c51aa62227fe8f7a241531fd2494b19f57ef060f8fdf621b9a201
+size 2000992712

model-00022.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13cf2d9649deaf1dc9e5e3c8edbd35ce7bef19cfd0850952e970e4fb1d1d0cbd
+size 2001764976

model-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a1e765d2b6580584d20b894b9a216c2003668e40d94d14acb4318571edf9334
+size 2002444440

model-00024.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00397df054d39a3c46d5cb420e2b32aa4185af59cd446dff9b6b4b22c47b278c
+size 516386172

model-00025.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:175ed9f8672e181cf0a69756c2753dedb5ebdd2f8eeba7b8e3bb9904b4227aa1
+size 2030997248

model-00026.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7c9e0d5342e54f21e688d0d2501a3ab149ddac35ff4b4fb883e3cb246173163
+size 2000992712

model-00027.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72bdf841190912df3f015788d3d16e928e722f16739b9faa055fd149458205db
+size 2000992712

model-00028.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72b198d575ed8024f75726891d39e10f3eb854b685732aec2a9efd6ff27b6df7
+size 2001631792

model-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1921b006292a4e27366dd9878c260e146f735fa333ebcf6096c58a9fbc2073d2
+size 2002444432

model-00030.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac8af44ff55de695a8243165db4f373361a7190b6a4dc25675ed1e2f6cc8fb05
+size 2051858684

model-00031.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f5183292ce9806f11f5d7cbffe8308b97feb409583c55b38c2dcceba6e6cea4
+size 2000992584

model-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:051a8d3521cd759c3a5c37ad4288e28adc466c6e16861a06ba200fe40fbe9bf0
+size 2000992712

model-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27f35b92e3e34ff2feb36df813b111b6c3fea2763c8b803f5a81ebcf8281d849
+size 2001139168

model-00034.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d0bc7b8205504f286bdaea08bc9769837d3148f9fb7f38cd5c2b56b2c457eed
+size 2002444288

model-00035.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:434e08cc2fa08b6b742320a56e05f911912ae63e962efe96aa8113c465bf2bba
+size 2066573420

model-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d66ead03e2c3490a90defec16a78b6b726eee0b0d41502c397e0e453a8d8139
+size 2000992440

model-00037.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:643a31df0f5a9718389be9edd693dc9e402c76be9e504cab32e530aa2077a74f
+size 2000992712

model-00038.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0146a3f3756f35d0b8ac8aa152c7b5723dd02e44f3d6a04eb36269efe92aa307
+size 2000992712

model-00039.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4b6536ced36ca3320afb91a13c9ac71368f99e269fab8deb952ff9d58c22aae
+size 2002098000

model-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41c0ca23463509b0259da0ee8c7fc394db535b6daa10dca7a8ba956f284f3b96
+size 2002444440

model-00041.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcdcaf637963d1c289b8ccd0af1733e565c9c77a12c634bde1c8a3701dcbf254
+size 2051433276

model-00042.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fd6edfb263e7b62846206bf47e4039899f79fc9bf66cc9224cabf86ca80e6a0
+size 2000993152