Upload model files

README.md

@@ -0,0 +1,127 @@
+---
+license: mit
+library_name: transformers
+base_model:
+  - deepseek-ai/DeepSeek-V3.2-Exp-Base
+base_model_relation: finetune
+---
+# DeepSeek-V3.2: Efficient Reasoning & Agentic AI
+
+<!-- markdownlint-disable first-line-h1 -->
+<!-- markdownlint-disable html -->
+<!-- markdownlint-disable no-duplicate-header -->
+
+<div align="center">
+  <img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/logo.svg?raw=true" width="60%" alt="DeepSeek-V3" />
+</div>
+<hr>
+<div align="center" style="line-height: 1;">
+  <a href="https://www.deepseek.com/" target="_blank" style="margin: 2px;">
+    <img alt="Homepage" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/badge.svg?raw=true" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://chat.deepseek.com/" target="_blank" style="margin: 2px;">
+    <img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-DeepSeek%20V3-536af5?color=536af5&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://huggingface.co/deepseek-ai" target="_blank" style="margin: 2px;">
+    <img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-DeepSeek%20AI-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+</div>
+<div align="center" style="line-height: 1;">
+  <a href="https://discord.gg/Tc7c45Zzu5" target="_blank" style="margin: 2px;">
+    <img alt="Discord" src="https://img.shields.io/badge/Discord-DeepSeek%20AI-7289da?logo=discord&logoColor=white&color=7289da" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/qr.jpeg?raw=true" target="_blank" style="margin: 2px;">
+    <img alt="Wechat" src="https://img.shields.io/badge/WeChat-DeepSeek%20AI-brightgreen?logo=wechat&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://twitter.com/deepseek_ai" target="_blank" style="margin: 2px;">
+    <img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-deepseek_ai-white?logo=x&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+</div>
+<div align="center" style="line-height: 1;">
+  <a href="LICENSE" style="margin: 2px;">
+    <img alt="License" src="https://img.shields.io/badge/License-MIT-f5de53?&color=f5de53" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+</div>
+
+<p align="center">
+  <a href="https://huggingface.co/deepseek-ai/DeepSeek-V3.2/blob/main/assets/paper.pdf"><b>Technical Report</b>👁️</a>
+</p>
+
+## Introduction
+
+We introduce **DeepSeek-V3.2**, a model that harmonizes high computational efficiency with superior reasoning and agent performance. Our approach is built upon three key technical breakthroughs:
+
+1. **DeepSeek Sparse Attention (DSA):** We introduce DSA, an efficient attention mechanism that substantially reduces computational complexity while preserving model performance, specifically optimized for long-context scenarios.
+2. **Scalable Reinforcement Learning Framework:** By implementing a robust RL protocol and scaling post-training compute, *DeepSeek-V3.2* performs comparably to GPT-5. Notably, our high-compute variant, **DeepSeek-V3.2-Speciale**, **surpasses GPT-5** and exhibits reasoning proficiency on par with Gemini-3.0-Pro.
+    - *Achievement:* 🥇 **Gold-medal performance** in the 2025 International Mathematical Olympiad (IMO) and International Olympiad in Informatics (IOI).
+3. **Large-Scale Agentic Task Synthesis Pipeline:** To integrate **reasoning into tool-use** scenarios, we developed a novel synthesis pipeline that systematically generates training data at scale. This facilitates scalable agentic post-training, improving compliance and generalization in complex interactive environments.
+
+<div align="center">
+ <img src="assets/benchmark.png" >
+</div>
+
+We have also released the final submissions for IOI 2025, ICPC World Finals, IMO 2025 and CMO 2025, which were selected based on our designed pipeline. These materials are provided for the community to conduct secondary verification. The files can be accessed at `assets/olympiad_cases`.
+
+## Chat Template
+
+DeepSeek-V3.2 introduces significant updates to its chat template compared to prior versions. The primary changes involve a revised format for tool calling and the introduction of a "thinking with tools" capability.
+
+To assist the community in understanding and adapting to this new template, we have provided a dedicated `encoding` folder, which contains Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model and how to parse the model's text output.
+
+A brief example is illustrated below:
+
+```python
+import transformers
+# encoding/encoding_dsv32.py
+from encoding_dsv32 import encode_messages, parse_message_from_completion_text
+
+tokenizer = transformers.AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V3.2")
+
+messages = [
+    {"role": "user", "content": "hello"},
+    {"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."},
+    {"role": "user", "content": "1+1=?"}
+]
+encode_config = dict(thinking_mode="thinking", drop_thinking=True, add_default_bos_token=True)
+
+# messages -> string
+prompt = encode_messages(messages, **encode_config)
+# Output: "<｜begin▁of▁sentence｜><｜User｜>hello<｜Assistant｜></think>Hello! I am DeepSeek.<｜end▁of▁sentence｜><｜User｜>1+1=?<｜Assistant｜><think>"
+
+# string -> tokens
+tokens = tokenizer.encode(prompt)
+# Output: [0, 128803, 33310, 128804, 128799, 19923, 3, 342, 1030, 22651, 4374, 1465, 16, 1, 128803, 19, 13, 19, 127252, 128804, 128798]
+```
+
+Important Notes:
+
+1. This release does not include a Jinja-format chat template. Please refer to the Python code mentioned above.
+2. The output parsing function included in the code is designed to handle well-formatted strings only. It does not attempt to correct or recover from malformed output that the model might occasionally generate. It is not suitable for production use without robust error handling.
+3. A new role named `developer` has been introduced in the chat template. This role is dedicated exclusively to search agent scenarios and is designated for no other tasks. The official API does not accept messages assigned to `developer`.
+
+## How to Run Locally
+
+The model structure of DeepSeek-V3.2 and DeepSeek-V3.2-Speciale are the same as DeepSeek-V3.2-Exp. Please visit [DeepSeek-V3.2-Exp](https://github.com/deepseek-ai/DeepSeek-V3.2-Exp) repo for more information about running this model locally.
+
+Usage Recommendations:
+
+1. For local deployment, we recommend setting the sampling parameters to `temperature = 1.0, top_p = 0.95`.
+2. Please note that the DeepSeek-V3.2-Speciale variant is designed exclusively for deep reasoning tasks and does not support the tool-calling functionality.
+
+## License
+
+This repository and the model weights are licensed under the [MIT License](LICENSE).
+
+## Citation
+
+```
+@misc{deepseekai2025deepseekv32,
+      title={DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models}, 
+      author={DeepSeek-AI},
+      year={2025},
+}
+```
+
+## Contact
+
+If you have any questions, please raise an issue or contact us at [service@deepseek.com](service@deepseek.com).

added_tokens.json

@@ -0,0 +1,56 @@
+{
+  "</minimax:tool_call>": 200053,
+  "</think>": 200051,
+  "<add_file>": 200036,
+  "<code_context>": 200043,
+  "<code_interpreter>": 200023,
+  "<commit_after>": 200018,
+  "<commit_before>": 200016,
+  "<commit_message>": 200040,
+  "<commit_msg>": 200017,
+  "<delete_file>": 200037,
+  "<edit_file>": 200039,
+  "<empty_output>": 200015,
+  "<empty_source_file>": 200041,
+  "<file_content>": 200044,
+  "<file_sep>": 200049,
+  "<filename>": 200006,
+  "<filepath>": 200048,
+  "<fim_middle>": 200002,
+  "<fim_pad>": 200004,
+  "<fim_prefix>": 200001,
+  "<fim_suffix>": 200003,
+  "<function_call>": 200022,
+  "<gh_stars>": 200007,
+  "<issue_closed>": 200010,
+  "<issue_comment>": 200009,
+  "<issue_start>": 200008,
+  "<jupyter_code>": 200013,
+  "<jupyter_error>": 200035,
+  "<jupyter_output>": 200014,
+  "<jupyter_start>": 200011,
+  "<jupyter_text>": 200012,
+  "<minimax:tool_call>": 200052,
+  "<pr_start>": 200046,
+  "<rename_file>": 200038,
+  "<repo_struct>": 200042,
+  "<reponame>": 200005,
+  "<review_comment>": 200047,
+  "<source_files>": 200045,
+  "<think>": 200050,
+  "[e~[": 200020,
+  "]!d~[": 200021,
+  "]!p~[": 200000,
+  "]<]end of image[>[": 200030,
+  "]<]end of speech[>[": 200028,
+  "]<]end of video[>[": 200032,
+  "]<]image[>[": 200025,
+  "]<]speech[>[": 200024,
+  "]<]start of image[>[": 200029,
+  "]<]start of speech[>[": 200027,
+  "]<]start of video[>[": 200031,
+  "]<]video[>[": 200026,
+  "]<]vision pad[>[": 200033,
+  "]~!b[": 200034,
+  "]~b]": 200019
+}

chat_template.jinja

@@ -0,0 +1,159 @@
+{# ----------‑‑‑ special token variables ‑‑‑---------- #}
+{%- set toolcall_begin_token   = '<minimax:tool_call>'         -%}
+{%- set toolcall_end_token     = '</minimax:tool_call>'        -%}
+{#- Tool Rendering Functions ============================================== -#}
+{%- macro render_tool_namespace(namespace_name, tool_list) -%}
+{%- for tool in tool_list -%}
+<tool>{{ tool.function | tojson(ensure_ascii=False) }}</tool>
+{% endfor -%}
+{%- endmacro -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{ content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{#- System Message Construction ============================================ -#}
+{%- macro build_system_message(system_message) -%}
+    {%- if system_message and system_message.content -%}
+        {{- visible_text(system_message.content) }}
+    {%- else -%}
+        {%- if model_identity is not defined -%}
+            {%- set model_identity = "You are a helpful assistant." -%}
+        {%- endif -%}
+        {{- model_identity }}
+    {%- endif -%}
+    
+    {#- Handle current_date -#}
+    {%- if system_message and system_message.current_date -%}
+        {{- '\n' ~ 'Current date: ' + system_message.current_date }}
+    {%- endif -%}
+    {#- Handle current_location -#}
+    {%- if system_message and system_message.current_location -%}
+        {{- '\n' ~ 'Current location: ' + system_message.current_location }}
+    {%- endif -%}
+{%- endmacro -%}
+{#- Main Template Logic ================================================= -#}
+{#- Extract system message (only first message if it's system) -#}
+{%- set system_message = none -%}
+{%- set conversation_messages = messages -%}
+{%- if messages and messages[0].role == "system" -%}
+    {%- set system_message = messages[0] -%}
+    {%- set conversation_messages = messages[1:] -%}
+{%- endif -%}
+{#- Get the last user message turn, for interleved thinking -#}
+{%- set ns = namespace(last_user_index=-1) %}
+{% for m in conversation_messages %}
+    {%- if m.role == 'user' %}
+        {% set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{#- Render system message -#}
+{{- ']~!b[' ~ ']~b]system' ~ '\n' }}
+{{- build_system_message(system_message) }}
+{#- Render tools if available -#}
+{%- if tools -%}
+    {{- '\n\n' ~ '# Tools' ~ '\n' ~ 'You may call one or more tools to assist with the user query.\nHere are the tools available in JSONSchema format:' ~ '\n' }}
+    {{- '\n' ~ '<tools>' ~ '\n' }}
+    {{- render_tool_namespace("functions", tools) }}
+    {{- '</tools>' ~ '\n\n' }}
+{{- 'When making tool calls, use XML format to invoke tools and pass parameters:' ~ '\n' }}
+{{- '\n' ~ toolcall_begin_token }}
+<invoke name="tool-name-1">
+<parameter name="param-key-1">param-value-1</parameter>
+<parameter name="param-key-2">param-value-2</parameter>
+...
+</invoke>
+{{- '\n' ~ toolcall_end_token }}
+{%- endif -%}
+{{- '[e~[\n' }}
+
+{#- Render messages -#}
+{%- set last_tool_call = namespace(name=none) -%}
+{%- for message in conversation_messages -%}
+    {%- if message.role == 'assistant' -%}
+        {#- Only render reasoning_content if no user message follows -#}
+        {{- ']~b]ai' ~ '\n' }}
+
+        {%- set reasoning_content = '' %}
+        {%- set content = visible_text(message.content) %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].strip('\n').split('<think>')[-1].strip('\n') %}
+                {%- set content = content.split('</think>')[-1].strip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if reasoning_content and loop.index0 > ns.last_user_index -%}
+            {{- '<think>' ~ '\n' ~ reasoning_content ~ '\n' ~ '</think>' ~ '\n\n' }}
+        {%- endif -%}
+        {%- if content -%}
+            {{- content }}
+        {%- endif -%}
+        {%- if message.tool_calls -%}
+            {{- '\n' ~ toolcall_begin_token ~ '\n' }}
+
+            {%- for tool_call in message.tool_calls -%}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<invoke name="' + tool_call.name + '">' }}
+                {% set _args = tool_call.arguments %}
+                {%- for k, v in _args.items() %}
+                {{- '<parameter name="' + k + '">' }}
+                {{- v | tojson(ensure_ascii=False) if v is not string else v }}
+                {{- '</parameter>' }}
+                {% endfor %}
+                {{- '</invoke>' ~ '\n' }}
+            {%- endfor -%}
+            
+            {{- toolcall_end_token}}
+            {%- set last_tool_call.name = message.tool_calls[-1].name -%}
+        {%- else -%}
+            {%- set last_tool_call.name = none -%}
+        {%- endif -%}
+        {{- '[e~[' ~ '\n' }}
+        
+    {%- elif message.role == 'tool' -%}
+    {%- if last_tool_call.name is none -%}
+        {{- raise_exception("Message has tool role, but there was no previous assistant message with a tool call!") }}
+    {%- endif -%}
+    {%- if loop.first or (conversation_messages[loop.index0 - 1].role != 'tool') -%}
+        {{- ']~b]tool' }}
+    {%- endif -%}
+    {%- if message.content is string -%}
+        {{- '\n<response>' }}
+        {{- message.content }}
+        {{- '</response>' }}
+    {%- else -%}
+        {%- for tr in message.content -%}
+            {{- '\n<response>' }}
+            {{- tr.output if tr.output is defined else (tr.text if tr.type == 'text' and tr.text is defined else tr) }}
+            {{- '\n</response>' }}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if loop.last or (conversation_messages[loop.index0 + 1].role != 'tool') -%}
+        {{- '[e~[\n' -}}
+    {%- endif -%}
+        
+    {%- elif message.role == 'user' -%}
+        {{- ']~b]user' ~ '\n' }}
+        {{- visible_text(message.content) }}
+        {{- '[e~[' ~ '\n' }}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Generation prompt -#}
+{%- if add_generation_prompt -%}
+{{- ']~b]ai' ~ '\n' ~ '<think>' ~ '\n' }}
+{%- endif -%}

config.json

@@ -0,0 +1,614 @@
+{
+    "architectures": [
+        "MiniMaxM2ForCausalLM"
+    ],
+    "attention_dropout": 0.0,
+    "attn_type_list": [
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1,
+        1
+    ],
+    "bos_token_id": null,
+    "eos_token_id": null,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 3072,
+    "initializer_range": 0.02,
+    "intermediate_size": 1536,
+    "layernorm_full_attention_beta": 1.0,
+    "layernorm_linear_attention_beta": 1.0,
+    "layernorm_mlp_beta": 1.0,
+    "max_position_embeddings": 196608,
+    "mlp_intermediate_size": 8192,
+    "model_type": "minimax",
+    "mtp_transformer_layers": 1,
+    "num_attention_heads": 48,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 62,
+    "num_key_value_heads": 8,
+    "num_local_experts": 256,
+    "num_mtp_modules": 3,
+    "output_router_logits": false,
+    "qk_norm_type": "per_layer",
+    "quantization": {
+        "group_size": 64,
+        "bits": 5,
+        "mode": "affine",
+        "model.layers.0.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.1.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.2.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.3.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.4.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.5.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.6.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.7.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.8.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.9.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.10.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.11.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.12.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.13.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.14.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.15.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.16.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.17.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.18.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.19.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.20.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.21.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.22.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.23.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.24.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.25.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.26.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.27.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.28.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.29.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.30.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.31.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.32.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.33.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.34.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.35.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.36.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.37.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.38.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.39.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.40.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.41.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.42.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.43.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.44.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.45.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.46.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.47.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.48.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.49.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.50.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.51.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.52.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.53.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.54.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.55.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.56.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.57.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.58.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.59.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.60.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.61.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        }
+    },
+    "quantization_config": {
+        "group_size": 64,
+        "bits": 5,
+        "mode": "affine",
+        "model.layers.0.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.1.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.2.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.3.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.4.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.5.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.6.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.7.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.8.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.9.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.10.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.11.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.12.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.13.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.14.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.15.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.16.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.17.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.18.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.19.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.20.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.21.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.22.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.23.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.24.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.25.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.26.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.27.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.28.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.29.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.30.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.31.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.32.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.33.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.34.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.35.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.36.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.37.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.38.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.39.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.40.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.41.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.42.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.43.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.44.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.45.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.46.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.47.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.48.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.49.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.50.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.51.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.52.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.53.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.54.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.55.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.56.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.57.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.58.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.59.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.60.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        },
+        "model.layers.61.block_sparse_moe.gate": {
+            "group_size": 64,
+            "bits": 8
+        }
+    },
+    "rms_norm_eps": 1e-06,
+    "rope_theta": 5000000,
+    "rotary_dim": 64,
+    "router_aux_loss_coef": 0.001,
+    "router_jitter_noise": 0.0,
+    "scoring_func": "sigmoid",
+    "shared_intermediate_size": 0,
+    "shared_moe_mode": "sigmoid",
+    "sliding_window": null,
+    "tie_word_embeddings": false,
+    "transformers_version": "4.46.1",
+    "use_cache": true,
+    "use_mtp": true,
+    "use_qk_norm": true,
+    "use_routing_bias": true,
+    "vocab_size": 200064
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "do_sample": true,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "top_k": 40,
+  "transformers_version": "4.46.1"
+}