Commit ·
54e4e69
0
Parent(s):
Init MiniMaxAI/MiniMax-M3.1-Flash
Browse filesCo-authored-by: ryanlee-dev <ryanlee-dev@users.noreply.huggingface.co>
Co-authored-by: Xuebi <Xuebi@users.noreply.huggingface.co>
Co-authored-by: rogeryoungh <rogeryoungh@users.noreply.huggingface.co>
This view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +36 -0
- LICENSE +17 -0
- README.md +289 -0
- README_zh.md +287 -0
- added_tokens.json +63 -0
- chat_template.jinja +248 -0
- config.json +357 -0
- configuration_minimax_m3_vl.py +111 -0
- docs/FlagOS-Multi-Chip-Support-and-Downloads.md +105 -0
- dspark/config.json +297 -0
- dspark/model-dspark-00001.safetensors +3 -0
- dspark/model.safetensors.index.json +104 -0
- figures/logo.svg +16 -0
- generation_config.json +8 -0
- image_processor.py +223 -0
- merges.txt +0 -0
- model-00001-of-00048.safetensors +3 -0
- model-00002-of-00048.safetensors +3 -0
- model-00003-of-00048.safetensors +3 -0
- model-00004-of-00048.safetensors +3 -0
- model-00005-of-00048.safetensors +3 -0
- model-00006-of-00048.safetensors +3 -0
- model-00007-of-00048.safetensors +3 -0
- model-00008-of-00048.safetensors +3 -0
- model-00009-of-00048.safetensors +3 -0
- model-00010-of-00048.safetensors +3 -0
- model-00011-of-00048.safetensors +3 -0
- model-00012-of-00048.safetensors +3 -0
- model-00013-of-00048.safetensors +3 -0
- model-00014-of-00048.safetensors +3 -0
- model-00015-of-00048.safetensors +3 -0
- model-00016-of-00048.safetensors +3 -0
- model-00017-of-00048.safetensors +3 -0
- model-00018-of-00048.safetensors +3 -0
- model-00019-of-00048.safetensors +3 -0
- model-00020-of-00048.safetensors +3 -0
- model-00021-of-00048.safetensors +3 -0
- model-00022-of-00048.safetensors +3 -0
- model-00023-of-00048.safetensors +3 -0
- model-00024-of-00048.safetensors +3 -0
- model-00025-of-00048.safetensors +3 -0
- model-00026-of-00048.safetensors +3 -0
- model-00027-of-00048.safetensors +3 -0
- model-00028-of-00048.safetensors +3 -0
- model-00029-of-00048.safetensors +3 -0
- model-00030-of-00048.safetensors +3 -0
- model-00031-of-00048.safetensors +3 -0
- model-00032-of-00048.safetensors +3 -0
- model-00033-of-00048.safetensors +3 -0
- model-00034-of-00048.safetensors +3 -0
.gitattributes
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
/model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
|
LICENSE
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MINIMAX COMMUNITY LICENSE
|
| 2 |
+
Copyright (c) 2026 MiniMax
|
| 3 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software for non-commercial purposes, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or provide copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
| 4 |
+
1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
| 5 |
+
2. If the Software (or any derivative works thereof) is used for any Commercial Use for your products or services:
|
| 6 |
+
1. you shall prominently display “Built with MiniMax M3.1-Flash” on a related website, user interface, blogpost, about page or product documentation.
|
| 7 |
+
2. you shall obtain a separate, prior written authorization from MiniMax by contacting api@minimax.io with the subject line “M3.1-Flash licensing - authorization request”, if such products and services generate more than 20 million US dollars (or equivalent in other currencies) in yearly revenue; otherwise, you only need to send a one-time notice to api@minimax.io with the subject “M3.1-Flash licensing — notice”.
|
| 8 |
+
3. “Commercial Use” means any use of the Software or any derivative work thereof that is primarily intended for commercial advantage or monetary compensation, which includes, without limitation: (i) offering products or services to third parties for a fee, which utilize, incorporate, or rely on the Software or its derivatives, (ii) the commercial use of APIs provided by or for the Software or its derivatives, including to support or enable commercial products, services, or operations, whether in a cloud-based, hosted, or other similar environment, and (iii) the deployment or provision of the Software or its derivatives that have been subjected to post-training, fine-tuning, instruction-tuning, or any other form of modification, for any commercial purpose.
|
| 9 |
+
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
| 10 |
+
|
| 11 |
+
Appendix: Prohibited Uses
|
| 12 |
+
You agree you will not use, or allow others to use, the Software or any derivatives of the Software to:
|
| 13 |
+
1. Generate or disseminate content prohibited by applicable laws or regulations.
|
| 14 |
+
2. Assist with, engage in or otherwise support any military purpose.
|
| 15 |
+
3. Exploit, harm, or attempt to exploit or harm minors.
|
| 16 |
+
4. Generate or disseminate false or misleading information with the intent to cause harm.
|
| 17 |
+
5. Promote discrimination, hate speech, or harmful behavior against individuals or groups based on race or ethnic origin, religion, disability, age, nationality and national origin, veteran status, sexual orientation, gender or gender identity, caste, immigration status, or any other characteristic that is associated with systemic discrimination or marginalization.
|
README.md
ADDED
|
@@ -0,0 +1,289 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-text-to-text
|
| 3 |
+
license: other
|
| 4 |
+
license_name: minimax-community
|
| 5 |
+
license_link: LICENSE
|
| 6 |
+
library_name: transformers
|
| 7 |
+
tags:
|
| 8 |
+
- multimodal
|
| 9 |
+
- moe
|
| 10 |
+
- agent
|
| 11 |
+
- coding
|
| 12 |
+
- video
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
<div align="center">
|
| 16 |
+
<img width="60%" src="figures/logo.svg" alt="MiniMax">
|
| 17 |
+
</div>
|
| 18 |
+
<hr>
|
| 19 |
+
|
| 20 |
+
<p align="center">
|
| 21 |
+
<a href="https://agent.minimax.io/" target="_blank"><img src="https://img.shields.io/badge/MiniMax%20Agent-FF6C37?logo=minimax&logoColor=white" alt="MiniMax Agent"></a>
|
| 22 |
+
<a href="https://platform.minimax.io/docs/guides/text-generation" target="_blank"><img src="https://img.shields.io/badge/API-FF6C37?logo=minimax&logoColor=white" alt="API"></a>
|
| 23 |
+
<a href="https://www.minimax.io" target="_blank"><img src="https://img.shields.io/badge/MiniMax%20Website-FF6C37?logo=minimax&logoColor=white" alt="MiniMax Website"></a>
|
| 24 |
+
<br>
|
| 25 |
+
<a href="https://modelscope.cn/organization/minimax" target="_blank" rel="noopener noreferrer"><img alt="ModelScope MiniMax AI" src="https://img.shields.io/badge/ModelScope-MiniMax%20AI-white?labelColor=%23EF3D5D"></a>
|
| 26 |
+
<a href="https://platform.minimaxi.com/docs/faq/contact-us" target="_blank"><img src="https://img.shields.io/badge/WeChat-07C160?logo=wechat&logoColor=white" alt="WeChat"></a>
|
| 27 |
+
<a href="https://discord.com/invite/DPC4AHFCBw" target="_blank"><img src="https://img.shields.io/badge/Discord-5865F2?logo=discord&logoColor=white" alt="Discord"></a>
|
| 28 |
+
<a href="https://huggingface.co/MiniMaxAI" target="_blank"><img src="https://img.shields.io/badge/Hugging%20Face-FFD21E?logo=huggingface&logoColor=black" alt="Hugging Face"></a>
|
| 29 |
+
<a href="https://github.com/MiniMax-AI" target="_blank"><img src="https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white" alt="GitHub"></a>
|
| 30 |
+
<a href="https://arxiv.org/abs/2606.13392" target="_blank"><img src="https://img.shields.io/badge/arXiv-2606.13392-B31B1B?logo=arxiv&logoColor=white" alt="arXiv Paper"></a>
|
| 31 |
+
</p>
|
| 32 |
+
|
| 33 |
+
**MiniMax-M3.1-Flash** is a native multimodal mixture-of-experts model for coding, agentic workflows, computer use, and cybersecurity. It supports text, image, and video inputs with a context window of up to **1M tokens**.
|
| 34 |
+
|
| 35 |
+
Building on MiniMax-M3, M3.1-Flash improves task performance and inference efficiency with sparse attention throughout the model, quantization-aware training, and DSpark speculative decoding.
|
| 36 |
+
|
| 37 |
+
**Highlights:**
|
| 38 |
+
- **Agentic performance:** A generational leap in agentic tasks, coding, and cybersecurity — 76.2 on BankerToolBench, 73.7% on DeepSWE v1.1, 61.7 on OSWorld V2, 29.6 on Agents' Last Exam, and 82.2 on CyberGym.
|
| 39 |
+
- **Efficient by design:** Sparse attention throughout, quantization-aware training, and DSpark speculative decoding improve inference efficiency.
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
## Model at a Glance
|
| 43 |
+
|
| 44 |
+
- **Architecture:** Native multimodal mixture-of-experts (MoE)
|
| 45 |
+
- **Parameters:** Approximately 428B total; 23B activated per token
|
| 46 |
+
- **Inputs / output:** Text, image, and video inputs; text output
|
| 47 |
+
- **Context window:** Up to 1,000,000 tokens
|
| 48 |
+
- **Attention:** [MiniMax Sparse Attention](https://arxiv.org/abs/2606.13392) (MSA) throughout the model
|
| 49 |
+
- **Speculative decoding:** DSpark draft head included
|
| 50 |
+
|
| 51 |
+
## Benchmarks
|
| 52 |
+
|
| 53 |
+
<div style="overflow-x:auto;margin:16px 0">
|
| 54 |
+
<table style="width:100%;border-collapse:collapse;font-size:14px">
|
| 55 |
+
<thead>
|
| 56 |
+
<tr>
|
| 57 |
+
<th style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">Benchmark</th>
|
| 58 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:rgba(242,95,92,0.08);">MiniMax M3.1 Flash</th>
|
| 59 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">DeepSeek V4.1 Flash-1017</th>
|
| 60 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">DeepSeek V4.1 Flash</th>
|
| 61 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">Kimi K3</th>
|
| 62 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">GLM-5.3</th>
|
| 63 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">GPT-5.6 Sol</th>
|
| 64 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">Opus 5</th>
|
| 65 |
+
</tr>
|
| 66 |
+
</thead>
|
| 67 |
+
<tbody>
|
| 68 |
+
<tr>
|
| 69 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><div>Automation-Bench v1.0.6</div></td>
|
| 70 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);"><strong>62.8</strong></td>
|
| 71 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">60.5</td>
|
| 72 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">54.8</td>
|
| 73 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">46.7</td>
|
| 74 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">48.8</td>
|
| 75 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">45.8</td>
|
| 76 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">50.3</td>
|
| 77 |
+
</tr>
|
| 78 |
+
<tr>
|
| 79 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">BankerToolBench</td>
|
| 80 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);"><strong>76.2</strong></td>
|
| 81 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 82 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">73.7</td>
|
| 83 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">69.1</td>
|
| 84 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.5</td>
|
| 85 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.6</td>
|
| 86 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>76.2</strong></td>
|
| 87 |
+
</tr>
|
| 88 |
+
<tr>
|
| 89 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">GDP.pdf AA</td>
|
| 90 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">20.8</td>
|
| 91 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 92 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">12.8</td>
|
| 93 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">22.0</td>
|
| 94 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">11.2</td>
|
| 95 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>27.2</strong></td>
|
| 96 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">21.6</td>
|
| 97 |
+
</tr>
|
| 98 |
+
<tr>
|
| 99 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">Terminal Bench 4.0</td>
|
| 100 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">38.1</td>
|
| 101 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">42.3</td>
|
| 102 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">31.2</td>
|
| 103 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">12.6</td>
|
| 104 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">37.9</td>
|
| 105 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">39.9</td>
|
| 106 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>51.8</strong></td>
|
| 107 |
+
</tr>
|
| 108 |
+
<tr>
|
| 109 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">LOCA-Bench 256K</td>
|
| 110 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);"><strong>60.3</strong></td>
|
| 111 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 112 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 113 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">49.7</td>
|
| 114 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">50.3</td>
|
| 115 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">60.0</td>
|
| 116 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">53.7</td>
|
| 117 |
+
</tr>
|
| 118 |
+
<tr>
|
| 119 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>Agents’ Last Exam</div></td>
|
| 120 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">29.6</td>
|
| 121 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 122 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 123 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">28.3</td>
|
| 124 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 125 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">30.6</td>
|
| 126 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>32.2</strong></td>
|
| 127 |
+
</tr>
|
| 128 |
+
<tr>
|
| 129 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">OSWorld 2.0</td>
|
| 130 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">61.7</td>
|
| 131 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 132 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">54.3</td>
|
| 133 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">58.3</td>
|
| 134 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 135 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">65.7</td>
|
| 136 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>70.6</strong></td>
|
| 137 |
+
</tr>
|
| 138 |
+
<tr>
|
| 139 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>PostTrain Bench v1.1</div></td>
|
| 140 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">39.3</td>
|
| 141 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 142 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">30.5</td>
|
| 143 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">36.6</td>
|
| 144 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">37.5</td>
|
| 145 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">36.2</td>
|
| 146 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>40.4</strong></td>
|
| 147 |
+
</tr>
|
| 148 |
+
<tr>
|
| 149 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">DeepSWE v1.1</td>
|
| 150 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">73.7</td>
|
| 151 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>77.0</strong></td>
|
| 152 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.2</td>
|
| 153 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">67.5</td>
|
| 154 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">66.9</td>
|
| 155 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">73.0</td>
|
| 156 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.0</td>
|
| 157 |
+
</tr>
|
| 158 |
+
<tr>
|
| 159 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">NL2Repo-Bench</td>
|
| 160 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">61.2</td>
|
| 161 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">68.2</td>
|
| 162 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">65.4</td>
|
| 163 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">58.0</td>
|
| 164 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">58.0</td>
|
| 165 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">56.8</td>
|
| 166 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>75.3</strong></td>
|
| 167 |
+
</tr>
|
| 168 |
+
<tr>
|
| 169 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">SWE Refactor Bench</td>
|
| 170 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">23.0</td>
|
| 171 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 172 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">11.8</td>
|
| 173 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">19.5</td>
|
| 174 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">10.4</td>
|
| 175 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">28.5</td>
|
| 176 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>31.0</strong></td>
|
| 177 |
+
</tr>
|
| 178 |
+
<tr>
|
| 179 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>CyberGym</div></td>
|
| 180 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">82.2</td>
|
| 181 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 182 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 183 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">80.0</td>
|
| 184 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>84.5</strong></td>
|
| 185 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">83.6</td>
|
| 186 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 187 |
+
</tr>
|
| 188 |
+
<tr>
|
| 189 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>BabyVision (with tools)</div></td>
|
| 190 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">91.0</td>
|
| 191 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">92.5</td>
|
| 192 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">89.6</td>
|
| 193 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">85.7</td>
|
| 194 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 195 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">88.9</td>
|
| 196 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>94.1</strong></td>
|
| 197 |
+
</tr>
|
| 198 |
+
<tr>
|
| 199 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">HLE (without tools)</td>
|
| 200 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">43.4</td>
|
| 201 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">42.6</td>
|
| 202 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">39.2</td>
|
| 203 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">43.5</td>
|
| 204 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">42.3</td>
|
| 205 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">49.5</td>
|
| 206 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>55.6</strong></td>
|
| 207 |
+
</tr>
|
| 208 |
+
<tr>
|
| 209 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">CharXiv RQ</td>
|
| 210 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">82.5</td>
|
| 211 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 212 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">69.0</td>
|
| 213 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">84.8</td>
|
| 214 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 215 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>85.2</strong></td>
|
| 216 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">83.7</td>
|
| 217 |
+
</tr>
|
| 218 |
+
</tbody>
|
| 219 |
+
</table>
|
| 220 |
+
</div>
|
| 221 |
+
|
| 222 |
+
<details>
|
| 223 |
+
<summary><strong>Evaluation Notes</strong></summary>
|
| 224 |
+
|
| 225 |
+
- **Automation-Bench v1.0.6:** 600 publicly released tasks.
|
| 226 |
+
- **BankerToolBench:** OpenCode 1.17.18; MiniMax-M2.7 judge.
|
| 227 |
+
- **GDP.pdf AA:** Taken directly from Artificial Analysis.
|
| 228 |
+
- **Terminal Bench 4.0:** Codex 0.149.1.
|
| 229 |
+
- **LOCA-Bench 256K:** Official implementation.
|
| 230 |
+
- **Agents’ Last Exam:** Official Overall setting; full-set results from the official ALE leaderboard only. Blank = no official result (CLI-only excluded), not zero.
|
| 231 |
+
- **OSWorld 2.0:** 0808 Full release; highest reported per model at release, except DeepSeek V4.1 (in-house reproduction).
|
| 232 |
+
- **PostTrain Bench v1.1:** Claude Code 2.1.251, 512K context; otherwise official setup.
|
| 233 |
+
- **DeepSWE v1.1:** mini-SWE-agent 2.4.5, 512K context.
|
| 234 |
+
- **NL2Repo-Bench:** Claude Code 2.1.233, 512K context; reward hacking = 0; two inconsistent tasks corrected (scores use the corrected set).
|
| 235 |
+
- **SWE Refactor Bench:** Claude Code 2.1.220, 200K context; otherwise official setup.
|
| 236 |
+
- **CyberGym:** Claude Code 2.1.263, 1M context.
|
| 237 |
+
- **BabyVision (with tools):** Claude Code 2.1.285; min image side 1,500 px.
|
| 238 |
+
- **HLE (without tools):** Text-only subset (2,158 questions).
|
| 239 |
+
- **CharXiv RQ:** GPT-5.5 judge.
|
| 240 |
+
|
| 241 |
+
</details>
|
| 242 |
+
|
| 243 |
+
Results are for the official weights; third-party quantizations may differ.
|
| 244 |
+
|
| 245 |
+
## What’s New in M3.1-Flash
|
| 246 |
+
|
| 247 |
+
M3.1-Flash retains M3’s core mixture-of-experts architecture, with three main changes:
|
| 248 |
+
|
| 249 |
+
- **Sparse attention throughout:** the first three full-attention layers are replaced with sparse attention.
|
| 250 |
+
- **Quantization-aware training:** the routed MoE experts and the attention paths are quantized during training, reducing memory footprint and improving serving efficiency.
|
| 251 |
+
- **DSpark speculative decoding:** DSpark replaces the earlier EAGLE-like MTP approach. The release includes the draft head for use with compatible serving implementations.
|
| 252 |
+
|
| 253 |
+
|
| 254 |
+
## How to Use
|
| 255 |
+
|
| 256 |
+
- [MiniMax Agent](https://agent.minimax.io/)
|
| 257 |
+
- [MiniMax API](https://platform.minimax.io/)
|
| 258 |
+
|
| 259 |
+
### Local Deployment
|
| 260 |
+
|
| 261 |
+
Download the model:
|
| 262 |
+
|
| 263 |
+
```bash
|
| 264 |
+
hf download MiniMaxAI/MiniMax-M3.1-Flash --local-dir MiniMax-M3.1-Flash
|
| 265 |
+
```
|
| 266 |
+
|
| 267 |
+
We recommend the following inference frameworks to serve the model:
|
| 268 |
+
|
| 269 |
+
- [SGLang](https://docs.sglang.io/) — see [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1).
|
| 270 |
+
- [vLLM](https://github.com/vllm-project/vllm) — see [vLLM recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash).
|
| 271 |
+
- [TokenSpeed](https://github.com/lightseekorg/tokenspeed) — see [TokenSpeed recipe](https://lightseek.org/tokenspeed/recipes/models#minimax-m3-1).
|
| 272 |
+
|
| 273 |
+
Additional integrations in progress: Transformers, KTransformers, unsloth, and ATOM.
|
| 274 |
+
|
| 275 |
+
### Multi-Chip Deployment
|
| 276 |
+
|
| 277 |
+
[FlagOS](https://github.com/flagos-ai) — see [Multi-Chip Support and Downloads](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash/blob/main/docs/FlagOS-Multi-Chip-Support-and-Downloads.md).
|
| 278 |
+
|
| 279 |
+
## Inference Settings
|
| 280 |
+
|
| 281 |
+
Recommended sampling settings: `temperature=1.0` and `top_p=0.95`.
|
| 282 |
+
|
| 283 |
+
For reasoning, set `reasoning_effort` in the chat template: `low`, `medium`, `high`, `xhigh`, or `max`.
|
| 284 |
+
|
| 285 |
+
The model supports a context window of up to 1M tokens. Usable context length and throughput depend on GPU memory and the serving configuration.
|
| 286 |
+
|
| 287 |
+
## Contact Us
|
| 288 |
+
|
| 289 |
+
Contact us at [model@minimax.io](mailto:model@minimax.io).
|
README_zh.md
ADDED
|
@@ -0,0 +1,287 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-text-to-text
|
| 3 |
+
license: other
|
| 4 |
+
license_name: minimax-community
|
| 5 |
+
license_link: LICENSE
|
| 6 |
+
library_name: transformers
|
| 7 |
+
tags:
|
| 8 |
+
- multimodal
|
| 9 |
+
- moe
|
| 10 |
+
- agent
|
| 11 |
+
- coding
|
| 12 |
+
- video
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
<p align="right"><a href="README.md">English</a></p>
|
| 17 |
+
|
| 18 |
+
# MiniMax-M3.1-Flash
|
| 19 |
+
|
| 20 |
+
<div align="center">
|
| 21 |
+
<img width="60%" src="figures/logo.svg" alt="MiniMax">
|
| 22 |
+
</div>
|
| 23 |
+
<hr>
|
| 24 |
+
|
| 25 |
+
<p align="center">
|
| 26 |
+
<a href="https://agent.minimax.io/" target="_blank"><img src="https://img.shields.io/badge/MiniMax%20Agent-FF6C37?logo=minimax&logoColor=white" alt="MiniMax Agent"></a>
|
| 27 |
+
<a href="https://platform.minimax.io/docs/guides/text-generation" target="_blank"><img src="https://img.shields.io/badge/API-FF6C37?logo=minimax&logoColor=white" alt="API"></a>
|
| 28 |
+
<a href="https://www.minimax.io" target="_blank"><img src="https://img.shields.io/badge/MiniMax%20Website-FF6C37?logo=minimax&logoColor=white" alt="MiniMax Website"></a>
|
| 29 |
+
<br>
|
| 30 |
+
<a href="https://modelscope.cn/organization/minimax" target="_blank" rel="noopener noreferrer"><img alt="ModelScope MiniMax AI" src="https://img.shields.io/badge/ModelScope-MiniMax%20AI-white?labelColor=%23EF3D5D"></a>
|
| 31 |
+
<a href="https://platform.minimaxi.com/docs/faq/contact-us" target="_blank"><img src="https://img.shields.io/badge/WeChat-07C160?logo=wechat&logoColor=white" alt="WeChat"></a>
|
| 32 |
+
<a href="https://discord.com/invite/DPC4AHFCBw" target="_blank"><img src="https://img.shields.io/badge/Discord-5865F2?logo=discord&logoColor=white" alt="Discord"></a>
|
| 33 |
+
<a href="https://huggingface.co/MiniMaxAI" target="_blank"><img src="https://img.shields.io/badge/Hugging%20Face-FFD21E?logo=huggingface&logoColor=black" alt="Hugging Face"></a>
|
| 34 |
+
<a href="https://github.com/MiniMax-AI" target="_blank"><img src="https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white" alt="GitHub"></a>
|
| 35 |
+
<a href="https://arxiv.org/abs/2606.13392" target="_blank"><img src="https://img.shields.io/badge/arXiv-2606.13392-B31B1B?logo=arxiv&logoColor=white" alt="arXiv Paper"></a>
|
| 36 |
+
</p>
|
| 37 |
+
|
| 38 |
+
**MiniMax-M3.1-Flash** 是一款原生多模态混合专家（MoE）模型，面向编程、智能体工作流、计算机使用和网络安全等场景。它支持文本、图像和视频输入，上下文窗口最高可达 **100 万 tokens**。
|
| 39 |
+
|
| 40 |
+
在 MiniMax-M3 的基础上，M3.1-Flash 通过全模型稀疏注意力、量化感知训练和 DSpark 投机解码，进一步提升了任务表现与推理效率。
|
| 41 |
+
|
| 42 |
+
**亮点：**
|
| 43 |
+
- **智能体能力：** 在智能体任务、编程和网络安全方面实现代际跃升——BankerToolBench 76.2、DeepSWE v1.1 73.7%、OSWorld V2 61.7、Agents' Last Exam 29.6、CyberGym 82.2。
|
| 44 |
+
- **高效设计：** 全模型稀疏注意力、量化感知训练和 DSpark 投机解码共同提升推理效率。
|
| 45 |
+
|
| 46 |
+
## 模型概览
|
| 47 |
+
|
| 48 |
+
- **架构：** 原生多模态混合专家（MoE）
|
| 49 |
+
- **参数：** 总参数约 428B；每 token 激活约 23B
|
| 50 |
+
- **输入 / 输出：** 文本、图像和视频输入；文本输出
|
| 51 |
+
- **上下文窗口：** 最高 1,000,000 tokens
|
| 52 |
+
- **注意力：** 全模型采用 [MiniMax Sparse Attention](https://arxiv.org/abs/2606.13392)（MSA）
|
| 53 |
+
- **投机解码：** 包含 DSpark draft head
|
| 54 |
+
|
| 55 |
+
## 基准测试
|
| 56 |
+
|
| 57 |
+
<div style="overflow-x:auto;margin:16px 0">
|
| 58 |
+
<table style="width:100%;border-collapse:collapse;font-size:14px">
|
| 59 |
+
<thead>
|
| 60 |
+
<tr>
|
| 61 |
+
<th style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">基准测试</th>
|
| 62 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:rgba(242,95,92,0.08);">MiniMax M3.1 Flash</th>
|
| 63 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">DeepSeek V4.1 Flash-1017</th>
|
| 64 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">DeepSeek V4.1 Flash</th>
|
| 65 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">Kimi K3</th>
|
| 66 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">GLM-5.3</th>
|
| 67 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">GPT-5.6 Sol</th>
|
| 68 |
+
<th style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;font-weight:600;border-bottom:2px solid #f25f5c;background:#f7f7f8;">Opus 5</th>
|
| 69 |
+
</tr>
|
| 70 |
+
</thead>
|
| 71 |
+
<tbody>
|
| 72 |
+
<tr>
|
| 73 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><div>Automation-Bench v1.0.6</div></td>
|
| 74 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);"><strong>62.8</strong></td>
|
| 75 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">60.5</td>
|
| 76 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">54.8</td>
|
| 77 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">46.7</td>
|
| 78 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">48.8</td>
|
| 79 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">45.8</td>
|
| 80 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">50.3</td>
|
| 81 |
+
</tr>
|
| 82 |
+
<tr>
|
| 83 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">BankerToolBench</td>
|
| 84 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);"><strong>76.2</strong></td>
|
| 85 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 86 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">73.7</td>
|
| 87 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">69.1</td>
|
| 88 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.5</td>
|
| 89 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.6</td>
|
| 90 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>76.2</strong></td>
|
| 91 |
+
</tr>
|
| 92 |
+
<tr>
|
| 93 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">GDP.pdf AA</td>
|
| 94 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">20.8</td>
|
| 95 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 96 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">12.8</td>
|
| 97 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">22.0</td>
|
| 98 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">11.2</td>
|
| 99 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>27.2</strong></td>
|
| 100 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">21.6</td>
|
| 101 |
+
</tr>
|
| 102 |
+
<tr>
|
| 103 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">Terminal Bench 4.0</td>
|
| 104 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">38.1</td>
|
| 105 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">42.3</td>
|
| 106 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">31.2</td>
|
| 107 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">12.6</td>
|
| 108 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">37.9</td>
|
| 109 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">39.9</td>
|
| 110 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>51.8</strong></td>
|
| 111 |
+
</tr>
|
| 112 |
+
<tr>
|
| 113 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">LOCA-Bench 256K</td>
|
| 114 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);"><strong>60.3</strong></td>
|
| 115 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 116 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 117 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">49.7</td>
|
| 118 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">50.3</td>
|
| 119 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">60.0</td>
|
| 120 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">53.7</td>
|
| 121 |
+
</tr>
|
| 122 |
+
<tr>
|
| 123 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>Agents’ Last Exam</div></td>
|
| 124 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">29.6</td>
|
| 125 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 126 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 127 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">28.3</td>
|
| 128 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 129 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">30.6</td>
|
| 130 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>32.2</strong></td>
|
| 131 |
+
</tr>
|
| 132 |
+
<tr>
|
| 133 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">OSWorld 2.0</td>
|
| 134 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">61.7</td>
|
| 135 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 136 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">54.3</td>
|
| 137 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">58.3</td>
|
| 138 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 139 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">65.7</td>
|
| 140 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>70.6</strong></td>
|
| 141 |
+
</tr>
|
| 142 |
+
<tr>
|
| 143 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>PostTrain Bench v1.1</div></td>
|
| 144 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">39.3</td>
|
| 145 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 146 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">30.5</td>
|
| 147 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">36.6</td>
|
| 148 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">37.5</td>
|
| 149 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">36.2</td>
|
| 150 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>40.4</strong></td>
|
| 151 |
+
</tr>
|
| 152 |
+
<tr>
|
| 153 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">DeepSWE v1.1</td>
|
| 154 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">73.7</td>
|
| 155 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>77.0</strong></td>
|
| 156 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.2</td>
|
| 157 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">67.5</td>
|
| 158 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">66.9</td>
|
| 159 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">73.0</td>
|
| 160 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">74.0</td>
|
| 161 |
+
</tr>
|
| 162 |
+
<tr>
|
| 163 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">NL2Repo-Bench</td>
|
| 164 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">61.2</td>
|
| 165 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">68.2</td>
|
| 166 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">65.4</td>
|
| 167 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">58.0</td>
|
| 168 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">58.0</td>
|
| 169 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">56.8</td>
|
| 170 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>75.3</strong></td>
|
| 171 |
+
</tr>
|
| 172 |
+
<tr>
|
| 173 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">SWE Refactor Bench</td>
|
| 174 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">23.0</td>
|
| 175 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 176 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">11.8</td>
|
| 177 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">19.5</td>
|
| 178 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">10.4</td>
|
| 179 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">28.5</td>
|
| 180 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>31.0</strong></td>
|
| 181 |
+
</tr>
|
| 182 |
+
<tr>
|
| 183 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>CyberGym</div></td>
|
| 184 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">82.2</td>
|
| 185 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 186 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 187 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">80.0</td>
|
| 188 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>84.5</strong></td>
|
| 189 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">83.6</td>
|
| 190 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 191 |
+
</tr>
|
| 192 |
+
<tr>
|
| 193 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><div>BabyVision (with tools)</div></td>
|
| 194 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;background:rgba(242,95,92,0.08);">91.0</td>
|
| 195 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">92.5</td>
|
| 196 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">89.6</td>
|
| 197 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">85.7</td>
|
| 198 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">—</td>
|
| 199 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;">88.9</td>
|
| 200 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;border-top:2px solid #f1d6d3;"><strong>94.1</strong></td>
|
| 201 |
+
</tr>
|
| 202 |
+
<tr>
|
| 203 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">HLE (without tools)</td>
|
| 204 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">43.4</td>
|
| 205 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">42.6</td>
|
| 206 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">39.2</td>
|
| 207 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">43.5</td>
|
| 208 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">42.3</td>
|
| 209 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">49.5</td>
|
| 210 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>55.6</strong></td>
|
| 211 |
+
</tr>
|
| 212 |
+
<tr>
|
| 213 |
+
<td style="padding:10px 12px;text-align:left;vertical-align:middle;border-bottom:1px solid #e5e7eb;">CharXiv RQ</td>
|
| 214 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;background:rgba(242,95,92,0.08);">82.5</td>
|
| 215 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 216 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">69.0</td>
|
| 217 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">84.8</td>
|
| 218 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">—</td>
|
| 219 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;"><strong>85.2</strong></td>
|
| 220 |
+
<td style="padding:10px 12px;text-align:center;vertical-align:middle;border-bottom:1px solid #e5e7eb;">83.7</td>
|
| 221 |
+
</tr>
|
| 222 |
+
</tbody>
|
| 223 |
+
</table>
|
| 224 |
+
</div>
|
| 225 |
+
|
| 226 |
+
<details>
|
| 227 |
+
<summary><strong>评测说明</strong></summary>
|
| 228 |
+
|
| 229 |
+
- **Automation-Bench v1.0.6：** 600 个公开任务。
|
| 230 |
+
- **BankerToolBench：** OpenCode 1.17.18；MiniMax-M2.7 作为 judge。
|
| 231 |
+
- **GDP.pdf AA：** 直接取自 Artificial Analysis。
|
| 232 |
+
- **Terminal Bench 4.0：** Codex 0.149.1。
|
| 233 |
+
- **LOCA-Bench 256K：** 官方实现。
|
| 234 |
+
- **Agents’ Last Exam：** 采用官方 Overall 设置；仅收录官方 ALE 榜单的完整测试集结果。空白表示没有官方结果（仅 CLI 结果不计入），并非零分。
|
| 235 |
+
- **OSWorld 2.0：** 采用 0808 Full 版本；各模型取其发布时报告的最高分，DeepSeek V4.1 除外（内部复现）。
|
| 236 |
+
- **PostTrain Bench v1.1：** Claude Code 2.1.251，512K 上下文；其余设置遵循官方评测。
|
| 237 |
+
- **DeepSWE v1.1：** mini-SWE-agent 2.4.5，512K 上下文。
|
| 238 |
+
- **NL2Repo-Bench：** Claude Code 2.1.233，512K 上下文；reward hacking 计 0 分；修正了两个规格与测试不一致的任务（分数基于修正后的测试集）。
|
| 239 |
+
- **SWE Refactor Bench：** Claude Code 2.1.220，200K 上下文；其余设置遵循官方评测。
|
| 240 |
+
- **CyberGym：** Claude Code 2.1.263，1M 上下文。
|
| 241 |
+
- **BabyVision（带工具）：** Claude Code 2.1.285；图像最短边 1,500 px。
|
| 242 |
+
- **HLE（不带工具）：** 纯文本子集（2,158 题）。
|
| 243 |
+
- **CharXiv RQ：** GPT-5.5 作为 judge。
|
| 244 |
+
|
| 245 |
+
</details>
|
| 246 |
+
|
| 247 |
+
以上结果基于官方权重；第三方量化版本可能有所不同。
|
| 248 |
+
|
| 249 |
+
## M3.1-Flash 的新特性
|
| 250 |
+
|
| 251 |
+
M3.1-Flash 延续 M3 的核心混合专家架构，主要有三项变化：
|
| 252 |
+
|
| 253 |
+
- **全模型稀疏注意力：** 前三个全注意力层替换为稀疏注意力。
|
| 254 |
+
- **量化感知训练：** 路由 MoE 专家与注意力路径在训练阶段完成量化，降低显存占用并提升推理效率。
|
| 255 |
+
- **DSpark 投机解码：** DSpark 取代此前的类 EAGLE MTP 方案。本次发布包含 draft head，可与兼容的 serving 实现配合使用。
|
| 256 |
+
|
| 257 |
+
## 如何使用
|
| 258 |
+
|
| 259 |
+
- [MiniMax Agent](https://agent.minimax.io/)
|
| 260 |
+
- [MiniMax API](https://platform.minimax.io/)
|
| 261 |
+
|
| 262 |
+
### 本地部署
|
| 263 |
+
|
| 264 |
+
下载模型：
|
| 265 |
+
|
| 266 |
+
```bash
|
| 267 |
+
hf download MiniMaxAI/MiniMax-M3.1-Flash --local-dir MiniMax-M3.1-Flash
|
| 268 |
+
```
|
| 269 |
+
|
| 270 |
+
推荐使用以下推理框架部署模型：
|
| 271 |
+
|
| 272 |
+
- [SGLang](https://docs.sglang.io/) — 参见 [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1)。
|
| 273 |
+
- [vLLM](https://github.com/vllm-project/vllm) — 参见 [vLLM recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash)。
|
| 274 |
+
|
| 275 |
+
更多集成正在推进中：Transformers、TokenSpeed、KTransformers、unsloth 和 ATOM。
|
| 276 |
+
|
| 277 |
+
## 推理设置
|
| 278 |
+
|
| 279 |
+
推荐采样设置：`temperature=1.0` 和 `top_p=0.95`。
|
| 280 |
+
|
| 281 |
+
推理时，在 chat template 中设置 `reasoning_effort`：`low`、`medium`、`high`、`xhigh` 或 `max`。
|
| 282 |
+
|
| 283 |
+
模型支持最高 100 万 tokens 的上下文窗口。可用上下文长度和吞吐量取决于 GPU 显存和 serving 配置。
|
| 284 |
+
|
| 285 |
+
## 联系我们
|
| 286 |
+
|
| 287 |
+
请联系 [model@minimax.io](mailto:model@minimax.io)。
|
added_tokens.json
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"]!p~[": 200000,
|
| 3 |
+
"<fim_prefix>": 200001,
|
| 4 |
+
"<fim_middle>": 200002,
|
| 5 |
+
"<fim_suffix>": 200003,
|
| 6 |
+
"<fim_pad>": 200004,
|
| 7 |
+
"<reponame>": 200005,
|
| 8 |
+
"<filename>": 200006,
|
| 9 |
+
"<gh_stars>": 200007,
|
| 10 |
+
"<issue_start>": 200008,
|
| 11 |
+
"<issue_comment>": 200009,
|
| 12 |
+
"<issue_closed>": 200010,
|
| 13 |
+
"<jupyter_start>": 200011,
|
| 14 |
+
"<jupyter_text>": 200012,
|
| 15 |
+
"<jupyter_code>": 200013,
|
| 16 |
+
"<jupyter_output>": 200014,
|
| 17 |
+
"<empty_output>": 200015,
|
| 18 |
+
"<commit_before>": 200016,
|
| 19 |
+
"<commit_msg>": 200017,
|
| 20 |
+
"<commit_after>": 200018,
|
| 21 |
+
"]~b]": 200019,
|
| 22 |
+
"[e~[": 200020,
|
| 23 |
+
"]!d~[": 200021,
|
| 24 |
+
"<function_call>": 200022,
|
| 25 |
+
"<code_interpreter>": 200023,
|
| 26 |
+
"]<]speech[>[": 200024,
|
| 27 |
+
"]<]image[>[": 200025,
|
| 28 |
+
"]<]video[>[": 200026,
|
| 29 |
+
"]<]start of speech[>[": 200027,
|
| 30 |
+
"]<]end of speech[>[": 200028,
|
| 31 |
+
"]<]start of image[>[": 200029,
|
| 32 |
+
"]<]end of image[>[": 200030,
|
| 33 |
+
"]<]start of video[>[": 200031,
|
| 34 |
+
"]<]end of video[>[": 200032,
|
| 35 |
+
"]<]vision pad[>[": 200033,
|
| 36 |
+
"]~!b[": 200034,
|
| 37 |
+
"<jupyter_error>": 200035,
|
| 38 |
+
"<add_file>": 200036,
|
| 39 |
+
"<delete_file>": 200037,
|
| 40 |
+
"<rename_file>": 200038,
|
| 41 |
+
"<edit_file>": 200039,
|
| 42 |
+
"<commit_message>": 200040,
|
| 43 |
+
"<empty_source_file>": 200041,
|
| 44 |
+
"<repo_struct>": 200042,
|
| 45 |
+
"<code_context>": 200043,
|
| 46 |
+
"<file_content>": 200044,
|
| 47 |
+
"<source_files>": 200045,
|
| 48 |
+
"<pr_start>": 200046,
|
| 49 |
+
"<review_comment>": 200047,
|
| 50 |
+
"<filepath>": 200048,
|
| 51 |
+
"<file_sep>": 200049,
|
| 52 |
+
"<think>": 200050,
|
| 53 |
+
"</think>": 200051,
|
| 54 |
+
"<tool_call>": 200052,
|
| 55 |
+
"</tool_call>": 200053,
|
| 56 |
+
"]<]frame[>[": 200054,
|
| 57 |
+
"]<]start of frame[>[": 200055,
|
| 58 |
+
"]<]end of frame[>[": 200056,
|
| 59 |
+
"<|content_altered_placeholder|>": 200057,
|
| 60 |
+
"]<]minimax[>[": 200058,
|
| 61 |
+
"<mm:think>": 200059,
|
| 62 |
+
"</mm:think>": 200060
|
| 63 |
+
}
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,248 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{# ---------- special token variables ---------- #}
|
| 2 |
+
{%- set ns_token = ']<]minimax[>[' -%}
|
| 3 |
+
{%- set bod_token = ']~!b[' -%}
|
| 4 |
+
{%- set bos_token = ']~b]' -%}
|
| 5 |
+
{%- set eos_token = '[e~[' -%}
|
| 6 |
+
{%- set toolcall_begin_token = ns_token ~ '<tool_call>' -%}
|
| 7 |
+
{%- set toolcall_end_token = ns_token ~ '</tool_call>' -%}
|
| 8 |
+
{%- set think_begin_token = '<mm:think>' -%}
|
| 9 |
+
{%- set think_end_token = '</mm:think>' -%}
|
| 10 |
+
{%- set image_token = ']<]image[>[' -%}
|
| 11 |
+
{%- set video_token = ']<]video[>[' -%}
|
| 12 |
+
{#- Recursive XML renderer for tool_call arguments ======================== -#}
|
| 13 |
+
{#- None values are intentionally skipped in mapping iteration so that
|
| 14 |
+
`<key>null</key>` (which would round-trip to the literal string "null")
|
| 15 |
+
never appears in the rendered tool_call. The convention is: omit the
|
| 16 |
+
field entirely. The top-level `_args` loop applies the same rule.
|
| 17 |
+
The `val is none` branch below is a safety net only — upstream cleaning
|
| 18 |
+
(drop_none_in_tool_arguments) should ensure no None ever reaches here. -#}
|
| 19 |
+
{%- macro to_xml(val, ns) -%}
|
| 20 |
+
{%- if val is mapping -%}
|
| 21 |
+
{%- for k, v in val.items() if v is not none -%}
|
| 22 |
+
{{ ns }}<{{ k }}>{{ to_xml(v, ns) }}{{ ns }}</{{ k }}>
|
| 23 |
+
{%- endfor -%}
|
| 24 |
+
{%- elif val is iterable and val is not string -%}
|
| 25 |
+
{%- for item in val -%}
|
| 26 |
+
{{ ns }}<item>{{ to_xml(item, ns) }}{{ ns }}</item>
|
| 27 |
+
{%- endfor -%}
|
| 28 |
+
{%- elif val is none -%}
|
| 29 |
+
{#- Should be unreachable when upstream cleaning is applied. -#}
|
| 30 |
+
{%- elif val is boolean -%}
|
| 31 |
+
{{ val | tojson }}
|
| 32 |
+
{%- else -%}
|
| 33 |
+
{{ val }}
|
| 34 |
+
{%- endif -%}
|
| 35 |
+
{%- endmacro -%}
|
| 36 |
+
{#- Tool Rendering Functions ============================================== -#}
|
| 37 |
+
{%- macro render_tool_namespace(namespace_name, tool_list) -%}
|
| 38 |
+
{%- for tool in tool_list -%}
|
| 39 |
+
<tool>{{ tool.function | tojson(ensure_ascii=False) }}</tool>
|
| 40 |
+
{% endfor -%}
|
| 41 |
+
{%- endmacro -%}
|
| 42 |
+
{%- macro visible_text(content) -%}
|
| 43 |
+
{%- if content is string -%}
|
| 44 |
+
{{ content }}
|
| 45 |
+
{%- elif content is iterable and content is not mapping -%}
|
| 46 |
+
{%- for item in content -%}
|
| 47 |
+
{%- if item is mapping and item.type == 'text' -%}
|
| 48 |
+
{{- item.text }}
|
| 49 |
+
{%- elif item is mapping and item.type == 'image' -%}
|
| 50 |
+
{{- image_token }}
|
| 51 |
+
{%- elif item is mapping and item.type == 'video' -%}
|
| 52 |
+
{{- video_token }}
|
| 53 |
+
{%- elif item is string -%}
|
| 54 |
+
{{- item }}
|
| 55 |
+
{%- endif -%}
|
| 56 |
+
{%- endfor -%}
|
| 57 |
+
{%- elif content is none -%}
|
| 58 |
+
{{- '' }}
|
| 59 |
+
{%- else -%}
|
| 60 |
+
{{- content }}
|
| 61 |
+
{%- endif -%}
|
| 62 |
+
{%- endmacro -%}
|
| 63 |
+
{#- System Message Construction ============================================ -#}
|
| 64 |
+
{%- macro build_system_message(system_message) -%}
|
| 65 |
+
{%- if system_message and system_message.content -%}
|
| 66 |
+
{{- visible_text(system_message.content) }}
|
| 67 |
+
{#- Handle current_date -#}
|
| 68 |
+
{%- if system_message.current_date -%}
|
| 69 |
+
{{- '\n' ~ 'Current date: ' + system_message.current_date }}
|
| 70 |
+
{%- endif -%}
|
| 71 |
+
{#- Handle current_location -#}
|
| 72 |
+
{%- if system_message.current_location -%}
|
| 73 |
+
{{- '\n' ~ 'Current location: ' + system_message.current_location }}
|
| 74 |
+
{%- endif -%}
|
| 75 |
+
{%- else -%}
|
| 76 |
+
{{- 'Your model version is MiniMax-M3.1, developed by MiniMax. Knowledge cutoff: January 2026. Founded in early 2022, MiniMax is a global AI foundation model company committed to advancing the frontiers of AI towards AGI.' }}
|
| 77 |
+
{%- endif -%}
|
| 78 |
+
|
| 79 |
+
{#- Thinking mode instructions -#}
|
| 80 |
+
{{- '\n\n<thinking_instructions>\n' }}
|
| 81 |
+
{{- 'You have a thinking capability that allows you to reason step by step before responding. When thinking is enabled, wrap your reasoning in ' ~ think_begin_token ~ think_end_token ~ ' tags before your response. When thinking is disabled, begin your response directly after the ' ~ think_end_token ~ ' prefix. When thinking is adaptive, decide on your own whether to think for the current turn.\n' }}
|
| 82 |
+
{{- 'Current thinking mode: adaptive. You are encouraged to think for complex decision-making, multi-step reasoning, or when analyzing function/tool results.\n' }}
|
| 83 |
+
{{- '</thinking_instructions>' }}
|
| 84 |
+
{{- '\n\n<model_id>8bb8149fe</model_id>' }}
|
| 85 |
+
{%- if reasoning_effort is defined and reasoning_effort is not none -%}
|
| 86 |
+
{{- '\n\n<effort>' ~ reasoning_effort ~ '</effort>' }}
|
| 87 |
+
{%- elif effort is defined and effort is not none -%}
|
| 88 |
+
{{- '\n\n<effort>' ~ effort ~ '</effort>' }}
|
| 89 |
+
{%- endif -%}
|
| 90 |
+
{%- endmacro -%}
|
| 91 |
+
{%- macro build_developer_message(developer_message) -%}
|
| 92 |
+
{%- if developer_message and developer_message.content -%}
|
| 93 |
+
{{- visible_text(developer_message.content) }}
|
| 94 |
+
{%- else -%}
|
| 95 |
+
{%- if model_identity is not defined -%}
|
| 96 |
+
{%- set model_identity = "You are a helpful assistant." -%}
|
| 97 |
+
{%- endif -%}
|
| 98 |
+
{{- model_identity }}
|
| 99 |
+
{%- endif -%}
|
| 100 |
+
{%- endmacro -%}
|
| 101 |
+
{#- Main Template Logic ================================================= -#}
|
| 102 |
+
{#- Role mapping: root -> system sp (high priority), system/developer -> developer sp (low priority) -#}
|
| 103 |
+
{%- set system_message = none -%}
|
| 104 |
+
{%- set developer_message = none -%}
|
| 105 |
+
{%- set conversation_messages = messages -%}
|
| 106 |
+
{%- if messages and messages[0].role == "root" -%}
|
| 107 |
+
{%- set system_message = messages[0] -%}
|
| 108 |
+
{%- set conversation_messages = messages[1:] -%}
|
| 109 |
+
{%- if conversation_messages and conversation_messages[0].role in ["system", "developer"] -%}
|
| 110 |
+
{%- set developer_message = conversation_messages[0] -%}
|
| 111 |
+
{%- set conversation_messages = conversation_messages[1:] -%}
|
| 112 |
+
{%- endif -%}
|
| 113 |
+
{%- elif messages and messages[0].role in ["system", "developer"] -%}
|
| 114 |
+
{%- set developer_message = messages[0] -%}
|
| 115 |
+
{%- set conversation_messages = messages[1:] -%}
|
| 116 |
+
{%- endif -%}
|
| 117 |
+
{#- Render system sp (higher priority, root role only) -#}
|
| 118 |
+
{{- bod_token ~ bos_token ~ 'system' ~ '\n' }}
|
| 119 |
+
{{- build_system_message(system_message) }}
|
| 120 |
+
{{- eos_token ~ '\n' }}
|
| 121 |
+
|
| 122 |
+
{#- Render developer sp (lower priority: system/developer role + tools) -#}
|
| 123 |
+
{{- bos_token ~ 'developer' ~ '\n' }}
|
| 124 |
+
{{- build_developer_message(developer_message) }}
|
| 125 |
+
{%- if tools -%}
|
| 126 |
+
{{- '\n\n' ~ '# Tools' ~ '\n' ~ 'You may call one or more tools to assist with the user query.\nHere are the tools available in JSONSchema format:' ~ '\n' }}
|
| 127 |
+
{{- '\n' ~ '<tools>' ~ '\n' }}
|
| 128 |
+
{{- render_tool_namespace("functions", tools) }}
|
| 129 |
+
{{- '</tools>' ~ '\n\n' }}
|
| 130 |
+
{{- 'To call tools, wrap all invocations in a single ' ~ toolcall_begin_token ~ toolcall_end_token ~ ' block. Parameter values containing nested objects or arrays are recursively expanded into XML elements. Example:\n' }}
|
| 131 |
+
{{- '\n' ~ toolcall_begin_token ~ '\n' }}
|
| 132 |
+
{{- ns_token + '<invoke name="tool-name-1">' }}
|
| 133 |
+
{{- ns_token + '<param-1>value-1' + ns_token + '</param-1>' }}
|
| 134 |
+
{{- ns_token + '<param-2>' }}
|
| 135 |
+
{{- ns_token + '<item>' }}
|
| 136 |
+
{{- ns_token + '<key-a>val-a' + ns_token + '</key-a>' }}
|
| 137 |
+
{{- ns_token + '<key-b>val-b' + ns_token + '</key-b>' }}
|
| 138 |
+
{{- ns_token + '</item>' }}
|
| 139 |
+
{{- ns_token + '</param-2>' }}
|
| 140 |
+
{{- ns_token + '</invoke>\n' }}
|
| 141 |
+
{{- ns_token + '<invoke name="tool-name-2">' }}
|
| 142 |
+
{{- ns_token + '<param-1>value-1' + ns_token + '</param-1>' }}
|
| 143 |
+
{{- ns_token + '</invoke>\n' }}
|
| 144 |
+
{{- toolcall_end_token }}
|
| 145 |
+
{%- endif -%}
|
| 146 |
+
{{- eos_token ~ '\n' }}
|
| 147 |
+
|
| 148 |
+
{#- Render messages -#}
|
| 149 |
+
{%- set last_tool_call = namespace(name=none) -%}
|
| 150 |
+
{%- for message in conversation_messages -%}
|
| 151 |
+
{%- if message.role == 'assistant' -%}
|
| 152 |
+
{{- bos_token ~ 'ai' ~ '\n' }}
|
| 153 |
+
|
| 154 |
+
{%- set reasoning_content = '' %}
|
| 155 |
+
{%- set content = visible_text(message.content) %}
|
| 156 |
+
{%- if message.reasoning_content is string %}
|
| 157 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 158 |
+
{%- else %}
|
| 159 |
+
{%- if think_end_token in content %}
|
| 160 |
+
{%- set reasoning_content = content.split(think_end_token)[0].strip('\n').split(think_begin_token)[-1].strip('\n') %}
|
| 161 |
+
{%- set content = content.split(think_end_token)[-1].strip('\n') %}
|
| 162 |
+
{%- endif %}
|
| 163 |
+
{%- endif %}
|
| 164 |
+
|
| 165 |
+
{%- if reasoning_content -%}
|
| 166 |
+
{#- Render thinking for every assistant turn (all-turn visible) -#}
|
| 167 |
+
{{- think_begin_token ~ reasoning_content ~ think_end_token }}
|
| 168 |
+
{%- else -%}
|
| 169 |
+
{#- No thinking rendered → prefix with think_end_token -#}
|
| 170 |
+
{{- think_end_token }}
|
| 171 |
+
{%- endif -%}
|
| 172 |
+
|
| 173 |
+
{%- if content -%}
|
| 174 |
+
{{- content }}
|
| 175 |
+
{%- endif -%}
|
| 176 |
+
{%- if message.tool_calls -%}
|
| 177 |
+
{{- toolcall_begin_token ~ '\n' }}
|
| 178 |
+
|
| 179 |
+
{%- for tool_call in message.tool_calls -%}
|
| 180 |
+
{%- if tool_call.function -%}
|
| 181 |
+
{%- set tool_call = tool_call.function -%}
|
| 182 |
+
{%- endif -%}
|
| 183 |
+
{{- ns_token + '<invoke name="' + tool_call.name + '">' }}
|
| 184 |
+
{%- set _args = tool_call.arguments -%}
|
| 185 |
+
{%- for k, v in _args.items() if v is not none %}
|
| 186 |
+
{{- ns_token + '<' + k + '>' -}}
|
| 187 |
+
{{- to_xml(v, ns_token) -}}
|
| 188 |
+
{{- ns_token + '</' + k + '>' }}
|
| 189 |
+
{%- endfor -%}
|
| 190 |
+
{{- ns_token + '</invoke>' ~ '\n' }}
|
| 191 |
+
{%- endfor -%}
|
| 192 |
+
|
| 193 |
+
{{- toolcall_end_token }}
|
| 194 |
+
{%- if message.tool_calls[-1].function -%}
|
| 195 |
+
{%- set last_tool_call.name = message.tool_calls[-1].function.name -%}
|
| 196 |
+
{%- else -%}
|
| 197 |
+
{%- set last_tool_call.name = message.tool_calls[-1].name -%}
|
| 198 |
+
{%- endif -%}
|
| 199 |
+
{%- else -%}
|
| 200 |
+
{%- set last_tool_call.name = none -%}
|
| 201 |
+
{%- endif -%}
|
| 202 |
+
{{- eos_token ~ '\n' }}
|
| 203 |
+
|
| 204 |
+
{%- elif message.role == 'tool' -%}
|
| 205 |
+
{%- if last_tool_call.name is none -%}
|
| 206 |
+
{{- raise_exception("Message has tool role, but there was no previous assistant message with a tool call!") }}
|
| 207 |
+
{%- endif -%}
|
| 208 |
+
{%- if loop.first or (conversation_messages[loop.index0 - 1].role != 'tool') -%}
|
| 209 |
+
{{- bos_token ~ 'tool' }}
|
| 210 |
+
{%- endif -%}
|
| 211 |
+
{{- '\n<response>' }}
|
| 212 |
+
{%- if message.content is string -%}
|
| 213 |
+
{{- message.content }}
|
| 214 |
+
{%- else -%}
|
| 215 |
+
{%- for tr in message.content -%}
|
| 216 |
+
{%- if tr is mapping and tr.type is defined and tr.type == 'image' -%}
|
| 217 |
+
{{- image_token }}
|
| 218 |
+
{%- elif tr is mapping and tr.type is defined and tr.type == 'video' -%}
|
| 219 |
+
{{- video_token }}
|
| 220 |
+
{%- else -%}
|
| 221 |
+
{{- tr.output if tr.output is defined else (tr.text if tr.type == 'text' and tr.text is defined else tr) }}
|
| 222 |
+
{%- endif -%}
|
| 223 |
+
{%- endfor -%}
|
| 224 |
+
{%- endif -%}
|
| 225 |
+
{{- '</response>' }}
|
| 226 |
+
{%- if loop.last or (conversation_messages[loop.index0 + 1].role != 'tool') -%}
|
| 227 |
+
{{- eos_token ~ '\n' -}}
|
| 228 |
+
{%- endif -%}
|
| 229 |
+
|
| 230 |
+
{%- elif message.role in ['system', 'developer'] -%}
|
| 231 |
+
{#- Mid-conversation system/developer messages keep their position and
|
| 232 |
+
map to the lower-priority M3 developer role. -#}
|
| 233 |
+
{{- bos_token ~ 'developer' ~ '\n' }}
|
| 234 |
+
{{- build_developer_message(message) }}
|
| 235 |
+
{{- eos_token ~ '\n' }}
|
| 236 |
+
|
| 237 |
+
{%- elif message.role == 'user' -%}
|
| 238 |
+
{{- bos_token ~ 'user' ~ '\n' }}
|
| 239 |
+
{{- visible_text(message.content) }}
|
| 240 |
+
{{- eos_token ~ '\n' }}
|
| 241 |
+
{%- endif -%}
|
| 242 |
+
{%- endfor -%}
|
| 243 |
+
|
| 244 |
+
{#- Generation prompt -#}
|
| 245 |
+
{%- if add_generation_prompt -%}
|
| 246 |
+
{{- bos_token ~ 'ai' ~ '\n' }}
|
| 247 |
+
{#- adaptive: no prefix, let model decide -#}
|
| 248 |
+
{%- endif -%}
|
config.json
ADDED
|
@@ -0,0 +1,357 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"MiniMaxM3SparseForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"auto_map": {
|
| 6 |
+
"AutoConfig": "configuration_minimax_m3_vl.MiniMaxM3VLConfig"
|
| 7 |
+
},
|
| 8 |
+
"model_type": "minimax_m3_vl",
|
| 9 |
+
"text_config": {
|
| 10 |
+
"hidden_size": 6144,
|
| 11 |
+
"intermediate_size": 3072,
|
| 12 |
+
"num_hidden_layers": 60,
|
| 13 |
+
"num_attention_heads": 64,
|
| 14 |
+
"num_key_value_heads": 4,
|
| 15 |
+
"head_dim": 128,
|
| 16 |
+
"vocab_size": 200064,
|
| 17 |
+
"max_position_embeddings": 1048576,
|
| 18 |
+
"rms_norm_eps": 1e-06,
|
| 19 |
+
"use_gemma_norm": true,
|
| 20 |
+
"attention_output_gate": false,
|
| 21 |
+
"rope_theta": 10000000,
|
| 22 |
+
"rotary_dim": 64,
|
| 23 |
+
"partial_rotary_factor": 0.5,
|
| 24 |
+
"hidden_act": "swigluoai",
|
| 25 |
+
"use_qk_norm": true,
|
| 26 |
+
"tie_word_embeddings": false,
|
| 27 |
+
"dense_intermediate_size": 12288,
|
| 28 |
+
"shared_intermediate_size": 3072,
|
| 29 |
+
"num_local_experts": 128,
|
| 30 |
+
"num_experts_per_tok": 4,
|
| 31 |
+
"n_shared_experts": 1,
|
| 32 |
+
"scoring_func": "sigmoid",
|
| 33 |
+
"use_routing_bias": true,
|
| 34 |
+
"moe_layer_freq": [
|
| 35 |
+
0,
|
| 36 |
+
0,
|
| 37 |
+
0,
|
| 38 |
+
1,
|
| 39 |
+
1,
|
| 40 |
+
1,
|
| 41 |
+
1,
|
| 42 |
+
1,
|
| 43 |
+
1,
|
| 44 |
+
1,
|
| 45 |
+
1,
|
| 46 |
+
1,
|
| 47 |
+
1,
|
| 48 |
+
1,
|
| 49 |
+
1,
|
| 50 |
+
1,
|
| 51 |
+
1,
|
| 52 |
+
1,
|
| 53 |
+
1,
|
| 54 |
+
1,
|
| 55 |
+
1,
|
| 56 |
+
1,
|
| 57 |
+
1,
|
| 58 |
+
1,
|
| 59 |
+
1,
|
| 60 |
+
1,
|
| 61 |
+
1,
|
| 62 |
+
1,
|
| 63 |
+
1,
|
| 64 |
+
1,
|
| 65 |
+
1,
|
| 66 |
+
1,
|
| 67 |
+
1,
|
| 68 |
+
1,
|
| 69 |
+
1,
|
| 70 |
+
1,
|
| 71 |
+
1,
|
| 72 |
+
1,
|
| 73 |
+
1,
|
| 74 |
+
1,
|
| 75 |
+
1,
|
| 76 |
+
1,
|
| 77 |
+
1,
|
| 78 |
+
1,
|
| 79 |
+
1,
|
| 80 |
+
1,
|
| 81 |
+
1,
|
| 82 |
+
1,
|
| 83 |
+
1,
|
| 84 |
+
1,
|
| 85 |
+
1,
|
| 86 |
+
1,
|
| 87 |
+
1,
|
| 88 |
+
1,
|
| 89 |
+
1,
|
| 90 |
+
1,
|
| 91 |
+
1,
|
| 92 |
+
1,
|
| 93 |
+
1,
|
| 94 |
+
1
|
| 95 |
+
],
|
| 96 |
+
"qk_norm_type": "per_head",
|
| 97 |
+
"swiglu_alpha": 1.702,
|
| 98 |
+
"swiglu_limit": 7.0,
|
| 99 |
+
"routed_scaling_factor": 2.0,
|
| 100 |
+
"sparse_attention_config": {
|
| 101 |
+
"use_sparse_attention": true,
|
| 102 |
+
"sparse_index_dim": 128,
|
| 103 |
+
"sparse_num_index_heads": 4,
|
| 104 |
+
"sparse_topk_blocks": 16,
|
| 105 |
+
"sparse_block_size": 128,
|
| 106 |
+
"sparse_score_type": "max",
|
| 107 |
+
"sparse_init_block": 0,
|
| 108 |
+
"sparse_local_block": 1,
|
| 109 |
+
"sparse_attention_freq": [
|
| 110 |
+
1,
|
| 111 |
+
1,
|
| 112 |
+
1,
|
| 113 |
+
1,
|
| 114 |
+
1,
|
| 115 |
+
1,
|
| 116 |
+
1,
|
| 117 |
+
1,
|
| 118 |
+
1,
|
| 119 |
+
1,
|
| 120 |
+
1,
|
| 121 |
+
1,
|
| 122 |
+
1,
|
| 123 |
+
1,
|
| 124 |
+
1,
|
| 125 |
+
1,
|
| 126 |
+
1,
|
| 127 |
+
1,
|
| 128 |
+
1,
|
| 129 |
+
1,
|
| 130 |
+
1,
|
| 131 |
+
1,
|
| 132 |
+
1,
|
| 133 |
+
1,
|
| 134 |
+
1,
|
| 135 |
+
1,
|
| 136 |
+
1,
|
| 137 |
+
1,
|
| 138 |
+
1,
|
| 139 |
+
1,
|
| 140 |
+
1,
|
| 141 |
+
1,
|
| 142 |
+
1,
|
| 143 |
+
1,
|
| 144 |
+
1,
|
| 145 |
+
1,
|
| 146 |
+
1,
|
| 147 |
+
1,
|
| 148 |
+
1,
|
| 149 |
+
1,
|
| 150 |
+
1,
|
| 151 |
+
1,
|
| 152 |
+
1,
|
| 153 |
+
1,
|
| 154 |
+
1,
|
| 155 |
+
1,
|
| 156 |
+
1,
|
| 157 |
+
1,
|
| 158 |
+
1,
|
| 159 |
+
1,
|
| 160 |
+
1,
|
| 161 |
+
1,
|
| 162 |
+
1,
|
| 163 |
+
1,
|
| 164 |
+
1,
|
| 165 |
+
1,
|
| 166 |
+
1,
|
| 167 |
+
1,
|
| 168 |
+
1,
|
| 169 |
+
1
|
| 170 |
+
],
|
| 171 |
+
"sparse_disable_index_value": [
|
| 172 |
+
1,
|
| 173 |
+
1,
|
| 174 |
+
1,
|
| 175 |
+
1,
|
| 176 |
+
1,
|
| 177 |
+
1,
|
| 178 |
+
1,
|
| 179 |
+
1,
|
| 180 |
+
1,
|
| 181 |
+
1,
|
| 182 |
+
1,
|
| 183 |
+
1,
|
| 184 |
+
1,
|
| 185 |
+
1,
|
| 186 |
+
1,
|
| 187 |
+
1,
|
| 188 |
+
1,
|
| 189 |
+
1,
|
| 190 |
+
1,
|
| 191 |
+
1,
|
| 192 |
+
1,
|
| 193 |
+
1,
|
| 194 |
+
1,
|
| 195 |
+
1,
|
| 196 |
+
1,
|
| 197 |
+
1,
|
| 198 |
+
1,
|
| 199 |
+
1,
|
| 200 |
+
1,
|
| 201 |
+
1,
|
| 202 |
+
1,
|
| 203 |
+
1,
|
| 204 |
+
1,
|
| 205 |
+
1,
|
| 206 |
+
1,
|
| 207 |
+
1,
|
| 208 |
+
1,
|
| 209 |
+
1,
|
| 210 |
+
1,
|
| 211 |
+
1,
|
| 212 |
+
1,
|
| 213 |
+
1,
|
| 214 |
+
1,
|
| 215 |
+
1,
|
| 216 |
+
1,
|
| 217 |
+
1,
|
| 218 |
+
1,
|
| 219 |
+
1,
|
| 220 |
+
1,
|
| 221 |
+
1,
|
| 222 |
+
1,
|
| 223 |
+
1,
|
| 224 |
+
1,
|
| 225 |
+
1,
|
| 226 |
+
1,
|
| 227 |
+
1,
|
| 228 |
+
1,
|
| 229 |
+
1,
|
| 230 |
+
1,
|
| 231 |
+
1
|
| 232 |
+
]
|
| 233 |
+
},
|
| 234 |
+
"architectures": [
|
| 235 |
+
"MiniMaxM3SparseForCausalLM"
|
| 236 |
+
]
|
| 237 |
+
},
|
| 238 |
+
"vision_config": {
|
| 239 |
+
"hidden_size": 1280,
|
| 240 |
+
"num_attention_heads": 16,
|
| 241 |
+
"num_hidden_layers": 32,
|
| 242 |
+
"intermediate_size": 5120,
|
| 243 |
+
"patch_size": 14,
|
| 244 |
+
"image_size": 672,
|
| 245 |
+
"projection_dim": 6144,
|
| 246 |
+
"position_embedding_type": "rope",
|
| 247 |
+
"rope_mode": "3d",
|
| 248 |
+
"rope_theta": 10000.0,
|
| 249 |
+
"attention_dropout": 0.0,
|
| 250 |
+
"hidden_act": "gelu",
|
| 251 |
+
"initializer_factor": 1.0,
|
| 252 |
+
"initializer_range": 0.02,
|
| 253 |
+
"layer_norm_eps": 1e-05,
|
| 254 |
+
"model_type": "clip_vision_model",
|
| 255 |
+
"num_channels": 3,
|
| 256 |
+
"vocab_size": 32000,
|
| 257 |
+
"img_token_compression_config": {
|
| 258 |
+
"image_token_compression_method": "patch_merge",
|
| 259 |
+
"spatial_merge_size": 2,
|
| 260 |
+
"temporal_patch_size": 2
|
| 261 |
+
},
|
| 262 |
+
"vision_segment_max_frames": 4
|
| 263 |
+
},
|
| 264 |
+
"img_token_compression_config": {
|
| 265 |
+
"image_token_compression_method": "patch_merge",
|
| 266 |
+
"spatial_merge_size": 2,
|
| 267 |
+
"temporal_patch_size": 2
|
| 268 |
+
},
|
| 269 |
+
"image_grid_pinpoints": "[(336, 336), (336, 672), (336, 1008), (336, 1344), (336, 1680), (336, 2016), (672, 336), (672, 672), (672, 1008), (672, 1344), (672, 1680), (672, 2016), (1008, 336), (1008, 672), (1008, 1008), (1008, 1344), (1008, 1680), (1008, 2016), (1344, 336), (1344, 672), (1344, 1008), (1344, 1344), (1344, 1680), (1344, 2016), (1680, 336), (1680, 672), (1680, 1008), (1680, 1344), (1680, 1680), (1680, 2016), (2016, 336), (2016, 672), (2016, 1008), (2016, 1344), (2016, 1680), (2016, 2016)]",
|
| 270 |
+
"image_seq_length": 576,
|
| 271 |
+
"image_token_index": 200025,
|
| 272 |
+
"video_token_index": 200026,
|
| 273 |
+
"multimodal_projector_bias": true,
|
| 274 |
+
"num_reward_heads": 0,
|
| 275 |
+
"process_image_mode": "dynamic_res",
|
| 276 |
+
"projector_hidden_act": "gelu",
|
| 277 |
+
"vision_feature_layer": -1,
|
| 278 |
+
"vision_feature_select_strategy": "full",
|
| 279 |
+
"torch_dtype": "bfloat16",
|
| 280 |
+
"transformers_version": "4.52.4",
|
| 281 |
+
"projector_hidden_size": 6144,
|
| 282 |
+
"quantization_config": {
|
| 283 |
+
"quant_method": "mxfp8",
|
| 284 |
+
"activation_scheme": "dynamic",
|
| 285 |
+
"weight_block_size": [
|
| 286 |
+
1,
|
| 287 |
+
32
|
| 288 |
+
],
|
| 289 |
+
"ignored_layers": [
|
| 290 |
+
"language_model.model.layers.10.block_sparse_moe.gate",
|
| 291 |
+
"language_model.model.layers.11.block_sparse_moe.gate",
|
| 292 |
+
"language_model.model.layers.12.block_sparse_moe.gate",
|
| 293 |
+
"language_model.model.layers.13.block_sparse_moe.gate",
|
| 294 |
+
"language_model.model.layers.14.block_sparse_moe.gate",
|
| 295 |
+
"language_model.model.layers.15.block_sparse_moe.gate",
|
| 296 |
+
"language_model.model.layers.16.block_sparse_moe.gate",
|
| 297 |
+
"language_model.model.layers.17.block_sparse_moe.gate",
|
| 298 |
+
"language_model.model.layers.18.block_sparse_moe.gate",
|
| 299 |
+
"language_model.model.layers.19.block_sparse_moe.gate",
|
| 300 |
+
"language_model.model.layers.20.block_sparse_moe.gate",
|
| 301 |
+
"language_model.model.layers.21.block_sparse_moe.gate",
|
| 302 |
+
"language_model.model.layers.22.block_sparse_moe.gate",
|
| 303 |
+
"language_model.model.layers.23.block_sparse_moe.gate",
|
| 304 |
+
"language_model.model.layers.24.block_sparse_moe.gate",
|
| 305 |
+
"language_model.model.layers.25.block_sparse_moe.gate",
|
| 306 |
+
"language_model.model.layers.26.block_sparse_moe.gate",
|
| 307 |
+
"language_model.model.layers.27.block_sparse_moe.gate",
|
| 308 |
+
"language_model.model.layers.28.block_sparse_moe.gate",
|
| 309 |
+
"language_model.model.layers.29.block_sparse_moe.gate",
|
| 310 |
+
"language_model.model.layers.3.block_sparse_moe.gate",
|
| 311 |
+
"language_model.model.layers.30.block_sparse_moe.gate",
|
| 312 |
+
"language_model.model.layers.31.block_sparse_moe.gate",
|
| 313 |
+
"language_model.model.layers.32.block_sparse_moe.gate",
|
| 314 |
+
"language_model.model.layers.33.block_sparse_moe.gate",
|
| 315 |
+
"language_model.model.layers.34.block_sparse_moe.gate",
|
| 316 |
+
"language_model.model.layers.35.block_sparse_moe.gate",
|
| 317 |
+
"language_model.model.layers.36.block_sparse_moe.gate",
|
| 318 |
+
"language_model.model.layers.37.block_sparse_moe.gate",
|
| 319 |
+
"language_model.model.layers.38.block_sparse_moe.gate",
|
| 320 |
+
"language_model.model.layers.39.block_sparse_moe.gate",
|
| 321 |
+
"language_model.model.layers.4.block_sparse_moe.gate",
|
| 322 |
+
"language_model.model.layers.40.block_sparse_moe.gate",
|
| 323 |
+
"language_model.model.layers.41.block_sparse_moe.gate",
|
| 324 |
+
"language_model.model.layers.42.block_sparse_moe.gate",
|
| 325 |
+
"language_model.model.layers.43.block_sparse_moe.gate",
|
| 326 |
+
"language_model.model.layers.44.block_sparse_moe.gate",
|
| 327 |
+
"language_model.model.layers.45.block_sparse_moe.gate",
|
| 328 |
+
"language_model.model.layers.46.block_sparse_moe.gate",
|
| 329 |
+
"language_model.model.layers.47.block_sparse_moe.gate",
|
| 330 |
+
"language_model.model.layers.48.block_sparse_moe.gate",
|
| 331 |
+
"language_model.model.layers.49.block_sparse_moe.gate",
|
| 332 |
+
"language_model.model.layers.5.block_sparse_moe.gate",
|
| 333 |
+
"language_model.model.layers.50.block_sparse_moe.gate",
|
| 334 |
+
"language_model.model.layers.51.block_sparse_moe.gate",
|
| 335 |
+
"language_model.model.layers.52.block_sparse_moe.gate",
|
| 336 |
+
"language_model.model.layers.53.block_sparse_moe.gate",
|
| 337 |
+
"language_model.model.layers.54.block_sparse_moe.gate",
|
| 338 |
+
"language_model.model.layers.55.block_sparse_moe.gate",
|
| 339 |
+
"language_model.model.layers.56.block_sparse_moe.gate",
|
| 340 |
+
"language_model.model.layers.57.block_sparse_moe.gate",
|
| 341 |
+
"language_model.model.layers.58.block_sparse_moe.gate",
|
| 342 |
+
"language_model.model.layers.59.block_sparse_moe.gate",
|
| 343 |
+
"language_model.model.layers.6.block_sparse_moe.gate",
|
| 344 |
+
"language_model.model.layers.7.block_sparse_moe.gate",
|
| 345 |
+
"language_model.model.layers.8.block_sparse_moe.gate",
|
| 346 |
+
"language_model.model.layers.9.block_sparse_moe.gate",
|
| 347 |
+
"lm_head",
|
| 348 |
+
"model.embed_tokens",
|
| 349 |
+
"multi_modal_projector",
|
| 350 |
+
"patch_merge_mlp",
|
| 351 |
+
"vision_tower"
|
| 352 |
+
],
|
| 353 |
+
"group_size": 16,
|
| 354 |
+
"moe_quant_algo": "NVFP4",
|
| 355 |
+
"moe_quant_format": "nvfp4-pack-quantized"
|
| 356 |
+
}
|
| 357 |
+
}
|
configuration_minimax_m3_vl.py
ADDED
|
@@ -0,0 +1,111 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""HuggingFace configs for the MiniMax VL family (M2 VL / M3 VL).
|
| 2 |
+
|
| 3 |
+
This file is bundled into every converted HF checkpoint so that loading via
|
| 4 |
+
``AutoConfig.from_pretrained(..., trust_remote_code=True)`` works without any
|
| 5 |
+
runtime dependency on sglang or other internal packages — only stock
|
| 6 |
+
``transformers`` is required.
|
| 7 |
+
|
| 8 |
+
The class definitions intentionally mirror
|
| 9 |
+
``sglang.srt.configs.minimax_vl``; if either side changes, keep them in sync.
|
| 10 |
+
|
| 11 |
+
The file is named ``configuration_minimax_m3_vl.py`` (matching the legacy
|
| 12 |
+
``model_type="minimax_m3_vl"`` and the converter's ``auto_map`` entry) so
|
| 13 |
+
that ckpts produced by this converter remain loadable by older sglang versions
|
| 14 |
+
that only know the ``MiniMaxM3VL*`` names. The canonical class is
|
| 15 |
+
``MiniMaxM3VLConfig``; ``MiniMaxM3VLConfig`` is a thin BC alias whose only
|
| 16 |
+
purpose is to be referenced from ``auto_map``.
|
| 17 |
+
"""
|
| 18 |
+
|
| 19 |
+
from typing import Optional
|
| 20 |
+
|
| 21 |
+
from transformers.configuration_utils import PretrainedConfig
|
| 22 |
+
from transformers.models.auto import CONFIG_MAPPING
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def _coerce_sub_config(
|
| 26 |
+
sub_config: Optional[dict], default_model_type: str
|
| 27 |
+
) -> Optional[PretrainedConfig]:
|
| 28 |
+
"""Convert a config dict to a ``PretrainedConfig`` instance.
|
| 29 |
+
|
| 30 |
+
If ``model_type`` is registered in HF ``CONFIG_MAPPING`` the corresponding
|
| 31 |
+
config class is used; otherwise we fall back to a generic
|
| 32 |
+
``PretrainedConfig`` so all dict keys still become real attributes (M3's
|
| 33 |
+
text backbone uses ``model_type="minimax_m2"`` which is not in
|
| 34 |
+
``CONFIG_MAPPING``).
|
| 35 |
+
"""
|
| 36 |
+
if not isinstance(sub_config, dict):
|
| 37 |
+
return sub_config
|
| 38 |
+
model_type = sub_config.get("model_type", default_model_type)
|
| 39 |
+
cls = CONFIG_MAPPING.get(model_type, PretrainedConfig)
|
| 40 |
+
return cls(**sub_config)
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
class MiniMaxVLBaseConfig(PretrainedConfig):
|
| 44 |
+
"""Base config shared by every MiniMax VL variant.
|
| 45 |
+
|
| 46 |
+
Handles vision/text sub-config coercion. Concrete subclasses only need to
|
| 47 |
+
declare a unique ``model_type`` string.
|
| 48 |
+
"""
|
| 49 |
+
|
| 50 |
+
def __init__(
|
| 51 |
+
self,
|
| 52 |
+
vision_config: Optional[dict] = None,
|
| 53 |
+
text_config: Optional[dict] = None,
|
| 54 |
+
image_token_index: int = 200025,
|
| 55 |
+
video_token_index: int = 200026,
|
| 56 |
+
image_seq_length: int = 576,
|
| 57 |
+
process_image_mode: str = "dynamic_res",
|
| 58 |
+
projector_hidden_act: str = "gelu",
|
| 59 |
+
multimodal_projector_bias: bool = True,
|
| 60 |
+
vision_feature_layer: int = -1,
|
| 61 |
+
vision_feature_select_strategy: str = "full",
|
| 62 |
+
img_token_compression_config: Optional[dict] = None,
|
| 63 |
+
image_grid_pinpoints: Optional[str] = None,
|
| 64 |
+
**kwargs,
|
| 65 |
+
):
|
| 66 |
+
self.vision_config = _coerce_sub_config(vision_config, "clip_vision_model")
|
| 67 |
+
self.text_config = _coerce_sub_config(text_config, "mixtral")
|
| 68 |
+
|
| 69 |
+
self.image_token_index = image_token_index
|
| 70 |
+
self.video_token_index = video_token_index
|
| 71 |
+
self.image_seq_length = image_seq_length
|
| 72 |
+
self.process_image_mode = process_image_mode
|
| 73 |
+
self.projector_hidden_act = projector_hidden_act
|
| 74 |
+
self.multimodal_projector_bias = multimodal_projector_bias
|
| 75 |
+
self.vision_feature_layer = vision_feature_layer
|
| 76 |
+
self.vision_feature_select_strategy = vision_feature_select_strategy
|
| 77 |
+
self.img_token_compression_config = img_token_compression_config or {}
|
| 78 |
+
self.image_grid_pinpoints = image_grid_pinpoints
|
| 79 |
+
|
| 80 |
+
super().__init__(**kwargs)
|
| 81 |
+
|
| 82 |
+
def __post_init__(self, **kwargs):
|
| 83 |
+
super().__post_init__(**kwargs)
|
| 84 |
+
if hasattr(self, "vision_config"):
|
| 85 |
+
self.vision_config = _coerce_sub_config(self.vision_config, "clip_vision_model")
|
| 86 |
+
if hasattr(self, "text_config"):
|
| 87 |
+
self.text_config = _coerce_sub_config(self.text_config, "mixtral")
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
class MiniMaxM2VLConfig(MiniMaxVLBaseConfig):
|
| 91 |
+
"""MiniMax M2 VL: vision tower + M2 (Mixtral-style MoE) text backbone."""
|
| 92 |
+
|
| 93 |
+
model_type = "minimax_m2_vl"
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
class MiniMaxM3VLConfig(MiniMaxVLBaseConfig):
|
| 97 |
+
"""MiniMax M3 VL: vision tower + M3 (mixed sparse/dense MoE) text backbone."""
|
| 98 |
+
|
| 99 |
+
model_type = "minimax_m3_vl"
|
| 100 |
+
|
| 101 |
+
|
| 102 |
+
class MiniMaxM2MiniVLConfig(MiniMaxM2VLConfig):
|
| 103 |
+
"""Legacy alias kept so old ``model_type="minimax_m2_mini_vl"`` ckpts load."""
|
| 104 |
+
|
| 105 |
+
model_type = "minimax_m2_mini_vl"
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
class MiniMaxM3VLConfig(MiniMaxM3VLConfig):
|
| 109 |
+
"""Legacy alias kept so old ``model_type="minimax_m3_vl"`` ckpts load."""
|
| 110 |
+
|
| 111 |
+
model_type = "minimax_m3_vl"
|
docs/FlagOS-Multi-Chip-Support-and-Downloads.md
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
### Multi-Chip Deployment with FlagOS
|
| 2 |
+
|
| 3 |
+
MiniMax-M3.1-Flash supports multi-chip deployment through FlagOS. The FlagOS community provides **W8A8 quantized versions** for platforms from **nine chip vendors**. The corresponding model repositories provide access to weights, deployment code, and container image instructions.
|
| 4 |
+
|
| 5 |
+
#### About FlagOS
|
| 6 |
+
|
| 7 |
+
The FlagOS Open Source Community was jointly established by the Beijing Academy of Artificial Intelligence (BAAI), research institutions, chip manufacturers, system vendors, and algorithm and software organizations. It develops a unified, open-source system software stack for diverse AI chips.
|
| 8 |
+
|
| 9 |
+
FlagOS includes the FlagGems operator library, FlagTree AI compiler, FlagScale parallel training and inference framework, and FlagCX communication library. These projects connect the model, system, and chip layers, helping developers reuse model code and software components while reducing migration and maintenance costs. For inference deployment, vLLM-plugin-FL and SGLang-plugin-FL extend hardware support while preserving the original framework interfaces and usage patterns.
|
| 10 |
+
|
| 11 |
+
Website: [flagos.io](https://flagos.io/) | Community: [GitHub](https://github.com/flagos-ai) / [GitCode](https://gitcode.com/flagos-ai)
|
| 12 |
+
|
| 13 |
+
#### Multi-Chip Support and Downloads
|
| 14 |
+
|
| 15 |
+
FlagRelease is the FlagOS team's platform for model migration, adaptation, and release across AI chips. MiniMax-M3.1-Flash is adapted for platforms from **nine vendors: T-Head, NVIDIA, MetaX, Hygon, Huawei Ascend, Moore Threads, Iluvatar, Kunlunxin, and Tsingmicro**, with W8A8 versions provided across these platforms.
|
| 16 |
+
|
| 17 |
+
FlagOS converts and requantizes the model's original QAT NVFP4 weights into W8A8 deployment versions, providing a common quantization format across supported chips. The table below lists the model repositories. NVIDIA, Huawei Ascend, Kunlunxin, and T-Head (Zhenwu) use Express versions, named `MiniMax-M3.1-Flash-W8A8-<platform>-FlagOS-Express`; the other platforms use `MiniMax-M3.1-Flash-W8A8-<platform>-FlagOS`.
|
| 18 |
+
|
| 19 |
+
| Chip Vendor / Platform | ModelScope | Hugging Face |
|
| 20 |
+
| --- | --- | --- |
|
| 21 |
+
| T-Head (Zhenwu) | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-zhenwu-FlagOS-Express) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-zhenwu-FlagOS-Express) |
|
| 22 |
+
| NVIDIA | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-nvidia-FlagOS-Express) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-nvidia-FlagOS-Express) |
|
| 23 |
+
| MetaX | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-metax-FlagOS) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-metax-FlagOS) |
|
| 24 |
+
| Hygon | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-hygon-FlagOS) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-hygon-FlagOS) |
|
| 25 |
+
| Huawei Ascend | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-ascend-FlagOS-Express) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-ascend-FlagOS-Express) |
|
| 26 |
+
| Moore Threads | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-mthreads-FlagOS) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-mthreads-FlagOS) |
|
| 27 |
+
| Iluvatar | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-iluvatar-FlagOS) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-iluvatar-FlagOS) |
|
| 28 |
+
| Kunlunxin | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-kunlunxin-FlagOS-Express) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-kunlunxin-FlagOS-Express) |
|
| 29 |
+
| Tsingmicro | [Model repository](https://modelscope.cn/models/FlagRelease/MiniMax-M3.1-Flash-W8A8-tsingmicro-FlagOS) | [Model repository](https://huggingface.co/FlagRelease/MiniMax-M3.1-Flash-W8A8-tsingmicro-FlagOS) |
|
| 30 |
+
|
| 31 |
+
#### Installation and Usage
|
| 32 |
+
|
| 33 |
+
##### Option 1: Use a FlagRelease Image (Recommended)
|
| 34 |
+
|
| 35 |
+
FlagRelease provides deployment environments for the supported platforms:
|
| 36 |
+
|
| 37 |
+
1. Select your target chip platform from the table above and open the corresponding model repository.
|
| 38 |
+
2. Follow the model card to prepare the host drivers and runtime, download the model weights, pull and start the container image, and configure device mappings and model directory mounts.
|
| 39 |
+
3. Start the inference service or run the examples inside the container using the commands in the model card. Refer to that card for accelerator count, memory requirements, parallelism, model paths, framework versions, and startup parameters.
|
| 40 |
+
|
| 41 |
+
##### Option 2: Install from Source
|
| 42 |
+
|
| 43 |
+
MiniMax-M3.1-Flash supports deployment through vLLM-plugin-FL and SGLang-plugin-FL. Refer to the corresponding model card for framework versions, plugin branches, and model-specific dependencies.
|
| 44 |
+
|
| 45 |
+
Official developer guides:
|
| 46 |
+
|
| 47 |
+
- **vLLM-plugin-FL**: [GitHub](https://github.com/flagos-ai/vllm-plugin-FL/blob/main/README.md) / [GitCode](https://gitcode.com/flagos-ai/vllm-plugin-FL/blob/main/README.md)
|
| 48 |
+
- **SGLang-plugin-FL**: [GitHub](https://github.com/flagos-ai/sglang-plugin-FL/blob/main/README.md) / [GitCode](https://gitcode.com/flagos-ai/sglang-plugin-FL/blob/main/README.md)
|
| 49 |
+
|
| 50 |
+
The following example installs the base components used for the **T-Head platform** in this adaptation: **vLLM v0.24.0, vllm-plugin-FL v0.3.0, and FlagGems v5.4.0**. First prepare the chip drivers, runtime, compatible PyTorch build, and build dependencies as described in the model card. Apply any model-specific patches and additional configuration required by that card.
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
# 1. Install vLLM v0.24.0 (T-Head example)
|
| 54 |
+
git clone --branch v0.24.0 https://github.com/vllm-project/vllm.git
|
| 55 |
+
cd vllm
|
| 56 |
+
VLLM_TARGET_DEVICE=empty python -m pip install -v --no-build-isolation --no-deps .
|
| 57 |
+
cd ..
|
| 58 |
+
|
| 59 |
+
# 2. Install vllm-plugin-FL v0.3.0
|
| 60 |
+
git clone --branch v0.3.0 https://github.com/flagos-ai/vllm-plugin-FL.git
|
| 61 |
+
cd vllm-plugin-FL
|
| 62 |
+
python -m pip install --no-build-isolation -e .
|
| 63 |
+
cd ..
|
| 64 |
+
|
| 65 |
+
# 3. Install FlagGems v5.4.0
|
| 66 |
+
python -m pip install scikit-build-core==0.11 pybind11 ninja cmake
|
| 67 |
+
git clone --branch v5.4.0 https://github.com/flagos-ai/FlagGems.git
|
| 68 |
+
cd FlagGems
|
| 69 |
+
python -m pip install --no-build-isolation -e .
|
| 70 |
+
cd ..
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
Install the [FlagTree unified compiler](https://github.com/flagos-ai/FlagTree) and [FlagCX communication library](https://github.com/flagos-ai/FlagCX) as required by your platform, and complete the environment and communication configuration described in the model card. For CUDA/HIP environments that use PyTorch's CUDA dispatch key, refer to the plugin installation guide for native extension build instructions.
|
| 74 |
+
|
| 75 |
+
##### Run Inference
|
| 76 |
+
|
| 77 |
+
The following text generation example uses **MiniMax-M3.1-Flash W8A8 Express** on T-Head with `tensor_parallel_size=8`. Run it in a configured vLLM-plugin-FL environment or the corresponding container, on hardware that meets the model card's requirements. Accelerator counts and parallelism settings for other platforms are specified in their model cards.
|
| 78 |
+
|
| 79 |
+
Save the code as `inference.py`:
|
| 80 |
+
|
| 81 |
+
```python
|
| 82 |
+
from vllm import LLM, SamplingParams
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
if __name__ == "__main__":
|
| 86 |
+
prompts = ["Please introduce MiniMax-M3.1-Flash."]
|
| 87 |
+
sampling_params = SamplingParams(max_tokens=512, temperature=0.0)
|
| 88 |
+
|
| 89 |
+
llm = LLM(
|
| 90 |
+
model="FlagRelease/MiniMax-M3.1-Flash-W8A8-zhenwu-FlagOS-Express",
|
| 91 |
+
tensor_parallel_size=8,
|
| 92 |
+
trust_remote_code=True,
|
| 93 |
+
)
|
| 94 |
+
|
| 95 |
+
outputs = llm.generate(prompts, sampling_params)
|
| 96 |
+
for output in outputs:
|
| 97 |
+
print(f"Prompt: {output.prompt!r}")
|
| 98 |
+
print(f"Generated: {output.outputs[0].text!r}")
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
```bash
|
| 102 |
+
VLLM_PLUGINS=fl python inference.py
|
| 103 |
+
```
|
| 104 |
+
|
| 105 |
+
To use local weights, replace `model` with the downloaded model directory. When changing chip platforms, select the corresponding repository from the table and adjust `tensor_parallel_size`, `max_model_len`, `max_num_batched_tokens`, and `max_num_seqs` according to its model card. Refer to the platform's model card for online serving, chat templates, multimodal inputs, DSpark speculative decoding, and `reasoning_effort` configuration.
|
dspark/config.json
ADDED
|
@@ -0,0 +1,297 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"transformers_version": "5.12.1",
|
| 3 |
+
"architectures": [
|
| 4 |
+
"DSparkMiniMaxDraftModel"
|
| 5 |
+
],
|
| 6 |
+
"output_hidden_states": false,
|
| 7 |
+
"return_dict": true,
|
| 8 |
+
"dtype": "bfloat16",
|
| 9 |
+
"chunk_size_feed_forward": 0,
|
| 10 |
+
"is_encoder_decoder": false,
|
| 11 |
+
"id2label": {
|
| 12 |
+
"0": "LABEL_0",
|
| 13 |
+
"1": "LABEL_1"
|
| 14 |
+
},
|
| 15 |
+
"label2id": {
|
| 16 |
+
"LABEL_0": 0,
|
| 17 |
+
"LABEL_1": 1
|
| 18 |
+
},
|
| 19 |
+
"problem_type": null,
|
| 20 |
+
"model_type": "minimax_m3_vl",
|
| 21 |
+
"text_config": {
|
| 22 |
+
"architectures": [
|
| 23 |
+
"DSparkMiniMaxDraftModel"
|
| 24 |
+
],
|
| 25 |
+
"output_hidden_states": false,
|
| 26 |
+
"return_dict": true,
|
| 27 |
+
"dtype": "bfloat16",
|
| 28 |
+
"chunk_size_feed_forward": 0,
|
| 29 |
+
"is_encoder_decoder": false,
|
| 30 |
+
"id2label": {
|
| 31 |
+
"0": "LABEL_0",
|
| 32 |
+
"1": "LABEL_1"
|
| 33 |
+
},
|
| 34 |
+
"label2id": {
|
| 35 |
+
"LABEL_0": 0,
|
| 36 |
+
"LABEL_1": 1
|
| 37 |
+
},
|
| 38 |
+
"problem_type": null,
|
| 39 |
+
"_name_or_path": "",
|
| 40 |
+
"hidden_size": 6144,
|
| 41 |
+
"intermediate_size": 3072,
|
| 42 |
+
"num_hidden_layers": 5,
|
| 43 |
+
"num_attention_heads": 64,
|
| 44 |
+
"num_key_value_heads": 4,
|
| 45 |
+
"head_dim": 128,
|
| 46 |
+
"vocab_size": 200064,
|
| 47 |
+
"max_position_embeddings": 1048576,
|
| 48 |
+
"rms_norm_eps": 1e-06,
|
| 49 |
+
"use_gemma_norm": true,
|
| 50 |
+
"attention_output_gate": false,
|
| 51 |
+
"rope_theta": 10000000,
|
| 52 |
+
"rotary_dim": 64,
|
| 53 |
+
"partial_rotary_factor": 0.5,
|
| 54 |
+
"hidden_act": "swigluoai",
|
| 55 |
+
"use_qk_norm": true,
|
| 56 |
+
"tie_word_embeddings": false,
|
| 57 |
+
"dense_intermediate_size": 12288,
|
| 58 |
+
"shared_intermediate_size": 3072,
|
| 59 |
+
"num_local_experts": 128,
|
| 60 |
+
"num_experts_per_tok": 4,
|
| 61 |
+
"n_shared_experts": 1,
|
| 62 |
+
"scoring_func": "sigmoid",
|
| 63 |
+
"use_routing_bias": true,
|
| 64 |
+
"moe_layer_freq": [
|
| 65 |
+
0,
|
| 66 |
+
0,
|
| 67 |
+
0,
|
| 68 |
+
0,
|
| 69 |
+
0
|
| 70 |
+
],
|
| 71 |
+
"qk_norm_type": "per_head",
|
| 72 |
+
"swiglu_alpha": 1.702,
|
| 73 |
+
"swiglu_limit": 7.0,
|
| 74 |
+
"routed_scaling_factor": 2.0,
|
| 75 |
+
"sparse_attention_config": null,
|
| 76 |
+
"dspark_block_size": 7,
|
| 77 |
+
"dspark_markov_rank": 256,
|
| 78 |
+
"dspark_markov_head_type": "vanilla",
|
| 79 |
+
"dspark_aux_gemm_precision": "bf16",
|
| 80 |
+
"dspark_noise_token_id": 200058,
|
| 81 |
+
"dspark_use_dense_ffn": true,
|
| 82 |
+
"dspark_ffn_hidden_size": 12288,
|
| 83 |
+
"dspark_target_layer_ids": [
|
| 84 |
+
3,
|
| 85 |
+
17,
|
| 86 |
+
31,
|
| 87 |
+
45,
|
| 88 |
+
59
|
| 89 |
+
],
|
| 90 |
+
"dspark_config": {
|
| 91 |
+
"markov_rank": 256,
|
| 92 |
+
"markov_head_type": "vanilla",
|
| 93 |
+
"mask_token_id": 200058,
|
| 94 |
+
"hybrid_context_fusion": false,
|
| 95 |
+
"aux_gemm_precision": "bf16",
|
| 96 |
+
"use_dense_ffn": true,
|
| 97 |
+
"ffn_hidden_size": 12288
|
| 98 |
+
},
|
| 99 |
+
"dspark_hybrid_context_fusion": false,
|
| 100 |
+
"dflash_config": {
|
| 101 |
+
"num_hidden_layers": 5,
|
| 102 |
+
"num_target_layers": 60,
|
| 103 |
+
"context_source": "target_layers",
|
| 104 |
+
"block_size": 7,
|
| 105 |
+
"mask_token_id": 200058,
|
| 106 |
+
"target_layer_ids": [
|
| 107 |
+
3,
|
| 108 |
+
17,
|
| 109 |
+
31,
|
| 110 |
+
45,
|
| 111 |
+
59
|
| 112 |
+
],
|
| 113 |
+
"layer_types": [
|
| 114 |
+
"sliding_attention",
|
| 115 |
+
"sliding_attention",
|
| 116 |
+
"sliding_attention",
|
| 117 |
+
"sliding_attention",
|
| 118 |
+
"sliding_attention"
|
| 119 |
+
],
|
| 120 |
+
"sliding_window": 4096
|
| 121 |
+
},
|
| 122 |
+
"enable_confidence_head": true,
|
| 123 |
+
"confidence_head_with_markov": true,
|
| 124 |
+
"num_nextn_predict_layers": 0,
|
| 125 |
+
"use_mtp": false,
|
| 126 |
+
"num_mtp_modules": 0,
|
| 127 |
+
"layer_types": [
|
| 128 |
+
"sliding_attention",
|
| 129 |
+
"sliding_attention",
|
| 130 |
+
"sliding_attention",
|
| 131 |
+
"sliding_attention",
|
| 132 |
+
"sliding_attention"
|
| 133 |
+
],
|
| 134 |
+
"sliding_window": 4096,
|
| 135 |
+
"v_head_dim": 128,
|
| 136 |
+
"swa_head_dim": 128,
|
| 137 |
+
"swa_v_head_dim": 128,
|
| 138 |
+
"model_type": "",
|
| 139 |
+
"output_attentions": false
|
| 140 |
+
},
|
| 141 |
+
"img_token_compression_config": {
|
| 142 |
+
"image_token_compression_method": "patch_merge",
|
| 143 |
+
"spatial_merge_size": 2,
|
| 144 |
+
"temporal_patch_size": 2
|
| 145 |
+
},
|
| 146 |
+
"image_grid_pinpoints": "[(336, 336), (336, 672), (336, 1008), (336, 1344), (336, 1680), (336, 2016), (672, 336), (672, 672), (672, 1008), (672, 1344), (672, 1680), (672, 2016), (1008, 336), (1008, 672), (1008, 1008), (1008, 1344), (1008, 1680), (1008, 2016), (1344, 336), (1344, 672), (1344, 1008), (1344, 1344), (1344, 1680), (1344, 2016), (1680, 336), (1680, 672), (1680, 1008), (1680, 1344), (1680, 1680), (1680, 2016), (2016, 336), (2016, 672), (2016, 1008), (2016, 1344), (2016, 1680), (2016, 2016)]",
|
| 147 |
+
"image_seq_length": 576,
|
| 148 |
+
"image_token_index": 200025,
|
| 149 |
+
"video_token_index": 200026,
|
| 150 |
+
"multimodal_projector_bias": true,
|
| 151 |
+
"num_reward_heads": 0,
|
| 152 |
+
"process_image_mode": "dynamic_res",
|
| 153 |
+
"projector_hidden_act": "gelu",
|
| 154 |
+
"vision_feature_layer": -1,
|
| 155 |
+
"vision_feature_select_strategy": "full",
|
| 156 |
+
"projector_hidden_size": 6144,
|
| 157 |
+
"quantization_config": {
|
| 158 |
+
"quant_method": "mxfp8",
|
| 159 |
+
"activation_scheme": "dynamic",
|
| 160 |
+
"weight_block_size": [
|
| 161 |
+
1,
|
| 162 |
+
32
|
| 163 |
+
],
|
| 164 |
+
"ignored_layers": [
|
| 165 |
+
"language_model.model.dspark.confidence_head.proj",
|
| 166 |
+
"language_model.model.dspark.fc",
|
| 167 |
+
"language_model.model.dspark.markov_head.markov_w1",
|
| 168 |
+
"language_model.model.dspark.markov_head.markov_w2",
|
| 169 |
+
"language_model.model.layers.10.block_sparse_moe.gate",
|
| 170 |
+
"language_model.model.layers.11.block_sparse_moe.gate",
|
| 171 |
+
"language_model.model.layers.12.block_sparse_moe.gate",
|
| 172 |
+
"language_model.model.layers.13.block_sparse_moe.gate",
|
| 173 |
+
"language_model.model.layers.14.block_sparse_moe.gate",
|
| 174 |
+
"language_model.model.layers.15.block_sparse_moe.gate",
|
| 175 |
+
"language_model.model.layers.16.block_sparse_moe.gate",
|
| 176 |
+
"language_model.model.layers.17.block_sparse_moe.gate",
|
| 177 |
+
"language_model.model.layers.18.block_sparse_moe.gate",
|
| 178 |
+
"language_model.model.layers.19.block_sparse_moe.gate",
|
| 179 |
+
"language_model.model.layers.20.block_sparse_moe.gate",
|
| 180 |
+
"language_model.model.layers.21.block_sparse_moe.gate",
|
| 181 |
+
"language_model.model.layers.22.block_sparse_moe.gate",
|
| 182 |
+
"language_model.model.layers.23.block_sparse_moe.gate",
|
| 183 |
+
"language_model.model.layers.24.block_sparse_moe.gate",
|
| 184 |
+
"language_model.model.layers.25.block_sparse_moe.gate",
|
| 185 |
+
"language_model.model.layers.26.block_sparse_moe.gate",
|
| 186 |
+
"language_model.model.layers.27.block_sparse_moe.gate",
|
| 187 |
+
"language_model.model.layers.28.block_sparse_moe.gate",
|
| 188 |
+
"language_model.model.layers.29.block_sparse_moe.gate",
|
| 189 |
+
"language_model.model.layers.3.block_sparse_moe.gate",
|
| 190 |
+
"language_model.model.layers.30.block_sparse_moe.gate",
|
| 191 |
+
"language_model.model.layers.31.block_sparse_moe.gate",
|
| 192 |
+
"language_model.model.layers.32.block_sparse_moe.gate",
|
| 193 |
+
"language_model.model.layers.33.block_sparse_moe.gate",
|
| 194 |
+
"language_model.model.layers.34.block_sparse_moe.gate",
|
| 195 |
+
"language_model.model.layers.35.block_sparse_moe.gate",
|
| 196 |
+
"language_model.model.layers.36.block_sparse_moe.gate",
|
| 197 |
+
"language_model.model.layers.37.block_sparse_moe.gate",
|
| 198 |
+
"language_model.model.layers.38.block_sparse_moe.gate",
|
| 199 |
+
"language_model.model.layers.39.block_sparse_moe.gate",
|
| 200 |
+
"language_model.model.layers.4.block_sparse_moe.gate",
|
| 201 |
+
"language_model.model.layers.40.block_sparse_moe.gate",
|
| 202 |
+
"language_model.model.layers.41.block_sparse_moe.gate",
|
| 203 |
+
"language_model.model.layers.42.block_sparse_moe.gate",
|
| 204 |
+
"language_model.model.layers.43.block_sparse_moe.gate",
|
| 205 |
+
"language_model.model.layers.44.block_sparse_moe.gate",
|
| 206 |
+
"language_model.model.layers.45.block_sparse_moe.gate",
|
| 207 |
+
"language_model.model.layers.46.block_sparse_moe.gate",
|
| 208 |
+
"language_model.model.layers.47.block_sparse_moe.gate",
|
| 209 |
+
"language_model.model.layers.48.block_sparse_moe.gate",
|
| 210 |
+
"language_model.model.layers.49.block_sparse_moe.gate",
|
| 211 |
+
"language_model.model.layers.5.block_sparse_moe.gate",
|
| 212 |
+
"language_model.model.layers.50.block_sparse_moe.gate",
|
| 213 |
+
"language_model.model.layers.51.block_sparse_moe.gate",
|
| 214 |
+
"language_model.model.layers.52.block_sparse_moe.gate",
|
| 215 |
+
"language_model.model.layers.53.block_sparse_moe.gate",
|
| 216 |
+
"language_model.model.layers.54.block_sparse_moe.gate",
|
| 217 |
+
"language_model.model.layers.55.block_sparse_moe.gate",
|
| 218 |
+
"language_model.model.layers.56.block_sparse_moe.gate",
|
| 219 |
+
"language_model.model.layers.57.block_sparse_moe.gate",
|
| 220 |
+
"language_model.model.layers.58.block_sparse_moe.gate",
|
| 221 |
+
"language_model.model.layers.59.block_sparse_moe.gate",
|
| 222 |
+
"language_model.model.layers.6.block_sparse_moe.gate",
|
| 223 |
+
"language_model.model.layers.7.block_sparse_moe.gate",
|
| 224 |
+
"language_model.model.layers.8.block_sparse_moe.gate",
|
| 225 |
+
"language_model.model.layers.9.block_sparse_moe.gate",
|
| 226 |
+
"lm_head",
|
| 227 |
+
"model.embed_tokens",
|
| 228 |
+
"multi_modal_projector",
|
| 229 |
+
"patch_merge_mlp",
|
| 230 |
+
"vision_tower"
|
| 231 |
+
],
|
| 232 |
+
"group_size": 16,
|
| 233 |
+
"moe_quant_algo": "NVFP4",
|
| 234 |
+
"moe_quant_format": "nvfp4-pack-quantized"
|
| 235 |
+
},
|
| 236 |
+
"tie_word_embeddings": false,
|
| 237 |
+
"dspark_block_size": 7,
|
| 238 |
+
"dspark_markov_rank": 256,
|
| 239 |
+
"dspark_markov_head_type": "vanilla",
|
| 240 |
+
"dspark_aux_gemm_precision": "bf16",
|
| 241 |
+
"dspark_noise_token_id": 200058,
|
| 242 |
+
"dspark_use_dense_ffn": true,
|
| 243 |
+
"dspark_ffn_hidden_size": 12288,
|
| 244 |
+
"dspark_target_layer_ids": [
|
| 245 |
+
3,
|
| 246 |
+
17,
|
| 247 |
+
31,
|
| 248 |
+
45,
|
| 249 |
+
59
|
| 250 |
+
],
|
| 251 |
+
"dspark_config": {
|
| 252 |
+
"markov_rank": 256,
|
| 253 |
+
"markov_head_type": "vanilla",
|
| 254 |
+
"mask_token_id": 200058,
|
| 255 |
+
"hybrid_context_fusion": false,
|
| 256 |
+
"aux_gemm_precision": "bf16",
|
| 257 |
+
"use_dense_ffn": true,
|
| 258 |
+
"ffn_hidden_size": 12288
|
| 259 |
+
},
|
| 260 |
+
"dspark_hybrid_context_fusion": false,
|
| 261 |
+
"enable_confidence_head": true,
|
| 262 |
+
"confidence_head_with_markov": true,
|
| 263 |
+
"dflash_config": {
|
| 264 |
+
"num_hidden_layers": 5,
|
| 265 |
+
"num_target_layers": 60,
|
| 266 |
+
"context_source": "target_layers",
|
| 267 |
+
"block_size": 7,
|
| 268 |
+
"mask_token_id": 200058,
|
| 269 |
+
"target_layer_ids": [
|
| 270 |
+
3,
|
| 271 |
+
17,
|
| 272 |
+
31,
|
| 273 |
+
45,
|
| 274 |
+
59
|
| 275 |
+
],
|
| 276 |
+
"layer_types": [
|
| 277 |
+
"sliding_attention",
|
| 278 |
+
"sliding_attention",
|
| 279 |
+
"sliding_attention",
|
| 280 |
+
"sliding_attention",
|
| 281 |
+
"sliding_attention"
|
| 282 |
+
],
|
| 283 |
+
"sliding_window": 4096
|
| 284 |
+
},
|
| 285 |
+
"markov_rank": 256,
|
| 286 |
+
"markov_head_type": "vanilla",
|
| 287 |
+
"hidden_size": 6144,
|
| 288 |
+
"vocab_size": 200064,
|
| 289 |
+
"num_hidden_layers": 5,
|
| 290 |
+
"num_nextn_predict_layers": 0,
|
| 291 |
+
"use_mtp": false,
|
| 292 |
+
"num_mtp_modules": 0,
|
| 293 |
+
"context_len": 1048576,
|
| 294 |
+
"encoder_only": false,
|
| 295 |
+
"language_only": false,
|
| 296 |
+
"output_attentions": false
|
| 297 |
+
}
|
dspark/model-dspark-00001.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a27aed923fa17697997b30f44da0cd098460a60fee06498175e6d0c880308b69
|
| 3 |
+
size 2301867426
|
dspark/model.safetensors.index.json
ADDED
|
@@ -0,0 +1,104 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"metadata": {
|
| 3 |
+
"total_size": 2301852674
|
| 4 |
+
},
|
| 5 |
+
"weight_map": {
|
| 6 |
+
"language_model.model.dspark.confidence_head.proj.bias": "model-dspark-00001.safetensors",
|
| 7 |
+
"language_model.model.dspark.confidence_head.proj.weight": "model-dspark-00001.safetensors",
|
| 8 |
+
"language_model.model.dspark.fc.weight": "model-dspark-00001.safetensors",
|
| 9 |
+
"language_model.model.dspark.final_norm.weight": "model-dspark-00001.safetensors",
|
| 10 |
+
"language_model.model.dspark.hidden_norm.weight": "model-dspark-00001.safetensors",
|
| 11 |
+
"language_model.model.dspark.layers.0.decoder_layer.input_layernorm.weight": "model-dspark-00001.safetensors",
|
| 12 |
+
"language_model.model.dspark.layers.0.decoder_layer.mlp.down_proj.weight": "model-dspark-00001.safetensors",
|
| 13 |
+
"language_model.model.dspark.layers.0.decoder_layer.mlp.down_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 14 |
+
"language_model.model.dspark.layers.0.decoder_layer.mlp.gate_proj.weight": "model-dspark-00001.safetensors",
|
| 15 |
+
"language_model.model.dspark.layers.0.decoder_layer.mlp.gate_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 16 |
+
"language_model.model.dspark.layers.0.decoder_layer.mlp.up_proj.weight": "model-dspark-00001.safetensors",
|
| 17 |
+
"language_model.model.dspark.layers.0.decoder_layer.mlp.up_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 18 |
+
"language_model.model.dspark.layers.0.decoder_layer.post_attention_layernorm.weight": "model-dspark-00001.safetensors",
|
| 19 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.k_norm.weight": "model-dspark-00001.safetensors",
|
| 20 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.k_proj.weight": "model-dspark-00001.safetensors",
|
| 21 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.k_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 22 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.o_proj.weight": "model-dspark-00001.safetensors",
|
| 23 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.o_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 24 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.q_norm.weight": "model-dspark-00001.safetensors",
|
| 25 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.q_proj.weight": "model-dspark-00001.safetensors",
|
| 26 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.q_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 27 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.v_proj.weight": "model-dspark-00001.safetensors",
|
| 28 |
+
"language_model.model.dspark.layers.0.decoder_layer.self_attn.v_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 29 |
+
"language_model.model.dspark.layers.1.decoder_layer.input_layernorm.weight": "model-dspark-00001.safetensors",
|
| 30 |
+
"language_model.model.dspark.layers.1.decoder_layer.mlp.down_proj.weight": "model-dspark-00001.safetensors",
|
| 31 |
+
"language_model.model.dspark.layers.1.decoder_layer.mlp.down_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 32 |
+
"language_model.model.dspark.layers.1.decoder_layer.mlp.gate_proj.weight": "model-dspark-00001.safetensors",
|
| 33 |
+
"language_model.model.dspark.layers.1.decoder_layer.mlp.gate_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 34 |
+
"language_model.model.dspark.layers.1.decoder_layer.mlp.up_proj.weight": "model-dspark-00001.safetensors",
|
| 35 |
+
"language_model.model.dspark.layers.1.decoder_layer.mlp.up_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 36 |
+
"language_model.model.dspark.layers.1.decoder_layer.post_attention_layernorm.weight": "model-dspark-00001.safetensors",
|
| 37 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.k_norm.weight": "model-dspark-00001.safetensors",
|
| 38 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.k_proj.weight": "model-dspark-00001.safetensors",
|
| 39 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.k_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 40 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.o_proj.weight": "model-dspark-00001.safetensors",
|
| 41 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.o_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 42 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.q_norm.weight": "model-dspark-00001.safetensors",
|
| 43 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.q_proj.weight": "model-dspark-00001.safetensors",
|
| 44 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.q_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 45 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.v_proj.weight": "model-dspark-00001.safetensors",
|
| 46 |
+
"language_model.model.dspark.layers.1.decoder_layer.self_attn.v_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 47 |
+
"language_model.model.dspark.layers.2.decoder_layer.input_layernorm.weight": "model-dspark-00001.safetensors",
|
| 48 |
+
"language_model.model.dspark.layers.2.decoder_layer.mlp.down_proj.weight": "model-dspark-00001.safetensors",
|
| 49 |
+
"language_model.model.dspark.layers.2.decoder_layer.mlp.down_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 50 |
+
"language_model.model.dspark.layers.2.decoder_layer.mlp.gate_proj.weight": "model-dspark-00001.safetensors",
|
| 51 |
+
"language_model.model.dspark.layers.2.decoder_layer.mlp.gate_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 52 |
+
"language_model.model.dspark.layers.2.decoder_layer.mlp.up_proj.weight": "model-dspark-00001.safetensors",
|
| 53 |
+
"language_model.model.dspark.layers.2.decoder_layer.mlp.up_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 54 |
+
"language_model.model.dspark.layers.2.decoder_layer.post_attention_layernorm.weight": "model-dspark-00001.safetensors",
|
| 55 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.k_norm.weight": "model-dspark-00001.safetensors",
|
| 56 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.k_proj.weight": "model-dspark-00001.safetensors",
|
| 57 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.k_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 58 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.o_proj.weight": "model-dspark-00001.safetensors",
|
| 59 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.o_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 60 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.q_norm.weight": "model-dspark-00001.safetensors",
|
| 61 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.q_proj.weight": "model-dspark-00001.safetensors",
|
| 62 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.q_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 63 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.v_proj.weight": "model-dspark-00001.safetensors",
|
| 64 |
+
"language_model.model.dspark.layers.2.decoder_layer.self_attn.v_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 65 |
+
"language_model.model.dspark.layers.3.decoder_layer.input_layernorm.weight": "model-dspark-00001.safetensors",
|
| 66 |
+
"language_model.model.dspark.layers.3.decoder_layer.mlp.down_proj.weight": "model-dspark-00001.safetensors",
|
| 67 |
+
"language_model.model.dspark.layers.3.decoder_layer.mlp.down_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 68 |
+
"language_model.model.dspark.layers.3.decoder_layer.mlp.gate_proj.weight": "model-dspark-00001.safetensors",
|
| 69 |
+
"language_model.model.dspark.layers.3.decoder_layer.mlp.gate_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 70 |
+
"language_model.model.dspark.layers.3.decoder_layer.mlp.up_proj.weight": "model-dspark-00001.safetensors",
|
| 71 |
+
"language_model.model.dspark.layers.3.decoder_layer.mlp.up_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 72 |
+
"language_model.model.dspark.layers.3.decoder_layer.post_attention_layernorm.weight": "model-dspark-00001.safetensors",
|
| 73 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.k_norm.weight": "model-dspark-00001.safetensors",
|
| 74 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.k_proj.weight": "model-dspark-00001.safetensors",
|
| 75 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.k_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 76 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.o_proj.weight": "model-dspark-00001.safetensors",
|
| 77 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.o_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 78 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.q_norm.weight": "model-dspark-00001.safetensors",
|
| 79 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.q_proj.weight": "model-dspark-00001.safetensors",
|
| 80 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.q_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 81 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.v_proj.weight": "model-dspark-00001.safetensors",
|
| 82 |
+
"language_model.model.dspark.layers.3.decoder_layer.self_attn.v_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 83 |
+
"language_model.model.dspark.layers.4.decoder_layer.input_layernorm.weight": "model-dspark-00001.safetensors",
|
| 84 |
+
"language_model.model.dspark.layers.4.decoder_layer.mlp.down_proj.weight": "model-dspark-00001.safetensors",
|
| 85 |
+
"language_model.model.dspark.layers.4.decoder_layer.mlp.down_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 86 |
+
"language_model.model.dspark.layers.4.decoder_layer.mlp.gate_proj.weight": "model-dspark-00001.safetensors",
|
| 87 |
+
"language_model.model.dspark.layers.4.decoder_layer.mlp.gate_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 88 |
+
"language_model.model.dspark.layers.4.decoder_layer.mlp.up_proj.weight": "model-dspark-00001.safetensors",
|
| 89 |
+
"language_model.model.dspark.layers.4.decoder_layer.mlp.up_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 90 |
+
"language_model.model.dspark.layers.4.decoder_layer.post_attention_layernorm.weight": "model-dspark-00001.safetensors",
|
| 91 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.k_norm.weight": "model-dspark-00001.safetensors",
|
| 92 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.k_proj.weight": "model-dspark-00001.safetensors",
|
| 93 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.k_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 94 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.o_proj.weight": "model-dspark-00001.safetensors",
|
| 95 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.o_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 96 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.q_norm.weight": "model-dspark-00001.safetensors",
|
| 97 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.q_proj.weight": "model-dspark-00001.safetensors",
|
| 98 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.q_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 99 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.v_proj.weight": "model-dspark-00001.safetensors",
|
| 100 |
+
"language_model.model.dspark.layers.4.decoder_layer.self_attn.v_proj.weight_scale_inv": "model-dspark-00001.safetensors",
|
| 101 |
+
"language_model.model.dspark.markov_head.markov_w1.weight": "model-dspark-00001.safetensors",
|
| 102 |
+
"language_model.model.dspark.markov_head.markov_w2.weight": "model-dspark-00001.safetensors"
|
| 103 |
+
}
|
| 104 |
+
}
|
figures/logo.svg
ADDED
|
generation_config.json
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token_id": 200019,
|
| 3 |
+
"do_sample": true,
|
| 4 |
+
"eos_token_id": 200020,
|
| 5 |
+
"temperature": 1.0,
|
| 6 |
+
"top_p": 0.95,
|
| 7 |
+
"transformers_version": "4.46.1"
|
| 8 |
+
}
|
image_processor.py
ADDED
|
@@ -0,0 +1,223 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Copyright 2023-2024 SGLang Team
|
| 2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
| 3 |
+
"""
|
| 4 |
+
MiniMax VL family HuggingFace-compatible Processor, ImageProcessor, VideoProcessor.
|
| 5 |
+
"""
|
| 6 |
+
import math
|
| 7 |
+
from typing import List, Tuple
|
| 8 |
+
|
| 9 |
+
import torch
|
| 10 |
+
from torchvision.transforms import InterpolationMode
|
| 11 |
+
from transformers import BatchFeature
|
| 12 |
+
from transformers.image_processing_utils_fast import (
|
| 13 |
+
BaseImageProcessorFast,
|
| 14 |
+
group_images_by_shape,
|
| 15 |
+
reorder_images,
|
| 16 |
+
)
|
| 17 |
+
from transformers.image_utils import PILImageResampling, SizeDict
|
| 18 |
+
from transformers.processing_utils import (
|
| 19 |
+
ImagesKwargs,
|
| 20 |
+
Unpack,
|
| 21 |
+
)
|
| 22 |
+
from transformers.utils import TensorType
|
| 23 |
+
|
| 24 |
+
MAX_RATIO = 200
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def round_by_factor(number: int, factor: int) -> int:
|
| 28 |
+
return round(number / factor) * factor
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def ceil_by_factor(number: int, factor: int) -> int:
|
| 32 |
+
return math.ceil(number / factor) * factor
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def floor_by_factor(number: int, factor: int) -> int:
|
| 36 |
+
return math.floor(number / factor) * factor
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def smart_resize(
|
| 40 |
+
height: int,
|
| 41 |
+
width: int,
|
| 42 |
+
factor: int = 28,
|
| 43 |
+
min_pixels: int = 4 * 28 * 28,
|
| 44 |
+
max_pixels: int = 451584,
|
| 45 |
+
) -> tuple[int, int]:
|
| 46 |
+
if max(height, width) / min(height, width) > MAX_RATIO:
|
| 47 |
+
raise ValueError(
|
| 48 |
+
f"absolute aspect ratio must be smaller than {MAX_RATIO}, "
|
| 49 |
+
f"got {max(height, width) / min(height, width)}"
|
| 50 |
+
)
|
| 51 |
+
h_bar = max(factor, round_by_factor(height, factor))
|
| 52 |
+
w_bar = max(factor, round_by_factor(width, factor))
|
| 53 |
+
if h_bar * w_bar > max_pixels:
|
| 54 |
+
beta = math.sqrt((height * width) / max_pixels)
|
| 55 |
+
h_bar = floor_by_factor(height / beta, factor)
|
| 56 |
+
w_bar = floor_by_factor(width / beta, factor)
|
| 57 |
+
elif h_bar * w_bar < min_pixels:
|
| 58 |
+
beta = math.sqrt(min_pixels / (height * width))
|
| 59 |
+
h_bar = ceil_by_factor(height * beta, factor)
|
| 60 |
+
w_bar = ceil_by_factor(width * beta, factor)
|
| 61 |
+
return h_bar, w_bar
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
# ==============================================================================
|
| 65 |
+
# MiniMax M3 VL Image Processor Fast (Fast Mode - Torch based)
|
| 66 |
+
# ==============================================================================
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
class MiniMaxM3VLImageProcessorKwargs(ImagesKwargs, total=False):
|
| 70 |
+
patch_size: int
|
| 71 |
+
temporal_patch_size: int
|
| 72 |
+
merge_size: int
|
| 73 |
+
max_pixels: int
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
class MiniMaxM3VLImageProcessor(BaseImageProcessorFast):
|
| 77 |
+
do_resize = True
|
| 78 |
+
resample = PILImageResampling.BICUBIC
|
| 79 |
+
size = {"height": 672, "width": 672} # required by base class validation, not used as resize bound
|
| 80 |
+
default_to_square = False
|
| 81 |
+
do_rescale = True
|
| 82 |
+
rescale_factor = 1 / 255
|
| 83 |
+
do_normalize = True
|
| 84 |
+
image_mean = [0.48145466, 0.4578275, 0.40821073]
|
| 85 |
+
image_std = [0.26862954, 0.26130258, 0.27577711]
|
| 86 |
+
do_convert_rgb = True
|
| 87 |
+
patch_size = 14
|
| 88 |
+
temporal_patch_size = 2
|
| 89 |
+
merge_size = 2
|
| 90 |
+
max_pixels = 451584 # 672*672
|
| 91 |
+
valid_kwargs = MiniMaxM3VLImageProcessorKwargs
|
| 92 |
+
model_input_names = ["pixel_values", "image_grid_thw"]
|
| 93 |
+
|
| 94 |
+
def __init__(self, **kwargs: Unpack[MiniMaxM3VLImageProcessorKwargs]):
|
| 95 |
+
super().__init__(**kwargs)
|
| 96 |
+
|
| 97 |
+
def preprocess(
|
| 98 |
+
self, images, **kwargs: Unpack[MiniMaxM3VLImageProcessorKwargs]
|
| 99 |
+
) -> BatchFeature:
|
| 100 |
+
return super().preprocess(images, **kwargs)
|
| 101 |
+
|
| 102 |
+
def _preprocess(
|
| 103 |
+
self,
|
| 104 |
+
images: List[torch.Tensor],
|
| 105 |
+
do_resize: bool,
|
| 106 |
+
size: SizeDict,
|
| 107 |
+
resample: PILImageResampling | InterpolationMode | int | None,
|
| 108 |
+
do_rescale: bool,
|
| 109 |
+
rescale_factor: float,
|
| 110 |
+
do_normalize: bool,
|
| 111 |
+
image_mean: float | List[float] | None,
|
| 112 |
+
image_std: float | List[float] | None,
|
| 113 |
+
patch_size: int,
|
| 114 |
+
temporal_patch_size: int,
|
| 115 |
+
merge_size: int,
|
| 116 |
+
max_pixels: int,
|
| 117 |
+
disable_grouping: bool | None,
|
| 118 |
+
return_tensors: str | TensorType | None,
|
| 119 |
+
**kwargs,
|
| 120 |
+
) -> BatchFeature:
|
| 121 |
+
grouped_images, grouped_images_index = group_images_by_shape(
|
| 122 |
+
images, disable_grouping=disable_grouping
|
| 123 |
+
)
|
| 124 |
+
resized_images_grouped = {}
|
| 125 |
+
factor = patch_size * merge_size
|
| 126 |
+
for shape, stacked_images in grouped_images.items():
|
| 127 |
+
height, width = stacked_images.shape[-2:]
|
| 128 |
+
if do_resize:
|
| 129 |
+
resized_height, resized_width = smart_resize(
|
| 130 |
+
height, width, factor=factor,
|
| 131 |
+
max_pixels=max_pixels,
|
| 132 |
+
)
|
| 133 |
+
stacked_images = self.resize(
|
| 134 |
+
stacked_images,
|
| 135 |
+
size=SizeDict(height=resized_height, width=resized_width),
|
| 136 |
+
resample=resample,
|
| 137 |
+
)
|
| 138 |
+
resized_images_grouped[shape] = stacked_images
|
| 139 |
+
|
| 140 |
+
resized_images = reorder_images(resized_images_grouped, grouped_images_index)
|
| 141 |
+
|
| 142 |
+
grouped_images, grouped_images_index = group_images_by_shape(
|
| 143 |
+
resized_images, disable_grouping=disable_grouping
|
| 144 |
+
)
|
| 145 |
+
processed_images_grouped = {}
|
| 146 |
+
processed_grids = {}
|
| 147 |
+
|
| 148 |
+
for shape, stacked_images in grouped_images.items():
|
| 149 |
+
resized_height, resized_width = stacked_images.shape[-2:]
|
| 150 |
+
|
| 151 |
+
patches = self.rescale_and_normalize(
|
| 152 |
+
stacked_images,
|
| 153 |
+
do_rescale,
|
| 154 |
+
rescale_factor,
|
| 155 |
+
do_normalize,
|
| 156 |
+
image_mean,
|
| 157 |
+
image_std,
|
| 158 |
+
)
|
| 159 |
+
if patches.ndim == 4:
|
| 160 |
+
patches = patches.unsqueeze(1)
|
| 161 |
+
|
| 162 |
+
if patches.shape[1] % temporal_patch_size != 0:
|
| 163 |
+
repeats = patches[:, -1:].repeat(
|
| 164 |
+
1,
|
| 165 |
+
temporal_patch_size - (patches.shape[1] % temporal_patch_size),
|
| 166 |
+
1,
|
| 167 |
+
1,
|
| 168 |
+
1,
|
| 169 |
+
)
|
| 170 |
+
patches = torch.cat([patches, repeats], dim=1)
|
| 171 |
+
|
| 172 |
+
batch_size, grid_t, channel = patches.shape[:3]
|
| 173 |
+
grid_t = grid_t // temporal_patch_size
|
| 174 |
+
grid_h, grid_w = resized_height // patch_size, resized_width // patch_size
|
| 175 |
+
|
| 176 |
+
patches = patches.view(
|
| 177 |
+
batch_size,
|
| 178 |
+
grid_t,
|
| 179 |
+
temporal_patch_size,
|
| 180 |
+
channel,
|
| 181 |
+
grid_h // merge_size,
|
| 182 |
+
merge_size,
|
| 183 |
+
patch_size,
|
| 184 |
+
grid_w // merge_size,
|
| 185 |
+
merge_size,
|
| 186 |
+
patch_size,
|
| 187 |
+
)
|
| 188 |
+
patches = patches.permute(0, 1, 4, 7, 5, 8, 3, 2, 6, 9)
|
| 189 |
+
|
| 190 |
+
flatten_patches = patches.reshape(
|
| 191 |
+
batch_size,
|
| 192 |
+
grid_t * grid_h * grid_w,
|
| 193 |
+
channel * temporal_patch_size * patch_size * patch_size,
|
| 194 |
+
)
|
| 195 |
+
|
| 196 |
+
processed_images_grouped[shape] = flatten_patches
|
| 197 |
+
processed_grids[shape] = [[grid_t, grid_h, grid_w]] * batch_size
|
| 198 |
+
|
| 199 |
+
processed_images = reorder_images(
|
| 200 |
+
processed_images_grouped, grouped_images_index
|
| 201 |
+
)
|
| 202 |
+
processed_grids = reorder_images(processed_grids, grouped_images_index)
|
| 203 |
+
|
| 204 |
+
pixel_values = torch.cat(processed_images, dim=0)
|
| 205 |
+
image_grid_thw = torch.tensor(processed_grids, dtype=torch.long)
|
| 206 |
+
|
| 207 |
+
return BatchFeature(
|
| 208 |
+
data={"pixel_values": pixel_values, "image_grid_thw": image_grid_thw},
|
| 209 |
+
tensor_type=return_tensors,
|
| 210 |
+
)
|
| 211 |
+
|
| 212 |
+
def get_number_of_image_patches(self, height: int, width: int, images_kwargs=None):
|
| 213 |
+
images_kwargs = images_kwargs or {}
|
| 214 |
+
patch_size = images_kwargs.get("patch_size", self.patch_size)
|
| 215 |
+
merge_size = images_kwargs.get("merge_size", self.merge_size)
|
| 216 |
+
max_pixels = images_kwargs.get("max_pixels", self.max_pixels)
|
| 217 |
+
|
| 218 |
+
resized_height, resized_width = smart_resize(
|
| 219 |
+
height, width, factor=patch_size * merge_size,
|
| 220 |
+
max_pixels=max_pixels,
|
| 221 |
+
)
|
| 222 |
+
grid_h, grid_w = resized_height // patch_size, resized_width // patch_size
|
| 223 |
+
return grid_h * grid_w
|
merges.txt
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
model-00001-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8b7c561664839e3b2d800c1cd7afe1beb529efa70ce0c2da82d7bb912bf2249b
|
| 3 |
+
size 10213649632
|
model-00002-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8c9094e8c305ab99c4c119fb8e5c959d2bebf562a4e516a908c13e95ffa1e928
|
| 3 |
+
size 4253008280
|
model-00003-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b792a81d0dd657d17a850aeaf8b84cf91f9fc3f0ec63e13395335c1752f1d8a6
|
| 3 |
+
size 4253008280
|
model-00004-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ad32f2c455aa2ab8de4b576ee508cc63ede81490330a5cb251bef533d3fb8299
|
| 3 |
+
size 8506022152
|
model-00005-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cff05d9e803e2a48ad7bcc2ce3ac1ebeb8c96d4a709cb8786cd5e0572fe5bcd2
|
| 3 |
+
size 8506022152
|
model-00006-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:12b767c0ad91bd685eb4d51586490657585629dea1656e9141620c7c1f0f185f
|
| 3 |
+
size 8506022152
|
model-00007-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bea23e0a3ba91f8718075d6f8fb2e2a84c418490e69fde07767610b1ddccc31c
|
| 3 |
+
size 8506022152
|
model-00008-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d270b43d5d85389a8f8e979a0769faa99d9c61263eb7bde2d234ff9f433c36e1
|
| 3 |
+
size 8506022152
|
model-00009-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f2b0c83a62826c3427d94b1914805193ea6eb044797946db1569a9ef13b64356
|
| 3 |
+
size 8506023712
|
model-00010-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5c524d0a7109402dd0f917c5c5c258058a6d9afc68ab506e7c0112bda4cb8baf
|
| 3 |
+
size 8506023712
|
model-00011-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9c53f2b363c8cb8f245c431885aa3d759f8a1ff2c4519dcf4bbc649bfad0234a
|
| 3 |
+
size 8506023712
|
model-00012-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:567b4ebecebefcaffc4350502d924352a330004cee77d43681c2972613436dab
|
| 3 |
+
size 8506023712
|
model-00013-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aa146faf2cd153211379d8a933df3abea68db75285d1c8f26e6f6ff02ae5f512
|
| 3 |
+
size 4721760520
|
model-00014-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:93d8d55a0761a3b6a9a4ee74eb795643673477d63f85c905d5b51af6786e1cd4
|
| 3 |
+
size 4253009184
|
model-00015-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:17e624b2e700020a81d8257c27e2cf39b690b3192059adcd9c87226d44e93f83
|
| 3 |
+
size 4253009184
|
model-00016-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ae52c17c04233240fb8ef5026e4318c4e3778c2d7726dd46e406659d7b8100f
|
| 3 |
+
size 4253009184
|
model-00017-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:97843980ad8b9a86ed482c034f8af2370f6b98e163d00fda4266b9cdb658290f
|
| 3 |
+
size 4253009184
|
model-00018-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5400d73757f2f24af63fc209c47636eb85648e25149c781e08c5341eee94e60f
|
| 3 |
+
size 4253009184
|
model-00019-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5ff419f097b4a03d5857d4be51159b7b5bd1462bb3a6e30dd7bc343c02a8b249
|
| 3 |
+
size 4253009184
|
model-00020-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4ef5a921908c37d450e6ca406d6a2fcfc22612f73a4eddba3910dbef1ebcc8c3
|
| 3 |
+
size 4253009184
|
model-00021-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:18bbad88842ac8a56b9ed44184779f1d495ef931d6e415ab397918270d85c45b
|
| 3 |
+
size 4253009184
|
model-00022-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1d2857772efc0405c58bfb86981101e7c6ca3f05939bf896f7941df14fce87b8
|
| 3 |
+
size 4253009184
|
model-00023-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a362968d7d7ffaff35dab4e5f69a268fc0856de98a0a0cac62661e63d9e5f345
|
| 3 |
+
size 4253009184
|
model-00024-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cb1369f1311d87bb86c6d129f86486fe318bb290615958edcc762da3eca85aa4
|
| 3 |
+
size 4253009184
|
model-00025-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4026f6d42bd44f56fb4780f67bb7e2339a1802cc67f5a9acd1b0b0ae116d5d5c
|
| 3 |
+
size 4253009184
|
model-00026-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2111d251931e13746596f918af85cec2169ea20733fe5b16d165c7733424a9c5
|
| 3 |
+
size 4253009184
|
model-00027-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1c6fecd1edd7fb9053420ac00e33668635fe7da5bc3e38753cbdaedc934bbc8f
|
| 3 |
+
size 4253009184
|
model-00028-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:38c4c2d584a8761bf15c76ddc42004113d0066d09db3c2e48648b5b80e2eb9d4
|
| 3 |
+
size 4253009184
|
model-00029-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5809047f21a9d6b948ff6973f03ef4e07907db7ecf3a5e963fa13ddf6f73de46
|
| 3 |
+
size 4253009184
|
model-00030-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4ab223b3f7c01280fc841cdd7580b0036cf414f78d323fa722c19673b41c546e
|
| 3 |
+
size 4253009184
|
model-00031-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0f76e599478e0e5ddd2ffa479adedc304efa43952ad587e0d1a4e39defecbac5
|
| 3 |
+
size 4253009184
|
model-00032-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fdb13c791e04c3c3d15b6ad9c9e611e5065b49c566493547d9f6304a1663a38a
|
| 3 |
+
size 4253009184
|
model-00033-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f3ac23810b4e6859297937b2640e29185cd26b47985e4feae09e14ca321fa600
|
| 3 |
+
size 4253009184
|
model-00034-of-00048.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:246139b00e2e64d66189b380fbba4a769b43bb02a88aed60698800276a7be039
|
| 3 |
+
size 4253009184
|