Add link to paper
#2
by
nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,17 +1,18 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- zh
|
| 5 |
- en
|
| 6 |
-
pipeline_tag: text-generation
|
| 7 |
library_name: transformers
|
|
|
|
|
|
|
| 8 |
---
|
|
|
|
| 9 |
<div align="center">
|
| 10 |
<img src="https://github.com/OpenBMB/MiniCPM/blob/main/assets/minicpm_logo.png?raw=true" width="500em" ></img>
|
| 11 |
</div>
|
| 12 |
|
| 13 |
<p align="center">
|
| 14 |
-
<a href="https://github.com/OpenBMB/MiniCPM/" target="_blank">GitHub Repo</a> |
|
| 15 |
<a href="https://github.com/OpenBMB/MiniCPM/tree/main/report/MiniCPM_4_Technical_Report.pdf" target="_blank">Technical Report</a>
|
| 16 |
</p>
|
| 17 |
<p align="center">
|
|
@@ -207,78 +208,4 @@ You can apply the LongRoPE factor modification by modifying the model files. Spe
|
|
| 207 |
"rope_scaling": {
|
| 208 |
"rope_type": "longrope",
|
| 209 |
"long_factor": [0.9977997200264581, 1.014658295992452, 1.0349680404997148, 1.059429246056193, 1.0888815016813513, 1.1243301355211495, 1.166977103606075, 1.2182568066927284, 1.2798772354275727, 1.3538666751582975, 1.4426259039919596, 1.5489853358570191, 1.6762658237220625, 1.8283407612492941, 2.0096956085876183, 2.225478927469756, 2.481536379650452, 2.784415934557119, 3.1413289096347365, 3.560047844772632, 4.048719380066383, 4.752651957515948, 5.590913044973868, 6.584005926629993, 7.7532214876576155, 9.119754865903639, 10.704443927019176, 12.524994176518703, 14.59739595363613, 16.93214476166354, 19.53823297353041, 22.417131025031697, 25.568260840911098, 28.991144156566317, 32.68408069090375, 36.65174474170465, 40.90396065611201, 45.4664008671033, 50.37147343433591, 55.6804490772103, 61.470816952306556, 67.8622707390618, 75.00516023410414, 83.11898235973767, 92.50044360202462, 103.57086856690864, 116.9492274587385, 118.16074567836519, 119.18497548708795, 120.04810876261652, 120.77352815196981, 121.38182790207875, 121.89094985353891, 122.31638758099915, 122.6714244963338, 122.9673822552567, 123.21386397019609, 123.41898278254268, 123.58957065488238, 123.73136519024158, 123.84917421274221, 123.94701903496814, 124.02825801299717, 124.09569231686116],
|
| 210 |
-
"short_factor": [0.9977997200264581, 1.014658295992452, 1.0349680404997148, 1.059429246056193, 1.0888815016813513, 1.1243301355211495, 1.166977103606075, 1.2182568066927284, 1.2798772354275727, 1.3538666751582975, 1.4426259039919596, 1.5489853358570191, 1.6762658237220625, 1.8283407612492941, 2.0096956085876183, 2.225478927469756, 2.481536379650452, 2.784415934557119, 3.1413289096347365, 3.560047844772632, 4.048719380066383, 4.752651957515948, 5.590913044973868, 6.584005926629993, 7.7532214876576155, 9.119754865903639, 10.704443927019176, 12.524994176518703, 14.59739595363613, 16.93214476166354, 19.53823297353041, 22.417131025031697, 25.568260840911098, 28.991144156566317, 32.68408069090375, 36.65174474170465, 40.90396065611201, 45.4664008671033, 50.37147343433591, 55.6804490772103, 61.470816952306556, 67.8622707390618, 75.00516023410414, 83.11898235973767, 92.50044360202462, 103.57086856690864, 116.9492274587385, 118.16074567836519, 119.18497548708795, 120.04810876261652, 120.77352815196981, 121.38182790207875, 121.89094985353891, 122.
|
| 211 |
-
"original_max_position_embeddings": 32768
|
| 212 |
-
}
|
| 213 |
-
}
|
| 214 |
-
```
|
| 215 |
-
|
| 216 |
-
### Inference with [SGLang](https://github.com/sgl-project/sglang)
|
| 217 |
-
|
| 218 |
-
For now, you need to install our forked version of SGLang.
|
| 219 |
-
```bash
|
| 220 |
-
git clone -b openbmb https://github.com/OpenBMB/sglang.git
|
| 221 |
-
cd sglang
|
| 222 |
-
|
| 223 |
-
pip install --upgrade pip
|
| 224 |
-
pip install -e "python[all]"
|
| 225 |
-
```
|
| 226 |
-
|
| 227 |
-
You can start the inference server by running the following command:
|
| 228 |
-
```bash
|
| 229 |
-
python -m sglang.launch_server --model openbmb/MiniCPM4-8B --trust-remote-code --port 30000 --chat-template chatml
|
| 230 |
-
```
|
| 231 |
-
|
| 232 |
-
Then you can use the chat interface by running the following command:
|
| 233 |
-
```python
|
| 234 |
-
import openai
|
| 235 |
-
|
| 236 |
-
client = openai.Client(base_url=f"http://localhost:30000/v1", api_key="None")
|
| 237 |
-
|
| 238 |
-
response = client.chat.completions.create(
|
| 239 |
-
model="openbmb/MiniCPM4-8B",
|
| 240 |
-
messages=[
|
| 241 |
-
{"role": "user", "content": "Write an article about Artificial Intelligence."},
|
| 242 |
-
],
|
| 243 |
-
temperature=0.7,
|
| 244 |
-
max_tokens=1024,
|
| 245 |
-
)
|
| 246 |
-
|
| 247 |
-
print(response.choices[0].message.content)
|
| 248 |
-
```
|
| 249 |
-
|
| 250 |
-
|
| 251 |
-
## Evaluation Results
|
| 252 |
-
On two typical end-side chips, Jetson AGX Orin and RTX 4090, MiniCPM4 demonstrates significantly faster processing speed compared to similar-size models in long text processing tasks. As text length increases, MiniCPM4's efficiency advantage becomes more pronounced. On the Jetson AGX Orin platform, compared to Qwen3-8B, MiniCPM4 achieves approximately 7x decoding speed improvement.
|
| 253 |
-
|
| 254 |
-

|
| 255 |
-
|
| 256 |
-
#### Comprehensive Evaluation
|
| 257 |
-
MiniCPM4 launches end-side versions with 8B and 0.5B parameter scales, both achieving best-in-class performance in their respective categories.
|
| 258 |
-
|
| 259 |
-

|
| 260 |
-
|
| 261 |
-
#### Long Text Evaluation
|
| 262 |
-
MiniCPM4 is pre-trained on 32K long texts and achieves length extension through YaRN technology. In the 128K long text needle-in-a-haystack task, MiniCPM4 demonstrates outstanding performance.
|
| 263 |
-
|
| 264 |
-

|
| 265 |
-
|
| 266 |
-
## Statement
|
| 267 |
-
- As a language model, MiniCPM generates content by learning from a vast amount of text.
|
| 268 |
-
- However, it does not possess the ability to comprehend or express personal opinions or value judgments.
|
| 269 |
-
- Any content generated by MiniCPM does not represent the viewpoints or positions of the model developers.
|
| 270 |
-
- Therefore, when using content generated by MiniCPM, users should take full responsibility for evaluating and verifying it on their own.
|
| 271 |
-
|
| 272 |
-
## LICENSE
|
| 273 |
-
- This repository and MiniCPM models are released under the [Apache-2.0](https://github.com/OpenBMB/MiniCPM/blob/main/LICENSE) License.
|
| 274 |
-
|
| 275 |
-
## Citation
|
| 276 |
-
- Please cite our [paper](https://github.com/OpenBMB/MiniCPM/tree/main/report/MiniCPM_4_Technical_Report.pdf) if you find our work valuable.
|
| 277 |
-
|
| 278 |
-
```bibtex
|
| 279 |
-
@article{minicpm4,
|
| 280 |
-
title={{MiniCPM4}: Ultra-Efficient LLMs on End Devices},
|
| 281 |
-
author={MiniCPM Team},
|
| 282 |
-
year={2025}
|
| 283 |
-
}
|
| 284 |
-
```
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
language:
|
| 3 |
- zh
|
| 4 |
- en
|
|
|
|
| 5 |
library_name: transformers
|
| 6 |
+
license: apache-2.0
|
| 7 |
+
pipeline_tag: text-generation
|
| 8 |
---
|
| 9 |
+
|
| 10 |
<div align="center">
|
| 11 |
<img src="https://github.com/OpenBMB/MiniCPM/blob/main/assets/minicpm_logo.png?raw=true" width="500em" ></img>
|
| 12 |
</div>
|
| 13 |
|
| 14 |
<p align="center">
|
| 15 |
+
<a href="https://github.com/OpenBMB/MiniCPM/\" target="_blank">GitHub Repo</a> |
|
| 16 |
<a href="https://github.com/OpenBMB/MiniCPM/tree/main/report/MiniCPM_4_Technical_Report.pdf" target="_blank">Technical Report</a>
|
| 17 |
</p>
|
| 18 |
<p align="center">
|
|
|
|
| 208 |
"rope_scaling": {
|
| 209 |
"rope_type": "longrope",
|
| 210 |
"long_factor": [0.9977997200264581, 1.014658295992452, 1.0349680404997148, 1.059429246056193, 1.0888815016813513, 1.1243301355211495, 1.166977103606075, 1.2182568066927284, 1.2798772354275727, 1.3538666751582975, 1.4426259039919596, 1.5489853358570191, 1.6762658237220625, 1.8283407612492941, 2.0096956085876183, 2.225478927469756, 2.481536379650452, 2.784415934557119, 3.1413289096347365, 3.560047844772632, 4.048719380066383, 4.752651957515948, 5.590913044973868, 6.584005926629993, 7.7532214876576155, 9.119754865903639, 10.704443927019176, 12.524994176518703, 14.59739595363613, 16.93214476166354, 19.53823297353041, 22.417131025031697, 25.568260840911098, 28.991144156566317, 32.68408069090375, 36.65174474170465, 40.90396065611201, 45.4664008671033, 50.37147343433591, 55.6804490772103, 61.470816952306556, 67.8622707390618, 75.00516023410414, 83.11898235973767, 92.50044360202462, 103.57086856690864, 116.9492274587385, 118.16074567836519, 119.18497548708795, 120.04810876261652, 120.77352815196981, 121.38182790207875, 121.89094985353891, 122.31638758099915, 122.6714244963338, 122.9673822552567, 123.21386397019609, 123.41898278254268, 123.58957065488238, 123.73136519024158, 123.84917421274221, 123.94701903496814, 124.02825801299717, 124.09569231686116],
|
| 211 |
+
"short_factor": [0.9977997200264581, 1.014658295992452, 1.0349680404997148, 1.059429246056193, 1.0888815016813513, 1.1243301355211495, 1.166977103606075, 1.2182568066927284, 1.2798772354275727, 1.3538666751582975, 1.4426259039919596, 1.5489853358570191, 1.6762658237220625, 1.8283407612492941, 2.0096956085876183, 2.225478927469756, 2.481536379650452, 2.784415934557119, 3.1413289096347365, 3.560047844772632, 4.048719380066383, 4.752651957515948, 5.590913044973868, 6.584005926629993, 7.7532214876576155, 9.119754865903639, 10.704443927019176, 12.524994176518703, 14.59739595363613, 16.93214476166354, 19.53823297353041, 22.417131025031697, 25.568260840911098, 28.991144156566317, 32.68408069090375, 36.65174474170465, 40.90396065611201, 45.4664008671033, 50.37147343433591, 55.6804490772103, 61.470816952306556, 67.8622707390618, 75.00516023410414, 83.11898235973767, 92.50044360202462, 103.57086856690864, 116.9492274587385, 118.16074567836519, 119.18497548708795, 120.04810876261652, 120.77352815196981, 121.38182790207875, 121.89094985353891, 122.316387580
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|