Instructions to use litert-community/MiniCPM5-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MiniCPM5-2B with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Plain wording in the litertlm_manifest.json notes
#8
by mlboydaisuke - opened
- litertlm_manifest.json +6 -4
litertlm_manifest.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
{
|
| 2 |
"manifest_schema": "0.1.3",
|
| 3 |
"repo": "litert-community/MiniCPM5-2B",
|
| 4 |
-
"generated": "2026-
|
| 5 |
"generator": "make_manifest.py",
|
| 6 |
"model": {
|
| 7 |
"display_name": "MiniCPM5-2B",
|
|
@@ -76,7 +76,7 @@
|
|
| 76 |
{
|
| 77 |
"platform": "ios",
|
| 78 |
"backend": "gpu",
|
| 79 |
-
"reason": "iPhone 17 Pro, 8-question on-device gate at a 2048-token budget: Metal gpu 7/8 (init 5.7 s) and cpu 7/8 (init 2.2 s), both
|
| 80 |
}
|
| 81 |
],
|
| 82 |
"requirements": {
|
|
@@ -193,7 +193,8 @@
|
|
| 193 |
],
|
| 194 |
"known_issues": [
|
| 195 |
"Thinking mode: reasoning chains are ~4x longer than int8/bf16 on GSM8K-class questions and frequently hit a 3584-token budget without emitting the answer (CPU reference 0/10 closed; the Metal GPU's default fp16 activations happen to close 22/30). Quantization damage of int4 on this 42-layer model, not a runtime bug; block-128 int4 was tried and is worse (GSM8K 79 vs 87)"
|
| 196 |
-
]
|
|
|
|
| 197 |
},
|
| 198 |
{
|
| 199 |
"file": "MiniCPM5-2B_int8.litertlm",
|
|
@@ -317,7 +318,8 @@
|
|
| 317 |
],
|
| 318 |
"known_issues": [
|
| 319 |
"GSM8K (thinking off, 2048-token budget) question 8: on the GPU the derivation runs to the budget mid-calculation and the extracted number is wrong; the CPU run on the same file answers it in 2,204 characters"
|
| 320 |
-
]
|
|
|
|
| 321 |
}
|
| 322 |
]
|
| 323 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"manifest_schema": "0.1.3",
|
| 3 |
"repo": "litert-community/MiniCPM5-2B",
|
| 4 |
+
"generated": "2026-11-14",
|
| 5 |
"generator": "make_manifest.py",
|
| 6 |
"model": {
|
| 7 |
"display_name": "MiniCPM5-2B",
|
|
|
|
| 76 |
{
|
| 77 |
"platform": "ios",
|
| 78 |
"backend": "gpu",
|
| 79 |
+
"reason": "iPhone 17 Pro, 8-question on-device gate at a 2048-token budget: Metal gpu 7/8 (init 5.7 s) and cpu 7/8 (init 2.2 s), both completed; the one miss on each leg is the rhyme line of the composite prompt, which the bf16 model also misses with thinking off. No speed rows were taken on the phone"
|
| 80 |
}
|
| 81 |
],
|
| 82 |
"requirements": {
|
|
|
|
| 193 |
],
|
| 194 |
"known_issues": [
|
| 195 |
"Thinking mode: reasoning chains are ~4x longer than int8/bf16 on GSM8K-class questions and frequently hit a 3584-token budget without emitting the answer (CPU reference 0/10 closed; the Metal GPU's default fp16 activations happen to close 22/30). Quantization damage of int4 on this 42-layer model, not a runtime bug; block-128 int4 was tried and is worse (GSM8K 79 vs 87)"
|
| 196 |
+
],
|
| 197 |
+
"context_length": 4096
|
| 198 |
},
|
| 199 |
{
|
| 200 |
"file": "MiniCPM5-2B_int8.litertlm",
|
|
|
|
| 318 |
],
|
| 319 |
"known_issues": [
|
| 320 |
"GSM8K (thinking off, 2048-token budget) question 8: on the GPU the derivation runs to the budget mid-calculation and the extracted number is wrong; the CPU run on the same file answers it in 2,204 characters"
|
| 321 |
+
],
|
| 322 |
+
"context_length": 4096
|
| 323 |
}
|
| 324 |
]
|
| 325 |
}
|