MiniCPM5-1B (LiteRT-LM)

This repository hosts the LiteRT-LM (LiteRT formerly known as TensorFlow Lite) version of MiniCPM5-1B, optimized for fully on-device inference on mobile and edge hardware.

---

Available Models

• minicpm_dynamic_wi8_afp32_gpu_opt.litertlm: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32), heavily optimized for GPU execution.
• minicpm_wi4b32_wi8_afp32.litertlm: This model features mixed INT4-block32(linear)/INT8(embed and lmhead) quantization (wi4b32_wi8) with FP32 activations (afp32).

Quantization Benchmark

The table below compares the accuracy of the two LiteRT-LM quantized models on the same evaluation benchmark. Scores are reported as percentages.

Category	W8 (minicpm_dynamic_wi8_afp32_gpu_opt.litertlm)	W4 (minicpm_wi4b32_wi8_afp32.litertlm)
Overall	63.78	60.91
STEM	56.35	53.36
Social Science	73.65	70.87
Humanities	66.68	64.35
Other	65.40	62.14
Hard	43.79	44.23

W8 uses dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32). W4 uses mixed INT4-block32 (linear) / INT8 (embed and lmhead) quantization (wi4b32_wi8) with FP32 activations (afp32). Thinking mode has been turned off to reduce context usage.

What is MiniCPM?

MiniCPM5-1B is the first model in the MiniCPM5 series from OpenBMB. It is a dense 1B-parameter Transformer built specifically for on-device, local, and resource-constrained deployment, while reaching 1B-class open-source SOTA in its size class.

Highlights

• 🏆 1B-class open-source SOTA — strongest in tool use, code generation, and difficult reasoning among comparable open models.
• 🧠 Hybrid Reasoning — a single checkpoint serves as both a fast assistant and a deliberate reasoner via a built-in <think> template (enable_thinking).
• 📏 Long context — native 131,072-token context length.
• 📱 Built for the edge — compact footprint designed for local assistants, coding agents, and tool-use workflows.

Model Information

Item	Value
Type	Causal Language Model
Architecture	Standard LlamaForCausalLM
Parameters	1,080,632,832 (~1B)
Non-Embedding Parameters	679,552,512
Layers	24
Attention Heads (GQA)	16 (Q) / 2 (KV)
Context Length	131,072

---

Use the model

Edge Gallery App (Android)

1. Get the App: Install the app from Google Play or download the latest APK from the GitHub releases page.
2. Importing the Model: Navigate to the Model manager within the app and click the "+" (plus) icon in the bottom-right corner. Two options will appear:
   • Import from HF (Recommended): Select this option, and a dialog box will appear showing an example Hugging Face model URL. Enter the HF link for the desired .litertlm model and click submit. The model will then appear in your list, and you can proceed to download it (a Hugging Face account login is required).
   • From local model file: First, download the .litertlm model directly to your Android device, OR download it to your computer and push it via ADB (e.g., adb push minicpm_dynamic_wi8_afp32_gpu_opt.litertlm /sdcard/Download/). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap "Import".

For full details on importing models and other features, see the Edge Gallery App Wiki.

To build the demo app from source, please follow the instructions from the GitHub repository.

Try It (Desktop/CLI)

Install uv and run the model directly from the LiteRT-LM command line:

uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B minicpm_dynamic_wi8_afp32_gpu_opt.litertlm --prompt="What is the capital of France?"

Links

• 🤗 Original model (BF16): openbmb/MiniCPM5-1B
• 📦 GitHub: OpenBMB/MiniCPM
• 🛠️ LiteRT docs: ai.google.dev/edge/litert

---

License

Released under the Apache-2.0 License, consistent with the upstream openbmb/MiniCPM5-1B.

Citation

@article{minicpm4,
  title={MiniCPM4: Ultra-efficient LLMs on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}