README.md

---
license: apache-2.0
language:
  - en
  - zh
base_model:
  - openbmb/MiniCPM5-1B
pipeline_tag: text-generation
library_name: litert
tags:
  - minicpm
  - minicpm5
  - litert
  - tflite
  - on-device
  - edge-ai
---

# MiniCPM5-1B (LiteRT-LM)

This repository hosts the [**LiteRT-LM**](https://ai.google.dev/edge/litert-lm) (LiteRT formerly known as TensorFlow Lite) version of **MiniCPM5-1B**, optimized for fully on-device inference on mobile and edge hardware.

---

## Available Models

* **`minicpm_dynamic_wi8_afp32_gpu_opt.litertlm`**: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32), heavily optimized for GPU execution.

## What is MiniCPM?

**MiniCPM5-1B** is the first model in the **MiniCPM5** series from [OpenBMB](https://huggingface.co/openbmb). It is a dense **1B-parameter** Transformer built specifically for **on-device, local, and resource-constrained deployment**, while reaching **1B-class open-source SOTA** in its size class.

### Highlights

- 🏆 **1B-class open-source SOTA** — strongest in tool use, code generation, and difficult reasoning among comparable open models.
- 🧠 **Hybrid Reasoning** — a single checkpoint serves as both a fast assistant and a deliberate reasoner via a built-in `<think>` template (`enable_thinking`).
- 📏 **Long context** — native **131,072**-token context length.
- 📱 **Built for the edge** — compact footprint designed for local assistants, coding agents, and tool-use workflows.

### Model Information

| Item | Value |
| --- | --- |
| Type | Causal Language Model |
| Architecture | Standard `LlamaForCausalLM` |
| Parameters | 1,080,632,832 (~1B) |
| Non-Embedding Parameters | 679,552,512 |
| Layers | 24 |
| Attention Heads (GQA) | 16 (Q) / 2 (KV) |
| Context Length | 131,072 |

---

## Use the model

### Android

#### Edge Gallery App
* Download or build the [app](https://github.com/google-ai-edge/gallery?tab=readme-ov-file#-get-started-in-minutes) from GitHub.
* Install the [app](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery&pli=1) from Google Play.
* Follow the instructions in the app.

To build the demo app from source, please follow the [instructions](https://github.com/google-ai-edge/gallery/blob/main/README.md) from the GitHub repository.

### Try It (Desktop/CLI)

Install `uv` and run the model directly from the LiteRT-LM command line:

```bash
uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B minicpm_dynamic_wi8_afp32_gpu_opt.litertlm --prompt="What is the capital of France?"
```

## Links

- 🤗 Original model (BF16): [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)
- 📦 GitHub: [OpenBMB/MiniCPM](https://github.com/OpenBMB/MiniCPM)
- 🛠️ LiteRT docs: [ai.google.dev/edge/litert](https://ai.google.dev/edge/litert)

---

## License

Released under the **Apache-2.0 License**, consistent with the upstream [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B).

## Citation

```bibtex
@article{minicpm4,
  title={MiniCPM4: Ultra-efficient LLMs on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}
```