Instructions to use litert-community/MiniCPM5-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MiniCPM5-1B with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Add GPU-optimized LiteRT model and update README
#1
by hvt4 - opened
- .gitattributes +1 -0
- README.md +25 -5
- minicpm_dynamic_wi8_afp32_gpu_opt.litertlm +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
*.litertlm filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -14,18 +14,18 @@ tags:
|
|
| 14 |
- tflite
|
| 15 |
- on-device
|
| 16 |
- edge-ai
|
| 17 |
-
- upcoming
|
| 18 |
---
|
| 19 |
|
| 20 |
# MiniCPM5-1B (LiteRT)
|
| 21 |
|
| 22 |
-
|
| 23 |
-
> Model weights are not available in this repository **yet**. Please **follow** this repo to be notified when they land.
|
| 24 |
-
|
| 25 |
-
This repository will host the [**LiteRT**](https://ai.google.dev/edge/litert) (formerly TensorFlow Lite) version of **MiniCPM5-1B**, optimized for fully on-device inference on mobile and edge hardware.
|
| 26 |
|
| 27 |
---
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
## What is MiniCPM?
|
| 30 |
|
| 31 |
**MiniCPM5-1B** is the first model in the **MiniCPM5** series from [OpenBMB](https://huggingface.co/openbmb). It is a dense **1B-parameter** Transformer built specifically for **on-device, local, and resource-constrained deployment**, while reaching **1B-class open-source SOTA** in its size class.
|
|
@@ -51,6 +51,26 @@ This repository will host the [**LiteRT**](https://ai.google.dev/edge/litert) (f
|
|
| 51 |
|
| 52 |
---
|
| 53 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
## Links
|
| 55 |
|
| 56 |
- 🤗 Original model (BF16): [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)
|
|
|
|
| 14 |
- tflite
|
| 15 |
- on-device
|
| 16 |
- edge-ai
|
|
|
|
| 17 |
---
|
| 18 |
|
| 19 |
# MiniCPM5-1B (LiteRT)
|
| 20 |
|
| 21 |
+
This repository hosts the [**LiteRT**](https://ai.google.dev/edge/litert) (formerly TensorFlow Lite) version of **MiniCPM5-1B**, optimized for fully on-device inference on mobile and edge hardware.
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
---
|
| 24 |
|
| 25 |
+
## Available Models
|
| 26 |
+
|
| 27 |
+
* **`minicpm_dynamic_wi8_afp32_gpu_opt.litertlm`**: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32), heavily optimized for GPU execution.
|
| 28 |
+
|
| 29 |
## What is MiniCPM?
|
| 30 |
|
| 31 |
**MiniCPM5-1B** is the first model in the **MiniCPM5** series from [OpenBMB](https://huggingface.co/openbmb). It is a dense **1B-parameter** Transformer built specifically for **on-device, local, and resource-constrained deployment**, while reaching **1B-class open-source SOTA** in its size class.
|
|
|
|
| 51 |
|
| 52 |
---
|
| 53 |
|
| 54 |
+
## Use the model
|
| 55 |
+
|
| 56 |
+
### Android
|
| 57 |
+
|
| 58 |
+
#### Edge Gallery App
|
| 59 |
+
* Download or build the [app](https://github.com/google-ai-edge/gallery?tab=readme-ov-file#-get-started-in-minutes) from GitHub.
|
| 60 |
+
* Install the [app](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery&pli=1) from Google Play.
|
| 61 |
+
* Follow the instructions in the app.
|
| 62 |
+
|
| 63 |
+
To build the demo app from source, please follow the [instructions](https://github.com/google-ai-edge/gallery/blob/main/README.md) from the GitHub repository.
|
| 64 |
+
|
| 65 |
+
### Try It (Desktop/CLI)
|
| 66 |
+
|
| 67 |
+
Install `uv` and run the model directly from the command line:
|
| 68 |
+
|
| 69 |
+
```bash
|
| 70 |
+
uv tool install litert-lm
|
| 71 |
+
uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B minicpm_dynamic_wi8_afp32_gpu_opt.litertlm --prompt="What is the capital of France?"
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
## Links
|
| 75 |
|
| 76 |
- 🤗 Original model (BF16): [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)
|
minicpm_dynamic_wi8_afp32_gpu_opt.litertlm
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ca61f7ccf2d34619f16f68155367e42ba5f38b29cd4c0e89b5548d35461308c4
|
| 3 |
+
size 1098475296
|