Add GPU-optimized LiteRT model and update README

#1
by hvt4 - opened
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ *.litertlm filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -14,18 +14,18 @@ tags:
14
  - tflite
15
  - on-device
16
  - edge-ai
17
- - upcoming
18
  ---
19
 
20
  # MiniCPM5-1B (LiteRT)
21
 
22
- > **Upcoming** The LiteRT (`.tflite`) build of MiniCPM5-1B is on the way.
23
- > Model weights are not available in this repository **yet**. Please **follow** this repo to be notified when they land.
24
-
25
- This repository will host the [**LiteRT**](https://ai.google.dev/edge/litert) (formerly TensorFlow Lite) version of **MiniCPM5-1B**, optimized for fully on-device inference on mobile and edge hardware.
26
 
27
  ---
28
 
 
 
 
 
29
  ## What is MiniCPM?
30
 
31
  **MiniCPM5-1B** is the first model in the **MiniCPM5** series from [OpenBMB](https://huggingface.co/openbmb). It is a dense **1B-parameter** Transformer built specifically for **on-device, local, and resource-constrained deployment**, while reaching **1B-class open-source SOTA** in its size class.
@@ -51,6 +51,26 @@ This repository will host the [**LiteRT**](https://ai.google.dev/edge/litert) (f
51
 
52
  ---
53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
  ## Links
55
 
56
  - 🤗 Original model (BF16): [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)
 
14
  - tflite
15
  - on-device
16
  - edge-ai
 
17
  ---
18
 
19
  # MiniCPM5-1B (LiteRT)
20
 
21
+ This repository hosts the [**LiteRT**](https://ai.google.dev/edge/litert) (formerly TensorFlow Lite) version of **MiniCPM5-1B**, optimized for fully on-device inference on mobile and edge hardware.
 
 
 
22
 
23
  ---
24
 
25
+ ## Available Models
26
+
27
+ * **`minicpm_dynamic_wi8_afp32_gpu_opt.litertlm`**: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32), heavily optimized for GPU execution.
28
+
29
  ## What is MiniCPM?
30
 
31
  **MiniCPM5-1B** is the first model in the **MiniCPM5** series from [OpenBMB](https://huggingface.co/openbmb). It is a dense **1B-parameter** Transformer built specifically for **on-device, local, and resource-constrained deployment**, while reaching **1B-class open-source SOTA** in its size class.
 
51
 
52
  ---
53
 
54
+ ## Use the model
55
+
56
+ ### Android
57
+
58
+ #### Edge Gallery App
59
+ * Download or build the [app](https://github.com/google-ai-edge/gallery?tab=readme-ov-file#-get-started-in-minutes) from GitHub.
60
+ * Install the [app](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery&pli=1) from Google Play.
61
+ * Follow the instructions in the app.
62
+
63
+ To build the demo app from source, please follow the [instructions](https://github.com/google-ai-edge/gallery/blob/main/README.md) from the GitHub repository.
64
+
65
+ ### Try It (Desktop/CLI)
66
+
67
+ Install `uv` and run the model directly from the command line:
68
+
69
+ ```bash
70
+ uv tool install litert-lm
71
+ uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B minicpm_dynamic_wi8_afp32_gpu_opt.litertlm --prompt="What is the capital of France?"
72
+ ```
73
+
74
  ## Links
75
 
76
  - 🤗 Original model (BF16): [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)
minicpm_dynamic_wi8_afp32_gpu_opt.litertlm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca61f7ccf2d34619f16f68155367e42ba5f38b29cd4c0e89b5548d35461308c4
3
+ size 1098475296