Add measured performance section (M4 Max)

c99338c verified 9 days ago

1.48 kB

license: apache-2.0
base_model: HuggingFaceTB/SmolLM2-135M-Instruct
pipeline_tag: text-generation
library_name: litert-lm
tags:
  - chat
  - litert-lm
  - smollm
  - on-device

litert-community/SmolLM2-135M-Instruct

This model provides a variant of HuggingFaceTB/SmolLM2-135M-Instruct that is ready for deployment on Android using the LiteRT-LM.

Use the model

Android

Edge Gallery App

Download or build the app from GitHub.
Install the app from Google Play.
Follow the instructions in the app.

To build the demo app from source, please follow the instructions from the GitHub repository.

Performance (Apple M4 Max, measured)

Measured with the LiteRT-LM CLI: litert-lm benchmark -p 256 -d 256 --runs 3 --cache no (litert-lm 0.15.0) on an idle Apple M4 Max (macOS); 256 prefill / 256 decode tokens, 3 iterations averaged by the tool. A desktop reference point — phone-side figures vary by SoC and backend.

Backend	Prefill (tokens/s)	Decode (tokens/s)	Time-to-first-token (s)
CPU	1,698	104.9	0.22
GPU	7,571	259.6	0.04