---
license: apache-2.0
base_model: HuggingFaceTB/SmolLM2-360M-Instruct
pipeline_tag: text-generation
library_name: litert-lm
tags:
- chat
- litert-lm
- smollm
- on-device
---

# litert-community/SmolLM2-360M-Instruct

This model provides a variant of
[HuggingFaceTB/SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct) that is ready for
deployment on Android using the
[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM).

## Use the model

### Android

#### Edge Gallery App
* Download or build the [app](https://github.com/google-ai-edge/gallery?tab=readme-ov-file#-get-started-in-minutes) from GitHub.

* Install the [app](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery&pli=1) from Google Play.

* Follow the instructions in the app.

To build the demo app from source, please follow the [instructions](https://github.com/google-ai-edge/gallery/blob/main/README.md)
from the GitHub repository.

## Performance (measured)

### Apple M4 Max

Measured with the LiteRT-LM CLI: `litert-lm benchmark -p 256 -d 256 --runs 3 --cache no`
(litert-lm 0.15.0) on an idle Apple M4 Max (macOS); 256 prefill / 256 decode tokens, 3 iterations
averaged by the tool. A desktop reference point — phone-side figures vary by SoC and backend.

| Backend | Prefill (tokens/s) | Decode (tokens/s) | Time-to-first-token (s) |
|---|---|---|---|
| CPU | 588 | 37.3 | 0.63 |
| GPU | 2,773 | 133.6 | 0.11 |

### Galaxy S26 — GPU vs CPU (litert-lm 0.16.0)

Measured on a physical Samsung Galaxy S26 (SM-S942Q, Snapdragon 8 Elite Gen 5 / SM8850, Android 16) with `litert_lm_advanced_main` from the litert-lm v0.16.0 release; the GPU backend is OpenCL (`LITERT_CL`). One fixed 205-token prompt text (206 tokens under this tokenizer), `--benchmark`. GPU: two runs back-to-back; CPU: three runs spread across device thermal states (skin temperature up to 41.9 °C recorded on the warmest) — small-model CPU speed on a phone moves a lot with temperature, and the CPU range below is that, not noise. Peak RSS is the process VmHWM. Before quoting, the same file was run on each backend with a real prompt: both produced a correct text answer.

| Backend | Prefill (206 tok) | Decode | Time-to-first-token | Peak RSS |
|---|---|---|---|---|
| **GPU (OpenCL)** | **540–541 tok/s** | **31.5–36.0 tok/s** | **0.41 s** | **546 MB** |
| CPU (XNNPACK) | 118–211 tok/s | 12.7–19.5 tok/s | 1.03–1.82 s | 1409 MB |

The GPU takes the whole graph (decode 1362/1362, prefill 1491/1491 ops on `LITERT_CL`) and wins across the board on this bundle: prefill ≥2.6× at any temperature, decode 1.6–2.8×, time-to-first-token 2.5–4.4×, and peak RSS 2.6× lower.