Instructions to use litert-community/InternVL3-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT-LM
How to use litert-community/InternVL3-1B with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli litert-lm run \ --from-huggingface-repo=litert-community/InternVL3-1B \ --prompt="Write me a poem"
- LiteRT
How to use litert-community/InternVL3-1B with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Card: verified one-command usage, measured Performance table (backends gated on real generation), accuracy note
Browse files
README.md
CHANGED
|
@@ -37,7 +37,43 @@ tiny, fast on-device VLM — give it an image and a question, get a grounded ans
|
|
| 37 |
| **Image input** | resized to 448×448 (ImageNet normalization baked into the vision encoder) |
|
| 38 |
| **Base model** | OpenGVLab/InternVL3-1B |
|
| 39 |
|
| 40 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
Output is coherent and image-grounded (CPU-verified; the vision tower converts bit-faithfully to the
|
| 43 |
reference, float CPU-parity corr ≈ 1.0). On-device behavior mirrors the larger InternVL3-2B build
|
|
|
|
| 37 |
| **Image input** | resized to 448×448 (ImageNet normalization baked into the vision encoder) |
|
| 38 |
| **Base model** | OpenGVLab/InternVL3-1B |
|
| 39 |
|
| 40 |
+
## How to use
|
| 41 |
+
|
| 42 |
+
**1. Install the runtime**
|
| 43 |
+
|
| 44 |
+
```bash
|
| 45 |
+
pip install litert-lm
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
**2. Run it in one command** — this downloads the bundle, encodes your image and answers:
|
| 49 |
+
|
| 50 |
+
```bash
|
| 51 |
+
litert-lm run --from-huggingface-repo litert-community/InternVL3-1B InternVL3-1B.litertlm \
|
| 52 |
+
--attachment photo.jpg \
|
| 53 |
+
--prompt "Describe this image in one sentence."
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
On the COCO sample image `huggingface/documentation-images/coco_sample.png` (two tabby cats on a pink blanket, remote controls beside them) this prints:
|
| 57 |
+
|
| 58 |
+
```
|
| 59 |
+
The image shows two cats lying on a pink bed with two remote controls.
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Drop `--prompt` for an interactive chat, and pass `--attachment` more than once for several images. `litert-lm serve` exposes the same bundle as a local OpenAI-compatible API. The same file runs on macOS, Linux and Windows.
|
| 63 |
+
|
| 64 |
+
## Performance
|
| 65 |
+
|
| 66 |
+
`litert-lm benchmark` (litert-lm 0.15.0) on an Apple M4 Max, `-p 256 -d 256 --runs 3` (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine. These figures cover the text path; the vision encoder runs once per image and is not included.
|
| 67 |
+
|
| 68 |
+
| Device | Backend | Prefill (256) | Decode | TTFT |
|
| 69 |
+
|---|---|---|---|---|
|
| 70 |
+
| Apple M4 Max (macOS) | CPU | 454 tok/s | 94.0 tok/s | 0.62 s |
|
| 71 |
+
|
| 72 |
+
Every desktop backend listed above was checked by actually generating a caption on it, not just by reading the benchmark tool's output.
|
| 73 |
+
|
| 74 |
+
**The macOS GPU backend is not usable for this bundle on litert-lm 0.15.0.** Asked to describe an image with `--backend gpu`, it returns a run of `!` characters instead of a caption. `litert-lm benchmark` still reports GPU numbers for it — 5,583 tok/s prefill and 342 tok/s decode — but they measure a path that produces no usable text, so no GPU row is quoted here. Use the CPU backend on the desktop. This is a desktop-runtime observation and says nothing about the iPhone or Android GPU paths described elsewhere on this card.
|
| 75 |
+
|
| 76 |
+
### Accuracy note
|
| 77 |
|
| 78 |
Output is coherent and image-grounded (CPU-verified; the vision tower converts bit-faithfully to the
|
| 79 |
reference, float CPU-parity corr ≈ 1.0). On-device behavior mirrors the larger InternVL3-2B build
|