InternVL3-1B — LiteRT-LM (on-device Vision-Language Model)

OpenGVLab/InternVL3-1B converted to the LiteRT-LM (.litertlm) format for on-device image+text inference with Google's LiteRT-LM runtime (the engine behind the official litert-community/* models).

InternVL3-1B is the smallest InternVL3 vision-language model: an InternViT vision encoder + pixel-shuffle + MLP projector feeding a Qwen2.5-0.5B language decoder. At 738 MB it is a tiny, fast on-device VLM — give it an image and a question, get a grounded answer, fully offline. (See InternVL3-2B-LiteRT for the larger sibling.)

File	InternVL3-1B.litertlm (~738 MB)
Vision	InternViT-300M encoder (4D-clean attention, GPU-friendly) + pixel-shuffle + MLP projector, int8 — single 448×448 image → 256 image tokens
Decoder	Qwen2.5-0.5B (896-dim, 24 layers), int4 weights (symmetric, blockwise-32 + OCTAV); input embedding INT8 (externalized)
Compute	integer
Context (KV cache)	2048
Image input	resized to 448×448 (ImageNet normalization baked into the vision encoder)
Base model	OpenGVLab/InternVL3-1B

How to use

1. Install the runtime

pip install litert-lm

2. Run it in one command — this downloads the bundle, encodes your image and answers:

litert-lm run --from-huggingface-repo litert-community/InternVL3-1B InternVL3-1B.litertlm \
  --attachment photo.jpg \
  --prompt "Describe this image in one sentence."

On the COCO sample image huggingface/documentation-images/coco_sample.png (two tabby cats on a pink blanket, remote controls beside them) this prints:

The image shows two cats lying on a pink bed with two remote controls.

Drop --prompt for an interactive chat, and pass --attachment more than once for several images. litert-lm serve exposes the same bundle as a local OpenAI-compatible API. The same file runs on macOS, Linux and Windows.

Performance

litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine. These figures cover the text path; the vision encoder runs once per image and is not included.

Device	Backend	Prefill (256)	Decode	TTFT
Apple M4 Max (macOS)	CPU	454 tok/s	94.0 tok/s	0.62 s

Every desktop backend listed above was checked by actually generating a caption on it, not just by reading the benchmark tool's output.

The macOS GPU backend is not usable for this bundle on litert-lm 0.15.0. Asked to describe an image with --backend gpu, it returns a run of ! characters instead of a caption. litert-lm benchmark still reports GPU numbers for it — 5,583 tok/s prefill and 342 tok/s decode — but they measure a path that produces no usable text, so no GPU row is quoted here. Use the CPU backend on the desktop. This is a desktop-runtime observation and says nothing about the iPhone or Android GPU paths described elsewhere on this card.

Accuracy note

Output is coherent and image-grounded (CPU-verified; the vision tower converts bit-faithfully to the reference, float CPU-parity corr ≈ 1.0). On-device behavior mirrors the larger InternVL3-2B build (same conversion recipe) — single-image VQA on GPU is fast and accurate; being 0.5B-decoder it is the fastest/smallest of the family.

⚠️ Known limitation — one image per conversation on the GPU backend

Single-image VQA — the primary use case — works great on GPU. But on the GPU (Metal) backend, a second image in the same conversation truncates the answer — ask about one image per chat (start a new conversation for a different image). This is GPU-delegate-specific, not a model/bundle issue: on the CPU backend, multi-image works perfectly (verified), and the same GPU truncation reproduces with other fast_vlm models. For reliable multi-image, run on the CPU backend.

Run on iPhone / macOS

Use the LiteRT-LM Swift runtime (swift-litert-lm / the LiteRTDemo sample). Load InternVL3-1B.litertlm with the image (vision) tower enabled (modalities Modality.textImage / [.vision] — a vision-only bundle, no audio tower), attach a photo, and ask a question.

Run on Android — Google AI Edge Gallery

Update (July 2026): Google AI Edge Gallery v1.0.16+ can import litert-lm models directly from Hugging Face inside the app (tap +) — no computer or adb needed. The manual steps below are only required on older builds or for sideloading a local file.

Run this model with image input in the official Google AI Edge Gallery app — no custom app needed (the bundle carries the tokenizer, chat template, and image preprocessing config):

1. Push the bundle onto the phone (or download it there directly from this repo): adb push InternVL3-1B.litertlm /sdcard/Download/
2. Open the Gallery app, tap the + icon (bottom-right) and pick InternVL3-1B.litertlm in the file picker.
3. In the Import Model dialog, check "Support image" (required for image input), pick GPU (fast) or CPU, then tap Import.
4. Open the Ask Image task, choose the imported model, attach a photo, and ask.

Tip: on the GPU backend use one image per conversation (a known GPU-delegate trait of fast_vlm models); pick CPU if you want multiple images in one chat.

Run on desktop (LiteRT-LM CLI)

The same .litertlm bundle runs on macOS / Linux / Windows with the official LiteRT-LM CLI — including as a local OpenAI-compatible API server:

pip install litert-lm
litert-lm import --from-huggingface-repo litert-community/InternVL3-1B InternVL3-1B.litertlm internvl3-1b
litert-lm run internvl3-1b     # interactive chat in the terminal
litert-lm serve           # local OpenAI-compatible API server

Conversion notes

• LiteRT-LM fast_vlm bundle: VISION_ENCODER ([1,448,448,3]→[1,256,4096]) + VISION_ADAPTER ([1,256,4096]→[1,256,896]) + single-token EMBEDDER + PREFILL_DECODE (embeddings-input).
• The vision encoder bakes ImageNet normalization + the NCHW transpose into the graph, and the InternViT attention is rewritten 4D-clean (qkv split before the head reshape — no GPU-rejected 5D reshape), numerically identical (corr ≈ 1.0).
• Decoder exported with externalized embedder; InternVL's dynamic-NTK rope_scaling is stripped to base RoPE (valid since the export cache ≤ the base context window).

License

MIT (the InternVL model) + Apache-2.0 (the Qwen2.5 language component). See the base model card. Converted artifacts are released under the same terms.