litert-community/iris-3b-Turbo-LiteRT

iris-3b Turbo — LiteRT

iris-3b Turbo (Speridlabs, Apache-2.0), converted with litert-torch, makes a 768×768 image from a text prompt on a Galaxy S26, with every graph on the phone's GPU through LiteRT CompiledModel: 212.6 s for the sample app's first image after the install, with the phone not throttled (Android thermal status 0; measured 2026-10-21).

Nothing leaves the phone: the tokenizer, the Qwen3-VL-4B text encoder, the 3B pixel-space diffusion transformer and the sampler all run on it. iris-3b draws pixels directly (no VAE), and Turbo needs 4 steps with no classifier-free guidance.

Every image above was made on the Pixel 8a, with every graph on its GPU at FP32.

How it runs

prompt -> Qwen3-VL tokenizer -> bf16 embedding lookup                                   (host)
       -> 12 text encoder graphs, one per hidden-state tap (decoder layers 2, 5, ..., 35)
       -> text adapter graph (once per prompt)
       -> 4 steps x (3 dual-stream graphs -> 3 single-stream graphs -> pixel-stage graph)
          -> sampler update                                                              (host)
       -> 768x768 RGB

Each graph is loaded for its run and closed again, so one graph's weights are in memory at a time (the largest graph file is 0.9 GB; the app's peak memory is 4.6–5.0 GiB, see Measured). On both phones below, every graph compiles whole onto the GPU delegate (Replacing N out of N node(s) with delegate (LITERT_CL) node, yielding 1 partitions). The fully-connected weights are stored as fp16 with a DEQUANTIZE in front of each; activations are fp32, and the GPU runs at FP32.

Minimal usage

model/ holds the 24 files both pipelines read (13.7 GB): 20 graphs, the token embedding table, the host tensors, the tokenizer tables and iris3b_config.json.

Python (CPU)

python/iris3b_litert.py is the whole pipeline in one file: the tokenizer, the embedding lookup and every host step, with each graph on LiteRT CompiledModel (CPU, XNNPACK). It runs the same files and the same steps as the Android engine, so it is the reference to check a port against.

hf download litert-community/iris-3b-Turbo-LiteRT --local-dir iris-3b-Turbo-LiteRT
cd iris-3b-Turbo-LiteRT/python
pip install -r requirements.txt
python iris3b_litert.py --prompt "a cozy wooden cabin in a snowy pine forest at night, warm light in the windows" --seed 7 --out cabin.png

From Python:

from iris3b_litert import Iris3B, save_png, seed_noise

pipe = Iris3B("../model", threads=8)  # reads model/iris3b_config.json; each graph is loaded for its run
image = pipe.generate("a red apple on a wooden table, studio lighting", seed_noise(0))
save_png(image, "apple.png")          # image: float32 [3, 768, 768], clamped to [-1, 1]

seed_noise(seed) is java.util.Random(seed).nextGaussian(), the starting noise the Android code below makes for the same seed (checked bit for bit against the JVM's java.util.Random for seeds 0 and 7). --reference-noise starts from the noise of the upstream PyTorch reference run instead. With --reference-noise, the apple prompt took 23.2 s on a Mac (Apple M4 Max, 8 CPU threads), and its image matches the PyTorch fp32 image at PSNR 106.45 dB / SSIM 1.0.

Kotlin (Android, GPU)

android/ is the engine of the iris-3b sample app: 7 Kotlin files in package com.iris3b that use only the JDK and LiteRT. Add implementation("com.google.ai.edge.litert:litert:2.2.0"), copy the files into your app, and put the 24 files of model/ in the app's files folder (android/README.md shows how). Then:

import android.graphics.Bitmap
import com.google.ai.edge.litert.Environment
import com.iris3b.HostOps
import com.iris3b.IrisPipeline
import com.iris3b.LiteRtExec
import com.iris3b.PromptTokens
import com.iris3b.RunFiles
import com.iris3b.StageLog
import java.io.File
import java.util.Random

// The 24 files of model/ are in the app's files folder. Run this off the main thread (minutes).
val dir = context.filesDir
File(dir, "iris3b_rows.json")
    .writeText("""{"kind":"generic","config":"iris3b_config.json","inputs":[],"rows":[]}""")
val files = RunFiles(dir, "iris3b_rows.json")  // reads iris3b_config.json: graphs, GPU FP32, host tensors
files.materializeTokenizer()                    // writes tok/: vocab, merges, template ids
val tokens = PromptTokens(File(dir, "tok"))
val env = Environment.create(context)           // one Environment for every graph of the process
val pipeline = IrisPipeline(LiteRtExec(env, dir), files.engineConfig(), files.host, tokens)

val random = Random(7L)                         // the starting noise of seed 7
val noise = FloatArray(HostOps.IMAGE_FLOATS) { random.nextGaussian().toFloat() }
val log = StageLog(clock = System::nanoTime, memory = { emptyMap() })
val result = pipeline.generate(
    "a cozy wooden cabin in a snowy pine forest at night, warm light in the windows", noise, log)

// result.image: CHW floats [3, 768, 768] in [-1, 1] -> an ARGB bitmap
val size = HostOps.IMAGE_SIZE
val plane = size * size
fun level(v: Float) = ((v + 1f) * 127.5f + 0.5f).toInt().coerceIn(0, 255)
val pixels = IntArray(plane) { i ->
    (0xFF shl 24) or (level(result.image[i]) shl 16) or
        (level(result.image[plane + i]) shl 8) or level(result.image[2 * plane + i])
}
val bitmap = Bitmap.createBitmap(pixels, size, size, Bitmap.Config.ARGB_8888)
tokens.close()
env.close()

IrisPipeline also takes stop, read before every graph (the sample app's Stop button and its free-memory guard both go through it), and StageLog takes a listener that sees every step as it happens.

Demo

The sample app on a Pixel 8a: the prompt in the editor, Generate, the image. The middle is sped up; the phone took 508.3 s (the app's first image after the install, with the screen being recorded).

Measured

	Pixel 8a	Galaxy S26
Runtime	LiteRT 2.2.0 (Kotlin), every graph on Accelerator.GPU at FP32, Android 16	LiteRT 2.2.0 (Kotlin), every graph on Accelerator.GPU at FP32, Android 16
Code	the android/ engine in the iris-3b sample app: its Generate path, started by the debug verify extra (reference noise)	the android/ engine in the iris-3b sample app: its Generate path, started by the debug verify extra (reference noise)
First image after the install (apple prompt)	468.8 s, thermal status 0 at the start	212.6 s, thermal status 0 at the start
Cubes / lighthouse prompts	503.7 s / 487.5 s, each the first image after a fresh install, thermal status 0 at the start	280.9 s / 320.1 s, the second and third images in a row, thermal status 2 at the start of each
First image, where the time goes (load = GPU compile)	text encoder 38.0 s load + 5.6 s run; adapter 5.1 + 2.0 s; transformer 267.1 + 141.5 s; closing graphs 7.8 s	text encoder 27.2 + 2.2 s; adapter 5.6 + 0.9 s; transformer 100.1 + 74.4 s; closing graphs 1.6 s
Peak memory of the app (VmHWM)	5.0 GiB	4.6 GiB
Free memory (MemAvailable): before an image / lowest during one	3.6–3.8 GiB / 288–444 MiB	6.4–7.6 GiB / 3.6–3.8 GiB
Other processes closed by Android's low-memory killer during one image	116–169, never the app	2–67, never the app
Image vs PyTorch fp32, PSNR (apple / cubes / lighthouse)	106.76 / 70.92 / 73.09 dB	106.84 / 74.02 / 72.57 dB
Image vs PyTorch fp32, SSIM	1.0 / 1.0 / 1.0	1.0 / 1.0 / 0.99999
Measured	2026-10-21	2026-10-21

Prompts: "a red apple on a wooden table, studio lighting", "three blue glass cubes, two red spheres & one green cone on a white table; 45° top light, 50mm f/1.8 (sharp focus!)" and "夕暮れの海辺に立つ白い灯台、　柔らかな光と長い影" (a white lighthouse by the sea at dusk, soft light and long shadows). Reference: the upstream PyTorch pipeline with the text encoder and the transformer in fp32, from the same starting noise; PSNR and SSIM over the 768×768 image.

Settings measured but not shipped

iris3b_config.json ships "residency": "P1" (each graph is loaded for its run) with all 20 graphs on the GPU at FP32; every number above uses it. Two other settings were measured on the Galaxy S26:

• Text encoder on the CPU. In an earlier build of the same engine in a test app, both runs from thermal status 1: with the 12 text encoder graphs on the CPU (XNNPACK, 4 threads) and the rest on the GPU, the first image took 186.2 s, 7.6 % less than the 201.6 s with every graph on the GPU.
• P4: compile the next graph while the current one runs. The engine has this mode. Through the app's BenchActivity, it made the first image 149.1 s (from thermal status 0) and the second 211.1 s. During the third image in a row, while a compile overlapped a run, the GPU driver ended the app: Exiting the process com.iris3b from function cl_a8x_cmdbuf_mgr_submit_ibs.

Files

File (in model/)	What it is	Size
iris_te_c00_f16w.tflite … iris_te_c11_f16w.tflite	text encoder: Qwen3-VL-4B decoder layers 1–2, then 3 layers per graph up to layer 35; each graph's output is one of the 12 hidden-state taps	12 files, 7.07 GB
iris_dit_adapter_f16w_dit.tflite	text adapter: the 12 taps → 300 conditioning tokens (once per prompt)	0.50 GB
iris_dit_dual{0,1,2}_f16w_dit.tflite	dual-stream blocks 1–3, 4–6, 7–8 (dual0 embeds the image patches)	2.42 GB
iris_dit_single{0,1,2}_f16w_dit.tflite	single-stream blocks 1–6, 7–12, 13–16	2.41 GB
iris_dit_pixel_f16w_dit.tflite	pixel stage: the velocity of every pixel	0.47 GB
iris_embed_bf16.f32	token embedding table, raw little-endian bf16 [151936, 2560]	0.78 GB
iris_host.f32	host tensors (float32): rotary tables, the timestep embeddings and modulation rows of the 4 fixed steps, the step sigmas, the reference starting noise	58.7 MB
iris_tokenizer.json	tokenizer tables (vocab, merges, added tokens) and the chat-template ids	6.6 MB
iris3b_config.json	every graph's file, inputs and outputs, GPU precision FP32; host tensor offsets	19 kB
SHA256SUMS	sha256 of the files above

How it was converted

• litert-torch 0.9.4 exported the upstream PyTorch modules (iris-3b Turbo at revision adc05b9, Qwen3-VL-4B-Instruct at ebb281e) as fp32 graphs. ai-edge-quantizer 0.9.0 then stored the fully-connected weights as fp16 with litert-torch's full_fp16_recipe (explicit DEQUANTIZE, fp32 compute). The checkpoint's weights are bf16, which fp16 holds exactly except for values smaller than 2^-17 in magnitude, which are rounded; the adapter's attention-query rows, which are not bf16 values in the exported fp32 graph, are rounded too.
• Turbo's 4 step sigmas are fixed, so the timestep embeddings and modulation rows of those steps are precomputed into iris_host.f32, with the rotary tables.
• The tokenizer is ported, not called: the Kotlin port and the Python port both give the Hugging Face tokenizer's ids on 15 test captions and 2,000 random strings.

Limits

• 768×768 only. Turbo was distilled at about one megapixel; in the upstream PyTorch pipeline, 512×512 and 256×256 come out broken.
• Prompts up to 295 tokens. After the chat template's 34-token prefix, the text encoder reads a 300-token window, and the template's suffix takes 5 of it; a longer prompt is cut (the sample app says so).
• FP32 on the GPU. The config sets FP32 for every graph. With fp16 GPU compute, the images drift from the reference for some prompts.
• No NPU path. On the Galaxy S26's NPU (Qualcomm HTP), the float versions (fp32 weights) of the first text-encoder graph and of the pixel stage, a one-layer text-encoder probe and a one-block transformer probe were tried. Each went onto the NPU whole (one partition), but the NPU computes in fp16, which does not hold the image: the transformer block's output had non-finite values.
• Storage and memory. The files take 13.7 GB on the phone. On the Pixel 8a, free memory was 3.6–3.8 GiB before an image and 288–444 MiB at its lowest during one, and Android closed 116–169 other processes during each image to make room. Close other apps first. The sample app reads free memory every 200 ms and stops before the next graph below 200,000 kB (about 195 MiB).
• Only the Turbo text-to-image checkpoint is converted (the upstream repository also has a base model, a depth model and an upscaler).

License

Apache-2.0 (LICENSE). The graphs, the embedding table, the host tensors and the tokenizer tables are derived from two Apache-2.0 works, listed with the changes in NOTICE:

• iris-3b — Copyright 2026 Speridlabs (model, code).
• Qwen3-VL-4B-Instruct — Alibaba Cloud (model): the text encoder layers, the token embedding table and the tokenizer.