frankdu/SD-Turbo-Mobile-INT8-LiteRT

SD-Turbo Mobile INT8 LiteRT

Powered by Stability AI. A compact, sharded 512×512 text-to-image pipeline, converted from the pinned FP32 LiteRT export with LiteRT CLI. The architecture and sampler are unchanged. Neural encoder/UNet weights use dynamic channelwise INT8; VAE weights use FP16 with float activations. Sampler graphs remain FP32.

Runtime artifacts: 1.32 GB, down from 5.03 GB. Largest graph: 99.3 MB. One neural graph is loaded at a time. Download size is not peak RAM: activations, dequantization, driver caches and the rest of Android also consume memory.

Intended phones and limits

The design targets the 12–16 GB RAM class of Pixel phones. Pixel 10 Pro uses Tensor G5 and 16 GB RAM; Pixel 11 Pro uses Tensor G6 and 12 or 16 GB depending on storage configuration. See Pixel 10 specs and Pixel 11 specs. This is a portable CPU pipeline, not a Tensor AOT/TPU compilation. Current GPU layouts fail LiteRT 2.3 compilation on the measured Tensor G5 and require a separate compatible GPU export; compressed weights alone do not fix that. Pixel 10 Pro and Pixel 11 Pro themselves have not been measured. Exact Android device/backend results, including failures, are in validation.json.

Measured on Pixel 10 Pro Fold / Tensor G5: CPU completed a 512x512 blue-teapot image in 26.92 seconds with 1.75 GiB process peak RSS. This is one smoke sample in a separate validation app using private model storage. GPU failed shape compilation and is unavailable for these graph layouts. Pixel 10 Pro and Pixel 11 Pro themselves remain unmeasured.

Validation

Three complete desktop LiteRT CPU generations (apple, blue teapot, orange cat) were compared with the FP32 source using identical Java Gaussian noise and seeds. All outputs are finite, nonuniform and visually recognizable. Quantization can change composition and detail; these outputs are not pixel-identical to FP32. Per-prompt PSNR, pixel correlation, process peak RSS and timings are published. Desktop process RSS includes Python/tokenizer overhead and is not a phone claim.

Run on CPU

hf download frankdu/SD-Turbo-Mobile-INT8-LiteRT --local-dir SD-Turbo-Mobile-INT8-LiteRT
cd SD-Turbo-Mobile-INT8-LiteRT
python3.12 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
python run_portable.py --prompt "A blue ceramic teapot on a wooden table, morning light." --seed 7 --out teapot.png

Android uses the word-image-litert-v1 manifest, LiteRT 2.3.0 CompiledModel, TensorBuffer int64 I/O, and one graph at a time. Reuse the input TensorBuffer for every output carrying aliasOfInput in the manifest, and close shared buffers only once. A passthrough output can be the same physical TFLite tensor as its input; allocating it separately overwrites valid token inputs with uninitialized memory. The Android app includes this fix. No automatic CPU fallback is advertised as GPU execution.

Reproduce and audit

quantization.json records versions, per-graph sizes, recipes and source manifest hash. conversion/quantize_phone.py invokes LiteRT CLI. Its guarded litert_quantize_conv.py entry point fixes an ai-edge-quantizer 0.10.0 bug in the

32 MiB convolution chunking path: broadcast scales in the original 4D shape, then reshape alongside weights. Installed packages are unchanged. The fix was checked bit-for-bit against unchunked per-channel INT8 arithmetic on a 36 MiB convolution tensor. No unsupported recipe name is silently substituted.

manifest.json pins runtime files; artifact_manifest.json and SHA256SUMS cover the complete publication. Original FP32 packages remain available separately. Source model card, LICENSE.md and NOTICE are retained. Model weights retain the Stability AI Community License and source usage terms; this publication does not relicense them. Companion code retains its original CODE_LICENSE.