litert-community/vit-base-patch16-224

Google ViT Base Patch16 224 — LiteRT

LiteRT/TFLite variants of google/vit-base-patch16-224, pinned to source revision 3f49326eb077187dfe1c2a2bb15fbd74e6ab91e3.

The package provides FP32, A16W8 and mixed A8W8 models for image classification. All three have FP32 input and output and require external image preprocessing. The .tflite files are portable source artifacts for target-specific NPU compilation; they do not contain device-specific compiled NPU programs.

Model files

File	Description
model_fp32.tflite	Floating-point reference with FP32 weights.
model_a16w8.tflite	INT16 transformer activations and INT8 fully connected weights; the primary quantized option for compatible NPUs.
model_a8w8.tflite	Experimental mixed precision: INT8 fully connected layers with INT16 attention, residual and LayerNorm boundaries.
config.json	Original source configuration and class mapping.
preprocessor_config.json	Source image preprocessing configuration.
labels.json	Class index to label mapping.
model_metadata.json	Source identity, tensor interfaces, quantization scope and model hashes.
run_image.py	CPU image-classification example.
requirements.txt	Runtime and preprocessing package versions.

Choosing a variant

• FP32 provides the floating-point reference and is the example's default.
• A16W8 offers the best observed accuracy/speed balance on the tested NPUs. Its fully connected layers use INT16 inputs and outputs, INT8 weights and INT64 biases. CPU fallback can be slow, and the tested LiteRT GPU IR importer does not support its INT16 quantization.
• Mixed A8W8 is retained as a reference that may suit devices limited to 32-bit integer accumulation in their fully connected kernels. It has a substantial accuracy tradeoff and retains INT16 regions, so backend support must be verified.

Extra notes: Apple ANE and QC NPUs support FP16 natively and the performances are usually very good. Deloying a FP16 model is much easier since it does not require calibration steps. Note that the A16W8 and A8W8 assets provided here were calibrated from a particular dataset and should not used in production directly. A FP16 model file is not provided here is because it is easy to be generated from the fp32 asset. It is actually a top choice.

TorchAO quantization recipe

Both variants use post-training static quantization, without QAT or fine-tuning. The run used PyTorch 2.14.1+cu130, TorchAO 0.18.0+cu130 and Transformers 5.21.0. To reproduce the quantization approach:

• Load the pinned source checkpoint in FP32 evaluation mode. Calibrate with 500 ImageNet-1k validation images, batch size one, using the included PIL preprocessing configuration. Keep calibration separate from evaluation.
• Observe inputs to all 73 torch.nn.Linear layers, including the classifier, with AffineQuantizedMinMaxObserver: asymmetric INT8 [-128, 127], PerTensor() granularity, FP32 scales and INT32 zero points.
• Quantize the original FP32 weights with symmetric PerRow() granularity (one scale per output channel), zero point zero and the explicit narrow range [-127, 127]. Use TorchAO's choose_qparams_affine and quantize_affine with these bounds; this weight-range customization is additional to the activation configuration below.

After calibrating each layer's input observer, its static configuration is:

from torchao.quantization import Int8StaticActivationInt8WeightConfig
from torchao.quantization.granularity import PerRow, PerTensor
from torchao.quantization.quant_primitives import MappingType

scale, zero_point = observer.calculate_qparams()
config = Int8StaticActivationInt8WeightConfig(
    act_quant_scale=scale,
    act_quant_zero_point=zero_point,
    granularity=[PerTensor(), PerRow()],
    act_mapping_type=MappingType.ASYMMETRIC,
    set_inductor_config=False,
)

The final TFLite export also needs calibrated linear-output and transformer activation ranges, collected using the same 500 images. A16W8 uses symmetric INT16 activations with zero point zero and INT64 fully connected biases. Mixed A8W8 keeps asymmetric INT8 fully connected inputs/outputs and INT32 biases, with INT16 attention, residual and LayerNorm boundaries. Both retain the same INT8 weight codes and scales; activation scales are per-tensor.

The transformer graphs keep activations quantized across attention, residual, reshape, transpose, softmax and GELU operations. LayerNorm uses composite operators with quantized interfaces and floating-point decomposition subgraphs for fallback execution. The patch embedding and public input/output remain floating-point, so these models do not use integer arithmetic throughout.

A16W8 retains input-scale coarsening for fully connected layers that needed it under a 32-bit accumulator profile, but omits explicit accumulator input clipping. Validate accumulator behavior and accuracy on another backend.

TorchAO provides the linear quantization stage; recreating the complete TFLite graph requires export support for the activation encodings above. The exact calibration image list is not distributed, so this recipe does not guarantee identical scales or files. See the TorchAO static quantization guide for the observer/calibration workflow.

Accuracy evaluation

The results below use the same 5,000-image subset of ImageNet-1k validation: five images per class, selected with seed 20261031. The selection excludes the 500 calibration images and 100 earlier diagnostic images. All runs use the same input tensors, produced with the included Hugging Face PIL preprocessing configuration.

The original PyTorch FP32 checkpoint scored 79.68% top-1 and 95.40% top-5, with TF32 and autocast disabled. The TFLite results below were measured with AOT-compiled models fully delegated to each device's NPU; all outputs were finite. FP32 denotes the source model, with FP16 execution permitted by the NPU compiler.

Model file	SM-S937U1 top-1 (%)	SM-S937U1 top-5 (%)	LJ9 top-1 (%)	LJ9 top-5 (%)
model_fp32.tflite	79.92	95.40	79.68	95.38
model_a16w8.tflite	79.72	95.32	79.66	95.32
model_a8w8.tflite	75.14	92.50	75.10	92.76

A16W8 stays close to the PyTorch baseline on this subset. Mixed A8W8 reduces top-1 accuracy by 4.54 percentage points on SM-S937U1 and 4.58 points on LJ9 relative to that baseline.

This subset was reused while comparing quantization recipes. It is not an independent final test set or a full 50,000-image ImageNet validation run. Small score differences should not be interpreted as general improvements over the original checkpoint. Raw outputs, input images and diagnostic reports are not included in this package.

The A16W8 scores were collected before unused tensor records were removed. That cleanup preserved all live computation, tensor encodings and weights, and left the Qualcomm compiled file identical. A fresh 100-image MTK check reproduced the earlier logits exactly; the full 5,000-image MTK run was not repeated after this metadata cleanup.

NPU performance

These are representative measurements from retail devices to give a rough sense of on-device runtime behavior, not a direct comparison between hardware platforms. All latency numbers were collected with LiteRT's compiled-model benchmark_model launched through adb shell on the connected device; they are not app-level measurements from an integrated Android application.

Warm batch-one inference latency, rounded to the nearest millisecond. Each value is the median of three process means, with five warmup invocations and 30 measured invocations per process. All configurations are AOT compiled and fully delegated to the NPU, with no residual CPU operations.

Model file	SM-S937U1(ms)	LJ9(ms)
model_fp32.tflite	6	129
model_a16w8.tflite	4	71
model_a8w8.tflite	10	52

FP32 identifies the source model. Qualcomm compiles it to FP16-based NPU execution, and MediaTek enables FP32-to-FP16 relaxation. The floating-source timings therefore do not imply FP32 arithmetic throughout the NPU program.

Measurements use LiteRT's compiled-model benchmark with persistent buffers and deterministic random input. They exclude model initialization, image preprocessing, file I/O and explicit host input/output buffer transfers. Some variants were measured in separate sessions using the same benchmark and dispatch runtime for each device.

The SM-S937U1 ran Android 17 with Qualcomm QNN/QAIRT 2.50 in burst performance mode. The LJ9 ran Android 15 with the MediaTek Neuron runtime in sustained speed mode. Power settings were not normalized, and energy consumption was not measured. These results apply to the listed device/compiler/runtime combinations and do not characterize JIT performance.

Hardware benchmark disclosure: Results were measured by us on retail devices purchased through normal channels. These results are not affiliated with, sponsored by, endorsed by, or verified by Samsung, TECNO, vivo, Qualcomm, MediaTek, Google, MLCommons, or Hugging Face. Results depend on device SKU, OS build, thermal state, battery mode, backend, model quantization, runtime version, and benchmark settings.

Input and output

• Input: FP32 image tensor, NCHW [1, 3, 224, 224].
• Output: FP32 classifier logits, [1, 1000], ordered by labels.json.
• Preprocessing: convert to RGB, resize to 224 × 224 with the included Transformers PIL processor, rescale by 1/255, then normalize each channel with mean and standard deviation 0.5. The example selects backend="pil".

The models have static batch and spatial dimensions. All 25 LayerNorm operations use TFLite STABLEHLO_COMPOSITE boundaries named odml.group_norm, with FP32 decomposition subgraphs.

Run a local image

Download the listed files into one directory and run the following from that directory. The CPU example and dependencies were checked with Python 3.13 on Linux:

uv venv --python 3.13
source .venv/bin/activate
uv pip install -r requirements.txt

python run_image.py --image image.jpg
python run_image.py --image image.jpg --model model_a8w8.tflite
python run_image.py --image image.jpg --model model_a16w8.tflite

The helper loads the included processor configuration and TFLite model, then prints the top classes with their raw logits. A16W8 CPU execution may be slow; the example is an interface check for that variant.

For NPU deployment, compile the selected .tflite for the target SoC using the matching LiteRT compiler plugin, then load the compiled model with the matching device dispatch runtime and select the NPU accelerator. Verify full delegation on that device. The CPU example above does not select the NPU.

Source and attribution

This artifact derives from the pinned Google checkpoint, whose model card declares the Apache-2.0 license. Refer to that card for training details and intended use. The architecture was introduced in An Image Is Worth 16x16 Words. The accompanying config.json describes the original Transformers model; inference here uses the .tflite file.