litert-community/vit_base_patch16_224.augreg_in1k

Vit Base Patch16 224

This repository contains LiteRT/TFLite exports of the TIMM image-classification model vit_base_patch16_224.augreg_in1k.

Model Description

The model files were converted from pretrained TIMM weights published at timm/vit_base_patch16_224.augreg_in1k.

Available Model Files

File	Description	Quantization
vit_base_patch16_224_fp32.tflite	Floating-point LiteRT/TFLite model.	Floating-point weights and activations.
vit_base_patch16_224_dynamic_wi8_afp32.tflite	Dynamic weight-quantized LiteRT/TFLite model.	INT8 weights with floating-point activations.
vit_base_patch16_224_int8_channelwise.tflite	Static INT8 LiteRT/TFLite model.	INT8 weights and INT8 activations, with channelwise weight quantization.

Quantization Schema

vit_base_patch16_224_int8_channelwise.tflite was quantized with AI Edge Quantizer's static W8A8 recipe (STATIC_WI8_AI8).

The schema is:

Tensor group	Quantization
Weights	INT8, symmetric, channelwise quantization.
Activations	INT8, asymmetric, tensorwise quantization.
Model input	INT8, tensorwise quantized NCHW image tensor with shape [1, 3, 224, 224].
Model output	INT8, tensorwise quantized logits tensor with shape [1, 1000].

Calibration used real ImageNet validation images with the TIMM preprocessing flow for vit_base_patch16_224.augreg_in1k. The resolved TIMM preprocessing config was {"crop_mode": "center", "crop_pct": 0.9, "input_size": [3, 224, 224], "interpolation": "bicubic", "mean": [0.5, 0.5, 0.5], "std": [0.5, 0.5, 0.5]}. When using APIs that expose raw tensor buffers, prepare the input and output using the quantization parameters stored in the model.

Runtime Compatibility

These artifacts are intended for LiteRT CPU and GPU execution.

The static INT8 channelwise artifact also AOT-compiled successfully through the LiteRT Qualcomm compiler plugin for SM8750, with the compiled model fully selected into Qualcomm NPU dispatch.

Enablement for other NPU backends is still under validation.

Intended Uses & Limitations

The model files were converted from pretrained weights from TIMM. The models may have their own licenses or terms and conditions derived from TIMM and the dataset used for training. It is your responsibility to determine whether you have permission to use the models for your use case.

Model Details

• Model Type: Image classification / feature backbone
• Model Stats:
  • Params (M): 86.6
  • GMACs: 16.9
  • Activations (M): 16.5
  • Image size: 224 x 224
• Papers:
  • How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers: https://arxiv.org/abs/2106.10270
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale: https://arxiv.org/abs/2010.11929v2
• Dataset: ImageNet-1k
• Original: https://github.com/google-research/vision_transformer

Citation

@article{steiner2021augreg,
  title={How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers},
  author={Steiner, Andreas and Kolesnikov, Alexander and and Zhai, Xiaohua and Wightman, Ross and Uszkoreit, Jakob and Beyer, Lucas},
  journal={arXiv preprint arXiv:2106.10270},
  year={2021}
}

@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}