nielsr/vit-base-patch16-224-r3al-int8

ViT Base Patch16 224 — R3AL Dynamic INT8 ONNX

Dynamic INT8 ONNX quantization of google/vit-base-patch16-224, produced with R3AL.AI r3alai 2.4.1 using ptq_dynamic.

The source model is a ViT image classifier fine-tuned on ImageNet-1k. Dynamic quantization targets MatMul/Gemm weights while preserving attention-sensitive operations such as Softmax, LayerNormalization, GELU/Erf, and adjacent attention MatMuls. See r3alai_manifest.json for the complete configuration and excluded-node list.

Results

Metric	FP32 ONNX	INT8 ONNX	Change
File size	346,397,372 bytes	89,432,268 bytes	74.18% smaller (3.87x)
Mean CPU latency	92.51 ms	37.05 ms	2.50x faster
Top-1 accuracy, 100 images	83%	81%	-2 points
Top-5 accuracy, 100 images	95%	95%	no change
Top-1 agreement	—	97%	—
Mean logit cosine similarity	—	0.9958	—

Latency was measured with ONNX Runtime CPUExecutionProvider on Apple arm64, using 5 warm-up and 30 measured runs at batch size 1. Quality was checked on the first 100 held-out ImageNet-1k validation samples with RGB resize to 224x224 and normalization using mean/std 0.5.

The 100-image measurement is a regression check, not a publication-grade full ImageNet evaluation. Results can vary by hardware, runtime, and preprocessing.

Usage

from optimum.onnxruntime import ORTModelForImageClassification
from transformers import AutoImageProcessor

repo_id = "nielsr/vit-base-patch16-224-r3al-int8"
processor = AutoImageProcessor.from_pretrained(repo_id)
model = ORTModelForImageClassification.from_pretrained(repo_id)

inputs = processor(images=image, return_tensors="pt")
logits = model(**inputs).logits
predicted_class_id = logits.argmax(-1).item()
print(model.config.id2label[predicted_class_id])

Install the runtime dependencies with:

uv pip install optimum-onnx onnxruntime transformers pillow

Files

• model.onnx: dynamic INT8 ONNX model
• config.json: ViT/ImageNet class configuration
• preprocessor_config.json: image resize and normalization settings
• r3alai_manifest.json: R3AL quantization provenance and configuration

Reproducibility

The sanitized Codex session trace covering installation, export, quantization, benchmarking, and quality evaluation is available in nielsr/r3al-vit-quantization-codex-trace.

The original model and this derived quantization use the Apache 2.0 license.