library_name: pytorch
license: apache-2.0
tags:
- foundation
- amd
- rocm
- zero-shot-image-classification
pipeline_tag: zero-shot-image-classification
SigLIP: Optimized for AMD ROCm
SigLIP is a vision-language embedding model evaluated here for zero-shot image classification on CIFAR-10. This repository packages inference for zero-shot image classification using PyTorch (CPU/GPU) and vLLM (GPU pooling runner), exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.
This is based on the implementation of SigLIP found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the siglip AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.
Task Overview
Task: Zero-shot image classification
Dataset: CIFAR-10 test split (10,000 images, 10 classes)
Output metrics: Accuracy (%), image throughput (img/s)
Model variants: Default is base-224 (
google/siglip-base-patch16-224). Override withMODEL_VARIANT=base-256|base-384|large-256|large-384|so400m-224|so400m-384.
vLLM note: vLLM's SigLIP integration handles one modality per request (text batch, then one image per request); the client combines embeddings manually — the same approach as the upstream evaluation scripts.
AMD ROCm Optimization
This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs. Key points:
- Validated backends: PyTorch (native ROCm HIP kernels, CPU and GPU) and vLLM (ROCm-enabled, built from source, pooling runner on GPU).
- No code changes required versus the upstream SigLIP implementation — only environment/runtime configuration differs.
- CPU fallback path supported via PyTorch for environments without a ROCm-capable GPU.
| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| PyTorch | FP32 / FP16 / BF16 | HIP (ROCm) | AMD CPU | — |
| PyTorch | FP32 / FP16 / BF16 | HIP (ROCm) | AMD Instinct™ / Radeon™ GPU | — |
| vLLM | FP32 / FP16 / BF16 | ROCm server (pooling runner) | AMD Instinct™ / Radeon™ GPU | One modality per request; CPU client scores embeddings |
Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the siglip on GitHub.
Model Details
Model Type: Vision-language embedding model (zero-shot image classification)
Base Model: google/siglip-base-patch16-224
Model Stats:
- Model variant: base-224 (default) — base-256 / base-384 / large-256 / large-384 / so400m-224 / so400m-384 also supported
- Precision tested: FP32, FP16, BF16 (CPU, GPU PyTorch, GPU vLLM)
Accuracy Pipeline
Higher accuracy means a larger fraction of images are assigned the correct CIFAR-10 label — 100% is perfect, 10% is random chance for this 10-way task. Values above ~90% on CIFAR-10 with SigLIP-base are typical; larger variants (large, so400m) often reach ~96–97%.
Metrics Explained
| Metric | Description |
|---|---|
| Accuracy (%) | Fraction of test images where the highest-scoring class prompt matches the ground-truth label. Primary quality metric; FP16/BF16 should track FP32 within a few tenths of a percent. |
| Image throughput (img/s) | Images encoded and scored per second during the timed loop (text embeddings computed once upfront and excluded from the timer). Higher is faster; PyTorch-direct GPU is typically faster than per-image vLLM HTTP calls. |
Accuracy Results
Full Dataset Evaluation (CIFAR-10 test) — siglip-base-patch16-224:
| Device | Backend | Precision | Accuracy (%) | Throughput (img/s) |
|---|---|---|---|---|
| CPU | PyTorch | BF16 | 92.57 | 40.84 |
| CPU | PyTorch | FP16 | 92.55 | 5.19 |
| CPU | PyTorch | FP32 | 92.54 | 18.35 |
| GPU | PyTorch | BF16 | 92.59 | 100.16 |
| GPU | PyTorch | FP16 | 92.56 | 100.75 |
| GPU | PyTorch | FP32 | 92.54 | 35.04 |
| GPU | vLLM | BF16 | 92.54 | 77.91 |
| GPU | vLLM | FP16 | 92.55 | 75.78 |
| GPU | vLLM | FP32 | 92.52 | 28.12 |
Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 View the full project on GitHub
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for both PyTorch and vLLM runners
- Additional model variants and datasets
- Benchmarking and reproduction instructions