---
library_name: pytorch
license: apache-2.0
tags:
  - foundation
  - amd
  - rocm
  - zero-shot-image-classification
pipeline_tag: zero-shot-image-classification
---

![](https://huggingface.co/AMD-PAVS-AI/siglip/resolve/main/siglip.jpg)

# SigLIP: Optimized for AMD ROCm

SigLIP is a vision-language embedding model evaluated here for zero-shot image classification on CIFAR-10. This repository packages inference for zero-shot image classification using **PyTorch** (CPU/GPU) and **vLLM** (GPU pooling runner), exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs and CPUs.

This is based on the implementation of SigLIP found [here](https://huggingface.co/google/siglip-base-patch16-224).
This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [siglip AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/siglip) to reproduce results or export with custom configurations. More details on model performance can be found [here](#accuracy-pipeline).

---

## Task Overview

**Task:** Zero-shot image classification

**Dataset:** CIFAR-10 test split (10,000 images, 10 classes)

**Output metrics:** Accuracy (%), image throughput (img/s)

> **Model variants:** Default is **base-224** (`google/siglip-base-patch16-224`). Override with `MODEL_VARIANT=base-256|base-384|large-256|large-384|so400m-224|so400m-384`.

> **vLLM note:** vLLM's SigLIP integration handles one modality per request (text batch, then one image per request); the client combines embeddings manually — the same approach as the upstream evaluation scripts.

---

## AMD ROCm Optimization

This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**, as well as AMD CPUs. Key points:

- Validated backends: **PyTorch** (native ROCm HIP kernels, CPU and GPU) and **vLLM** (ROCm-enabled, built from source, pooling runner on GPU).
- No code changes required versus the upstream SigLIP implementation — only environment/runtime configuration differs.
- CPU fallback path supported via PyTorch for environments without a ROCm-capable GPU.

| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| PyTorch | FP32 / FP16 / BF16 | HIP (ROCm) | AMD CPU | — |
| PyTorch | FP32 / FP16 / BF16 | HIP (ROCm) | AMD Instinct™ / Radeon™ GPU | — |
| vLLM | FP32 / FP16 / BF16 | ROCm server (pooling runner) | AMD Instinct™ / Radeon™ GPU | One modality per request; CPU client scores embeddings |

---

## Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the [siglip on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/siglip).

---

## Model Details

**Model Type:** Vision-language embedding model (zero-shot image classification)

**Base Model:** `google/siglip-base-patch16-224`

**Model Stats:**
- Model variant: base-224 (default) — base-256 / base-384 / large-256 / large-384 / so400m-224 / so400m-384 also supported
- Precision tested: FP32, FP16, BF16 (CPU, GPU PyTorch, GPU vLLM)

---

## Accuracy Pipeline

Higher accuracy means a larger fraction of images are assigned the correct CIFAR-10 label — 100% is perfect, 10% is random chance for this 10-way task. Values above ~90% on CIFAR-10 with SigLIP-base are typical; larger variants (large, so400m) often reach ~96–97%.

### Metrics Explained

| Metric | Description |
|--------|-------------|
| Accuracy (%) | Fraction of test images where the highest-scoring class prompt matches the ground-truth label. Primary quality metric; FP16/BF16 should track FP32 within a few tenths of a percent. |
| Image throughput (img/s) | Images encoded and scored per second during the timed loop (text embeddings computed once upfront and excluded from the timer). Higher is faster; PyTorch-direct GPU is typically faster than per-image vLLM HTTP calls. |

### Accuracy Results

**Full Dataset Evaluation (CIFAR-10 test)** — siglip-base-patch16-224:

<!-- accuracy-table-start -->
| Device | Backend | Precision | Accuracy (%) | Throughput (img/s) |
|--------|---------|-----------|--------------|--------------------|
| CPU | PyTorch | BF16 | 92.57 | 40.84 |
| CPU | PyTorch | FP16 | 92.55 | 5.19 |
| CPU | PyTorch | FP32 | 92.54 | 18.35 |
| GPU | PyTorch | BF16 | 92.59 | 100.16 |
| GPU | PyTorch | FP16 | 92.56 | 100.75 |
| GPU | PyTorch | FP32 | 92.54 | 35.04 |
| GPU | vLLM | BF16 | 92.54 | 77.91 |
| GPU | vLLM | FP16 | 92.55 | 75.78 |
| GPU | vLLM | FP32 | 92.52 | 28.12 |
<!-- accuracy-table-end -->

---

## Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/siglip)**

The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for both PyTorch and vLLM runners
- Additional model variants and datasets
- Benchmarking and reproduction instructions