File size: 4,351 Bytes
0aa9527 87d888c 0aa9527 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
library_name: onnx
license: other
tags:
- foundation
- amd
- rocm
- image-segmentation
pipeline_tag: image-segmentation
---
# SegFormer: Optimized for AMD ROCm
SegFormer is a semantic segmentation model that assigns a class label to every pixel across 19 Cityscapes categories. This repository packages inference for semantic segmentation using **ONNX Runtime**, exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs, CPUs, and NPUs.
This is based on the implementation of SegFormer found [here](https://github.com/NVlabs/SegFormer).
This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [segformer AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/segformer) to reproduce results or export with custom configurations. More details on model performance can be found [here](#accuracy-pipeline).
---
## Task Overview
**Task:** Semantic segmentation
**Dataset:** Cityscapes val (500 images, 19 classes)
**Output metrics:** mIoU (mean Intersection over Union)
> **Model:** SegFormer-B5 only at 1024×1024 — no `MODEL_SIZE` variants.
---
## AMD ROCm Optimization
This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**, as well as AMD CPUs and AMD Ryzen AI NPUs. Key points:
- Validated backends: **ONNX Runtime** across CPU (FP32/FP16/BF16/INT8), GPU (MIGraphX — FP32/FP16/BF16/INT8), and NPU (VitisAI — FP32/FP16/BF16/INT8).
- No code changes required versus the upstream SegFormer implementation — only environment/runtime configuration differs.
- NPU INT8 requires a separate Quark + Cityscapes calibration quantization step before evaluation.
| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| ONNX Runtime | FP32 / FP16 / BF16 / INT8 | CPU Execution Provider | AMD CPU | — |
| ONNX Runtime | FP32 / FP16 / BF16 / INT8 | MIGraphX Execution Provider | AMD Instinct™ / Radeon™ GPU (ROCm) | First run pays a 30+ minute graph-compilation cost |
| ONNX Runtime | FP32 / FP16 / BF16 / INT8 | VitisAI Execution Provider | AMD Ryzen AI NPU | INT8 requires Quark + Cityscapes calibration quantization step |
---
## Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the [segformer on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/segformer).
---
## Model Details
**Model Type:** Semantic segmentation (Transformer-based)
**Base Model:** SegFormer-B5, 1024×1024 input resolution
**Model Stats:**
- Input (`input`): `(1, 3, 1024, 1024)` float32
- Output (`logits`): `(1, 19, 1024, 1024)` float32
- Precision tested: FP32, FP16, BF16, INT8 (CPU/GPU/NPU)
---
## Accuracy Pipeline
Higher mIoU means predicted pixel labels agree more closely with ground truth across all 19 Cityscapes classes — 100% is perfect overlap, 0% is no agreement. Values above ~80% on Cityscapes val are considered strong for this task. Paper mIoU for SegFormer-B5 is 82.4%.
### Metrics Explained
| Metric | Description |
|--------|-------------|
| mIoU | Primary segmentation metric — mean Intersection-over-Union averaged across all 19 Cityscapes classes. Higher means better boundary alignment and class assignment across the full val set. |
### Accuracy Results
**Full Dataset Evaluation (Cityscapes val)** — SegFormer-B5:
<!-- accuracy-table-start -->
| Device | Precision | mIoU |
|--------|-----------|------|
| CPU | FP32 | 82.25% |
| GPU | FP32 | 82.25% |
| GPU | FP16 | 82.20% |
| GPU | BF16 | 82.26% |
| GPU | INT8 | 82.25% |
| NPU | FP32 | 82.25% |
| NPU | FP16 | 82.25% |
| NPU | BF16 | 82.23% |
| NPU | INT8 | 1.34% |
<!-- accuracy-table-end -->
---
## Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/segformer)**
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Cityscapes val dataset staging and mIoU evaluation pipeline via mmsegmentation
- NPU INT8 quantization workflow (AMD Quark + Cityscapes calibration)
- Benchmarking and reproduction instructions across CPU, GPU, and NPU
|