| --- |
| library_name: onnx |
| license: other |
| tags: |
| - foundation |
| - amd |
| - rocm |
| - image-segmentation |
| pipeline_tag: image-segmentation |
| --- |
| |
|  |
|
|
| # SegFormer: Optimized for AMD ROCm |
|
|
| SegFormer is a semantic segmentation model that assigns a class label to every pixel across 19 Cityscapes categories. This repository packages inference for semantic segmentation using **ONNX Runtime**, exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs, CPUs, and NPUs. |
|
|
| This is based on the implementation of SegFormer found [here](https://github.com/NVlabs/SegFormer). |
| This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [segformer AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/segformer) to reproduce results or export with custom configurations. More details on model performance can be found [here](#accuracy-pipeline). |
|
|
| --- |
|
|
| ## Task Overview |
|
|
| **Task:** Semantic segmentation |
|
|
| **Dataset:** Cityscapes val (500 images, 19 classes) |
|
|
| **Output metrics:** mIoU (mean Intersection over Union) |
|
|
| > **Model:** SegFormer-B5 only at 1024×1024 — no `MODEL_SIZE` variants. |
| |
| --- |
| |
| ## AMD ROCm Optimization |
| |
| This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**, as well as AMD CPUs and AMD Ryzen AI NPUs. Key points: |
| |
| - Validated backends: **ONNX Runtime** across CPU (FP32/FP16/BF16/INT8), GPU (MIGraphX — FP32/FP16/BF16/INT8), and NPU (VitisAI — FP32/FP16/BF16/INT8). |
| - No code changes required versus the upstream SegFormer implementation — only environment/runtime configuration differs. |
| - NPU INT8 requires a separate Quark + Cityscapes calibration quantization step before evaluation. |
| |
| | Runtime | Precision | Backend | Hardware | Notes | |
| |---|---|---|---|---| |
| | ONNX Runtime | FP32 / FP16 / BF16 / INT8 | CPU Execution Provider | AMD CPU | — | |
| | ONNX Runtime | FP32 / FP16 / BF16 / INT8 | MIGraphX Execution Provider | AMD Instinct™ / Radeon™ GPU (ROCm) | First run pays a 30+ minute graph-compilation cost | |
| | ONNX Runtime | FP32 / FP16 / BF16 / INT8 | VitisAI Execution Provider | AMD Ryzen AI NPU | INT8 requires Quark + Cityscapes calibration quantization step | |
| |
| --- |
| |
| ## Getting Started |
| |
| For setup instructions, evaluation scripts, and custom configuration options, see the [segformer on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/segformer). |
| |
| --- |
| |
| ## Model Details |
| |
| **Model Type:** Semantic segmentation (Transformer-based) |
| |
| **Base Model:** SegFormer-B5, 1024×1024 input resolution |
| |
| **Model Stats:** |
| - Input (`input`): `(1, 3, 1024, 1024)` float32 |
| - Output (`logits`): `(1, 19, 1024, 1024)` float32 |
| - Precision tested: FP32, FP16, BF16, INT8 (CPU/GPU/NPU) |
| |
| --- |
| |
| ## Accuracy Pipeline |
| |
| Higher mIoU means predicted pixel labels agree more closely with ground truth across all 19 Cityscapes classes — 100% is perfect overlap, 0% is no agreement. Values above ~80% on Cityscapes val are considered strong for this task. Paper mIoU for SegFormer-B5 is 82.4%. |
| |
| ### Metrics Explained |
| |
| | Metric | Description | |
| |--------|-------------| |
| | mIoU | Primary segmentation metric — mean Intersection-over-Union averaged across all 19 Cityscapes classes. Higher means better boundary alignment and class assignment across the full val set. | |
| |
| ### Accuracy Results |
| |
| **Full Dataset Evaluation (Cityscapes val)** — SegFormer-B5: |
| |
| <!-- accuracy-table-start --> |
| | Device | Precision | mIoU | |
| |--------|-----------|------| |
| | CPU | FP32 | 82.25% | |
| | GPU | FP32 | 82.25% | |
| | GPU | FP16 | 82.20% | |
| | GPU | BF16 | 82.26% | |
| | GPU | INT8 | 82.25% | |
| | NPU | FP32 | 82.25% | |
| | NPU | FP16 | 82.25% | |
| | NPU | BF16 | 82.23% | |
| | NPU | INT8 | 1.34% | |
| <!-- accuracy-table-end --> |
| |
| --- |
| |
| ## Dig Deeper |
| |
| Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples? |
| |
| 📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/segformer)** |
| |
| The GitHub repository includes: |
| - Setup and prerequisites for ROCm environments |
| - Cityscapes val dataset staging and mIoU evaluation pipeline via mmsegmentation |
| - NPU INT8 quantization workflow (AMD Quark + Cityscapes calibration) |
| - Benchmarking and reproduction instructions across CPU, GPU, and NPU |
| |
