Add optimized FP16 performance numbers (xOS-v4.34)
#1
by Vedang0201 - opened
- README.md +16 -2
- fp16/benchmarks/x5h_npu.yaml +14 -0
README.md
CHANGED
|
@@ -39,6 +39,15 @@ SigLIP is a dual-encoder model consisting of a vision encoder and a text encoder
|
|
| 39 |
|
| 40 |
## Performance
|
| 41 |
The following performance metrics were measured on the Renesas X5H board.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
| Precision | Device | VE Latency (ms) | TE Latency (ms) | NPU DDR (MB) |
|
| 43 |
|:---:|:---:|:---:|:---:|:---:|
|
| 44 |
| FP16 | NPX6 | 1229.9 | 46.1 | 1714.88 |
|
|
@@ -68,7 +77,7 @@ Both variants evaluated on 9 images (5 vandalism, 4 benign) x 10 text prompts:
|
|
| 68 |
## Prerequisites
|
| 69 |
To run the model, you need:
|
| 70 |
|
| 71 |
-
1. **Renesas X5H Board with SDK v4.32.0**
|
| 72 |
2. **Hugging Face CLI**: For downloading the model and installer.
|
| 73 |
|
| 74 |
## Board Setup
|
|
@@ -100,7 +109,12 @@ This loads kernel modules and starts all 14 NPU firmware cores.
|
|
| 100 |
## Deployment
|
| 101 |
|
| 102 |
### SigLIP-SO400M-patch14-384 (FP16)
|
| 103 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
2. Copy the installer to the X5H board and run the installer.
|
| 105 |
```bash
|
| 106 |
bash ./siglip-runner-3.0.0-Linux.sh --prefix=./ --exclude-subdir --skip-license
|
|
|
|
| 39 |
|
| 40 |
## Performance
|
| 41 |
The following performance metrics were measured on the Renesas X5H board.
|
| 42 |
+
|
| 43 |
+
### Optimized (xOS-v4.34)
|
| 44 |
+
TAPI kernel optimizations: L2/L1 tiling re-tuning, Q/K/V copy segment elimination, and pooling tiling. These optimizations reduce L3 DDR bandwidth without changing model accuracy.
|
| 45 |
+
|
| 46 |
+
| Precision | Device | VE Latency (ms) | TE Latency (ms) | NPU DDR (MB) |
|
| 47 |
+
|:---:|:---:|:---:|:---:|:---:|
|
| 48 |
+
| FP16 | NPX6 | 396.8 | 43.1 | 1708.04 |
|
| 49 |
+
|
| 50 |
+
### Unoptimized baseline (xOS-v4.32)
|
| 51 |
| Precision | Device | VE Latency (ms) | TE Latency (ms) | NPU DDR (MB) |
|
| 52 |
|:---:|:---:|:---:|:---:|:---:|
|
| 53 |
| FP16 | NPX6 | 1229.9 | 46.1 | 1714.88 |
|
|
|
|
| 77 |
## Prerequisites
|
| 78 |
To run the model, you need:
|
| 79 |
|
| 80 |
+
1. **Renesas X5H Board with SDK v4.32.0 or v4.34.0**
|
| 81 |
2. **Hugging Face CLI**: For downloading the model and installer.
|
| 82 |
|
| 83 |
## Board Setup
|
|
|
|
| 109 |
## Deployment
|
| 110 |
|
| 111 |
### SigLIP-SO400M-patch14-384 (FP16)
|
| 112 |
+
|
| 113 |
+
Two versions are available:
|
| 114 |
+
- **xOS-v4.34 (optimized)**: 3.1x faster VE encoding via TAPI kernel tiling optimizations
|
| 115 |
+
- **xOS-v4.32 (baseline)**: Original unoptimized build
|
| 116 |
+
|
| 117 |
+
1. Download the installer **siglip-runner-3.0.0-Linux.sh** from **Files and versions** tab under **fp16/binaries/rcar-x5hv1/xOS-v4.34/** (optimized) or **fp16/binaries/rcar-x5hv1/xOS-v4.32/** (baseline).
|
| 118 |
2. Copy the installer to the X5H board and run the installer.
|
| 119 |
```bash
|
| 120 |
bash ./siglip-runner-3.0.0-Linux.sh --prefix=./ --exclude-subdir --skip-license
|
fp16/benchmarks/x5h_npu.yaml
CHANGED
|
@@ -24,7 +24,14 @@ benchmark:
|
|
| 24 |
parameters:
|
| 25 |
batch_size: 1
|
| 26 |
|
|
|
|
| 27 |
performance:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
ve_encode_ms: 1229.9
|
| 29 |
te_encode_ms: 46.1
|
| 30 |
total_ms: 3298.6
|
|
@@ -32,7 +39,14 @@ performance:
|
|
| 32 |
metrics:
|
| 33 |
accuracy: null
|
| 34 |
|
|
|
|
| 35 |
memory:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
ve_npu_ddr_mb: 920.75
|
| 37 |
te_npu_ddr_mb: 794.12
|
| 38 |
total_npu_ddr_mb: 1714.88
|
|
|
|
| 24 |
parameters:
|
| 25 |
batch_size: 1
|
| 26 |
|
| 27 |
+
# Optimized (xOS-v4.34) — TAPI kernel tiling + segment elimination
|
| 28 |
performance:
|
| 29 |
+
ve_encode_ms: 396.8
|
| 30 |
+
te_encode_ms: 43.1
|
| 31 |
+
total_ms: 3311.7
|
| 32 |
+
|
| 33 |
+
# Unoptimized baseline (xOS-v4.32)
|
| 34 |
+
performance_baseline:
|
| 35 |
ve_encode_ms: 1229.9
|
| 36 |
te_encode_ms: 46.1
|
| 37 |
total_ms: 3298.6
|
|
|
|
| 39 |
metrics:
|
| 40 |
accuracy: null
|
| 41 |
|
| 42 |
+
# Optimized (xOS-v4.34)
|
| 43 |
memory:
|
| 44 |
+
ve_npu_ddr_mb: 915.30
|
| 45 |
+
te_npu_ddr_mb: 792.74
|
| 46 |
+
total_npu_ddr_mb: 1708.04
|
| 47 |
+
|
| 48 |
+
# Unoptimized baseline (xOS-v4.32)
|
| 49 |
+
memory_baseline:
|
| 50 |
ve_npu_ddr_mb: 920.75
|
| 51 |
te_npu_ddr_mb: 794.12
|
| 52 |
total_npu_ddr_mb: 1714.88
|