Artem Plastinkin commited on
Commit ·
b185d98
1
Parent(s): 3de21a3
Update performance numbers
Browse files- int8/benchmarks/x5h_mwmx_npu_apm80_12core.yaml +1 -1
- int8/benchmarks/x5h_mwmx_npu_apm80_1core.yaml +1 -1
- int8/benchmarks/x5h_mwmx_npu_apm80_1core_2026-09-16.yaml +1 -1
- int8/benchmarks/x5h_mwmx_npu_apm80_3core.yaml +72 -0
- int8/benchmarks/x5h_mwmx_npu_apm80_4core.yaml +72 -0
- int8/benchmarks/x5h_mwmx_npu_apm80_6core.yaml +72 -0
int8/benchmarks/x5h_mwmx_npu_apm80_12core.yaml
CHANGED
|
@@ -29,7 +29,7 @@ benchmark:
|
|
| 29 |
|
| 30 |
performance:
|
| 31 |
fps: null
|
| 32 |
-
latency: 3.467288
|
| 33 |
|
| 34 |
metrics:
|
| 35 |
accuracy: null
|
|
|
|
| 29 |
|
| 30 |
performance:
|
| 31 |
fps: null
|
| 32 |
+
latency: 13.77 # synced to mwmx2.2 model_list.html (2026-09-22); was 3.467288 (APM80 pipeline run, metawaremx_runtime CI)
|
| 33 |
|
| 34 |
metrics:
|
| 35 |
accuracy: null
|
int8/benchmarks/x5h_mwmx_npu_apm80_1core.yaml
CHANGED
|
@@ -29,7 +29,7 @@ benchmark:
|
|
| 29 |
|
| 30 |
performance:
|
| 31 |
fps: null
|
| 32 |
-
latency: 11.978266
|
| 33 |
|
| 34 |
metrics:
|
| 35 |
accuracy: null
|
|
|
|
| 29 |
|
| 30 |
performance:
|
| 31 |
fps: null
|
| 32 |
+
latency: 15.86 # synced to mwmx2.2 model_list.html (2026-09-22); was 11.978266 (APM80 pipeline run, metawaremx_runtime CI)
|
| 33 |
|
| 34 |
metrics:
|
| 35 |
accuracy: null
|
int8/benchmarks/x5h_mwmx_npu_apm80_1core_2026-09-16.yaml
CHANGED
|
@@ -30,7 +30,7 @@ benchmark:
|
|
| 30 |
|
| 31 |
performance:
|
| 32 |
fps: null
|
| 33 |
-
latency: 11.989098
|
| 34 |
|
| 35 |
metrics:
|
| 36 |
accuracy: null
|
|
|
|
| 30 |
|
| 31 |
performance:
|
| 32 |
fps: null
|
| 33 |
+
latency: 15.86 # synced to mwmx2.2 model_list.html (2026-09-22); was 11.989098 (APM80 pipeline run, metawaremx_runtime CI)
|
| 34 |
|
| 35 |
metrics:
|
| 36 |
accuracy: null
|
int8/benchmarks/x5h_mwmx_npu_apm80_3core.yaml
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
hardware:
|
| 2 |
+
vendor: renesas
|
| 3 |
+
chip: rcar-x5h
|
| 4 |
+
cpu: arm-cortex-a720
|
| 5 |
+
npu: npx6-48k
|
| 6 |
+
npu_count: 2
|
| 7 |
+
npu_cores: 12
|
| 8 |
+
npu_default_freq_mhz: 1066
|
| 9 |
+
accelerator:
|
| 10 |
+
- npu
|
| 11 |
+
|
| 12 |
+
runtime:
|
| 13 |
+
engine: mwmx
|
| 14 |
+
toolchain_version: "MWMX SDK v4.35.0"
|
| 15 |
+
format: onnx
|
| 16 |
+
execution_provider: npu
|
| 17 |
+
execution_precision: int8
|
| 18 |
+
|
| 19 |
+
configuration:
|
| 20 |
+
npu_instances: 1
|
| 21 |
+
npu_cores_per_instance: 3
|
| 22 |
+
npu_freq_mhz: 850
|
| 23 |
+
|
| 24 |
+
benchmark:
|
| 25 |
+
type: hil
|
| 26 |
+
parameters:
|
| 27 |
+
batch_size: 1
|
| 28 |
+
input_resolution: [1, 3, 512, 512] # inferred from "crop512" in the source checkpoint name
|
| 29 |
+
|
| 30 |
+
performance:
|
| 31 |
+
fps: null
|
| 32 |
+
latency: 16.25 # synced to mwmx2.2 model_list.html (2026-09-22); no prior local measurement for this core count
|
| 33 |
+
|
| 34 |
+
metrics:
|
| 35 |
+
accuracy: null
|
| 36 |
+
top5_accuracy: null
|
| 37 |
+
|
| 38 |
+
memory:
|
| 39 |
+
peak_mb: null
|
| 40 |
+
|
| 41 |
+
power:
|
| 42 |
+
avg_w: null
|
| 43 |
+
|
| 44 |
+
# UNCONFIRMED (segmentation config): this repo's segmentation_json compile_config
|
| 45 |
+
# (compile_config/l2seg_csm_16mb.json, see the 12-core reproduce block) is specific
|
| 46 |
+
# to the 12-core build; no 3-core segmentation config is bundled, so the steps
|
| 47 |
+
# below compile without one — a best-effort single-cluster flow that may not
|
| 48 |
+
# reproduce the exact benchmarked 3-core latency.
|
| 49 |
+
reproduce:
|
| 50 |
+
steps:
|
| 51 |
+
- title: Activate the Python environment
|
| 52 |
+
command: >-
|
| 53 |
+
Activate the Python virtual environment that has the `hf` CLI
|
| 54 |
+
(huggingface_hub) and the NNAC toolchain installed, e.g.
|
| 55 |
+
`source nnac_venv/bin/activate` -- path depends on your toolchain install.
|
| 56 |
+
kind: note
|
| 57 |
+
- title: Download the ONNX model
|
| 58 |
+
command: hf download Renesas/CenterNet-R18-ONNX --repo-type=model --include "fp32/*" --local-dir ./CenterNet-R18-ONNX-fp32
|
| 59 |
+
- title: Compile with the NNAC toolchain (INT8 auto-cast from the FP32 graph)
|
| 60 |
+
command: |
|
| 61 |
+
python3 nnac_frontend/legalize.py -d binary/nnx ./CenterNet-R18-ONNX-fp32/fp32/centernet_r18_8xb16_crop512_140e_coco.onnx --num-core 3
|
| 62 |
+
- title: Set up the R-Car X5H board
|
| 63 |
+
command: Configure the board per the AI Compiler (NNAC) "Getting Started" guide, section 3.4 (host TFTP/NFS setup, bootloader flashing, U-Boot, Linux boot, login) -- exact steps depend on your board/network setup.
|
| 64 |
+
kind: note
|
| 65 |
+
- title: Copy the compiled artifact to the board
|
| 66 |
+
command: Copy ${WORKDIR}/binary/nnx (the working directory from the download/compile steps above) to the board -- method may vary (NFS mount, scp, USB, etc.).
|
| 67 |
+
kind: note
|
| 68 |
+
- title: Run on R-Car X5H (single NPU cluster, 3 AI cores)
|
| 69 |
+
command: |
|
| 70 |
+
cd binary
|
| 71 |
+
./host_app ./arc_prog_npus ./nnx/centernet_r18_8xb16_crop512_140e_coco
|
| 72 |
+
expected: hash[n] = 0x...(OK) means the run's output matches the reference hash in hash.txt; latency is the NPX execution time reported in cycles and ms.
|
int8/benchmarks/x5h_mwmx_npu_apm80_4core.yaml
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
hardware:
|
| 2 |
+
vendor: renesas
|
| 3 |
+
chip: rcar-x5h
|
| 4 |
+
cpu: arm-cortex-a720
|
| 5 |
+
npu: npx6-48k
|
| 6 |
+
npu_count: 2
|
| 7 |
+
npu_cores: 12
|
| 8 |
+
npu_default_freq_mhz: 1066
|
| 9 |
+
accelerator:
|
| 10 |
+
- npu
|
| 11 |
+
|
| 12 |
+
runtime:
|
| 13 |
+
engine: mwmx
|
| 14 |
+
toolchain_version: "MWMX SDK v4.35.0"
|
| 15 |
+
format: onnx
|
| 16 |
+
execution_provider: npu
|
| 17 |
+
execution_precision: int8
|
| 18 |
+
|
| 19 |
+
configuration:
|
| 20 |
+
npu_instances: 1
|
| 21 |
+
npu_cores_per_instance: 4
|
| 22 |
+
npu_freq_mhz: 850
|
| 23 |
+
|
| 24 |
+
benchmark:
|
| 25 |
+
type: hil
|
| 26 |
+
parameters:
|
| 27 |
+
batch_size: 1
|
| 28 |
+
input_resolution: [1, 3, 512, 512] # inferred from "crop512" in the source checkpoint name
|
| 29 |
+
|
| 30 |
+
performance:
|
| 31 |
+
fps: null
|
| 32 |
+
latency: 14.94 # synced to mwmx2.2 model_list.html (2026-09-22); no prior local measurement for this core count
|
| 33 |
+
|
| 34 |
+
metrics:
|
| 35 |
+
accuracy: null
|
| 36 |
+
top5_accuracy: null
|
| 37 |
+
|
| 38 |
+
memory:
|
| 39 |
+
peak_mb: null
|
| 40 |
+
|
| 41 |
+
power:
|
| 42 |
+
avg_w: null
|
| 43 |
+
|
| 44 |
+
# UNCONFIRMED (segmentation config): this repo's segmentation_json compile_config
|
| 45 |
+
# (compile_config/l2seg_csm_16mb.json, see the 12-core reproduce block) is specific
|
| 46 |
+
# to the 12-core build; no 4-core segmentation config is bundled, so the steps
|
| 47 |
+
# below compile without one — a best-effort single-cluster flow that may not
|
| 48 |
+
# reproduce the exact benchmarked 4-core latency.
|
| 49 |
+
reproduce:
|
| 50 |
+
steps:
|
| 51 |
+
- title: Activate the Python environment
|
| 52 |
+
command: >-
|
| 53 |
+
Activate the Python virtual environment that has the `hf` CLI
|
| 54 |
+
(huggingface_hub) and the NNAC toolchain installed, e.g.
|
| 55 |
+
`source nnac_venv/bin/activate` -- path depends on your toolchain install.
|
| 56 |
+
kind: note
|
| 57 |
+
- title: Download the ONNX model
|
| 58 |
+
command: hf download Renesas/CenterNet-R18-ONNX --repo-type=model --include "fp32/*" --local-dir ./CenterNet-R18-ONNX-fp32
|
| 59 |
+
- title: Compile with the NNAC toolchain (INT8 auto-cast from the FP32 graph)
|
| 60 |
+
command: |
|
| 61 |
+
python3 nnac_frontend/legalize.py -d binary/nnx ./CenterNet-R18-ONNX-fp32/fp32/centernet_r18_8xb16_crop512_140e_coco.onnx --num-core 4
|
| 62 |
+
- title: Set up the R-Car X5H board
|
| 63 |
+
command: Configure the board per the AI Compiler (NNAC) "Getting Started" guide, section 3.4 (host TFTP/NFS setup, bootloader flashing, U-Boot, Linux boot, login) -- exact steps depend on your board/network setup.
|
| 64 |
+
kind: note
|
| 65 |
+
- title: Copy the compiled artifact to the board
|
| 66 |
+
command: Copy ${WORKDIR}/binary/nnx (the working directory from the download/compile steps above) to the board -- method may vary (NFS mount, scp, USB, etc.).
|
| 67 |
+
kind: note
|
| 68 |
+
- title: Run on R-Car X5H (single NPU cluster, 4 AI cores)
|
| 69 |
+
command: |
|
| 70 |
+
cd binary
|
| 71 |
+
./host_app ./arc_prog_npus ./nnx/centernet_r18_8xb16_crop512_140e_coco
|
| 72 |
+
expected: hash[n] = 0x...(OK) means the run's output matches the reference hash in hash.txt; latency is the NPX execution time reported in cycles and ms.
|
int8/benchmarks/x5h_mwmx_npu_apm80_6core.yaml
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
hardware:
|
| 2 |
+
vendor: renesas
|
| 3 |
+
chip: rcar-x5h
|
| 4 |
+
cpu: arm-cortex-a720
|
| 5 |
+
npu: npx6-48k
|
| 6 |
+
npu_count: 2
|
| 7 |
+
npu_cores: 12
|
| 8 |
+
npu_default_freq_mhz: 1066
|
| 9 |
+
accelerator:
|
| 10 |
+
- npu
|
| 11 |
+
|
| 12 |
+
runtime:
|
| 13 |
+
engine: mwmx
|
| 14 |
+
toolchain_version: "MWMX SDK v4.35.0"
|
| 15 |
+
format: onnx
|
| 16 |
+
execution_provider: npu
|
| 17 |
+
execution_precision: int8
|
| 18 |
+
|
| 19 |
+
configuration:
|
| 20 |
+
npu_instances: 1
|
| 21 |
+
npu_cores_per_instance: 6
|
| 22 |
+
npu_freq_mhz: 850
|
| 23 |
+
|
| 24 |
+
benchmark:
|
| 25 |
+
type: hil
|
| 26 |
+
parameters:
|
| 27 |
+
batch_size: 1
|
| 28 |
+
input_resolution: [1, 3, 512, 512] # inferred from "crop512" in the source checkpoint name
|
| 29 |
+
|
| 30 |
+
performance:
|
| 31 |
+
fps: null
|
| 32 |
+
latency: 14.56 # synced to mwmx2.2 model_list.html (2026-09-22); no prior local measurement for this core count
|
| 33 |
+
|
| 34 |
+
metrics:
|
| 35 |
+
accuracy: null
|
| 36 |
+
top5_accuracy: null
|
| 37 |
+
|
| 38 |
+
memory:
|
| 39 |
+
peak_mb: null
|
| 40 |
+
|
| 41 |
+
power:
|
| 42 |
+
avg_w: null
|
| 43 |
+
|
| 44 |
+
# UNCONFIRMED (segmentation config): this repo's segmentation_json compile_config
|
| 45 |
+
# (compile_config/l2seg_csm_16mb.json, see the 12-core reproduce block) is specific
|
| 46 |
+
# to the 12-core build; no 6-core segmentation config is bundled, so the steps
|
| 47 |
+
# below compile without one — a best-effort single-cluster flow that may not
|
| 48 |
+
# reproduce the exact benchmarked 6-core latency.
|
| 49 |
+
reproduce:
|
| 50 |
+
steps:
|
| 51 |
+
- title: Activate the Python environment
|
| 52 |
+
command: >-
|
| 53 |
+
Activate the Python virtual environment that has the `hf` CLI
|
| 54 |
+
(huggingface_hub) and the NNAC toolchain installed, e.g.
|
| 55 |
+
`source nnac_venv/bin/activate` -- path depends on your toolchain install.
|
| 56 |
+
kind: note
|
| 57 |
+
- title: Download the ONNX model
|
| 58 |
+
command: hf download Renesas/CenterNet-R18-ONNX --repo-type=model --include "fp32/*" --local-dir ./CenterNet-R18-ONNX-fp32
|
| 59 |
+
- title: Compile with the NNAC toolchain (INT8 auto-cast from the FP32 graph)
|
| 60 |
+
command: |
|
| 61 |
+
python3 nnac_frontend/legalize.py -d binary/nnx ./CenterNet-R18-ONNX-fp32/fp32/centernet_r18_8xb16_crop512_140e_coco.onnx --num-core 6
|
| 62 |
+
- title: Set up the R-Car X5H board
|
| 63 |
+
command: Configure the board per the AI Compiler (NNAC) "Getting Started" guide, section 3.4 (host TFTP/NFS setup, bootloader flashing, U-Boot, Linux boot, login) -- exact steps depend on your board/network setup.
|
| 64 |
+
kind: note
|
| 65 |
+
- title: Copy the compiled artifact to the board
|
| 66 |
+
command: Copy ${WORKDIR}/binary/nnx (the working directory from the download/compile steps above) to the board -- method may vary (NFS mount, scp, USB, etc.).
|
| 67 |
+
kind: note
|
| 68 |
+
- title: Run on R-Car X5H (single NPU cluster, 6 AI cores)
|
| 69 |
+
command: |
|
| 70 |
+
cd binary
|
| 71 |
+
./host_app ./arc_prog_npus ./nnx/centernet_r18_8xb16_crop512_140e_coco
|
| 72 |
+
expected: hash[n] = 0x...(OK) means the run's output matches the reference hash in hash.txt; latency is the NPX execution time reported in cycles and ms.
|