ONNX
onnxruntime
onnx-mlir
quantization
fp32
ONNX_Models / reports /README.md
purejomo's picture
Finalize public ONNX/ONNX-MLIR validation release
ed3aeeb
|
Raw
History Blame Contribute Delete
3 kB
# ๊ฒฐ๊ณผ
| ๊ฒฝ๋กœ | ๋‚ด์šฉ |
|---|---|
| `accuracy/model_accuracy.csv` | 21๊ฐœ ๋ชจ๋ธ์˜ FP32ยท์–‘์žํ™” ์ •ํ™•๋„์™€ ๋ณ€ํ™”๋Ÿ‰ |
| `accuracy/onnx_mlir_output_check.csv` | ์ƒ์„ฑ๋œ native ONNX-MLIR artifact์˜ ์ถœ๋ ฅ ๋น„๊ต |
| `conversion/pipeline_status.csv` | ์›๋ณธ, ONNX, ONNX-MLIR ๋ณ€ํ™˜ ์ƒํƒœ |
| `conversion/ir_stage_coverage.csv` | ONNX Dialect, Krnl, Affine/SCF/MemRef, LLVM ๋‹จ๊ณ„๋ณ„ ์ƒํƒœ |
| `conversion/quantization_preservation.csv` | ONNX์™€ MLIR์˜ ์–‘์žํ™” ์ •๋ณด ๋ณด์กด ์ƒํƒœ |
| `graphs/netron/` | Netron FP32ยท์–‘์žํ™” ONNX ๊ทธ๋ž˜ํ”„ ์ƒ์„ฑยท์ €์žฅ ์œ„์น˜ |
| `graphs/mlir/` | ONNX Dialect 42๊ฐœ์˜ ์ •์  operation ์ˆœ์„œ์™€ SSA ์—ฐ๊ฒฐ ๊ทธ๋ž˜ํ”„ |
## ๋ชจ๋ธ ์ •ํ™•๋„ ๊ฒฐ๊ณผ
์ „์ฒด CSV: [`accuracy/model_accuracy.csv`](accuracy/model_accuracy.csv)
| ID | ์ •ํ™•๋„ ์ง€ํ‘œ | FP32 | ์–‘์žํ™” | ๋ณ€ํ™”๋Ÿ‰ | ๊ณต๊ฐœ ์ˆ˜์น˜ ๋น„๊ต |
|---|---|---:|---:|---:|---|
| AD01 | AUC / pAUC | 0.876001 / 0.764121 | 0.840250 / 0.720049 | -0.035750 / -0.044071 | ๊ณต๊ฐœ FP32 ์ˆ˜์น˜์™€ ๋™์ผ |
| LM04 | AUROC / TPR@FPR 5% / 1% | 0.667078 / 0.318983 / 0.233873 | 0.668862 / 0.322615 / 0.235853 | +0.001785 / +0.003632 / +0.001981 | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| OD06 | COCO bbox mAP | 24.8751% | 24.2822% | -0.5929 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| OD07 | COCO bbox mAP | 31.8594% | 31.4191% | -0.4403 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| SG06 | mIoU | 75.6398% | 74.1290% | -1.5108 pp | ๊ณต๊ฐœ FP32 75.32%, ์ฐจ์ด +0.3198 pp |
| SG07 | mIoU | 70.6473% | 69.6191% | -1.0282 pp | ๊ณต๊ฐœ FP32 70.19%, ์ฐจ์ด +0.4573 pp |
| SG08 | CamVid mIoU | 50.6498% | 51.1600% | +0.5102 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| SP01 | Top-1 accuracy | 91.86% | 91.66% | -0.2045 pp | ๊ณต๊ฐœ ์‹œํ—˜ ์ •ํ™•๋„ ์•ฝ 92%์™€ ๊ทผ์ ‘ |
| SP02 | FP / FN | 5 / 6 | 4 / 6 | -1 / 0 | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| SP08 | Yes/No Top-1 accuracy | 94.05% | 94.05% | 0.0000 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| SP09 | Top-1 accuracy | 95.06% | 94.70% | -0.3590 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC01 | Top-1 accuracy | 85.10% | 85.60% | +0.5000 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC02 | Top-1 accuracy | 87.00% | 87.00% | 0.0000 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC03 | Top-1 / Top-5 accuracy | 49.80% / 74.20% | 48.00% / 72.80% | -1.8000 / -1.4000 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC04 | Top-1 / Top-5 accuracy | 63.30% / 84.90% | 60.70% / 83.20% | -2.6000 / -1.7000 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC05 | Top-1 accuracy | 58.13% | 56.77% | -1.3600 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC06 | Top-1 accuracy | 66.20% | 65.31% | -0.8900 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC09 | Top-1 / Top-5 accuracy | 56.85% / 79.87% | 56.48% / 79.76% | -0.3700 / -0.1100 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC11 | Top-1 accuracy | 75.10% | 74.40% | -0.7000 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC12 | Top-1 / Top-5 accuracy | 69.48% / 89.26% | 68.30% / 88.44% | -1.1800 / -0.8200 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
| VC13 | Top-1 / Top-5 error | 33.65% / 13.43% | 33.85% / 13.66% | +0.2000 / +0.2300 pp | ๊ณต๊ฐœ ์ˆ˜์น˜ ์—†์Œ. |
VC13์€ accuracy๊ฐ€ ์•„๋‹ˆ๋ผ error์ด๋ฏ€๋กœ ๊ฐ’์ด ๋‚ฎ์„์ˆ˜๋ก ์ข‹์Œ