ONNX
onnxruntime
onnx-mlir
quantization
fp32
ONNX_Models / models /README.md
purejomo's picture
Finalize public ONNX/ONNX-MLIR validation release
ed3aeeb
|
Raw
History Blame Contribute Delete
2.93 kB

๋ชจ๋ธ๋ณ„ ๋ณ€ํ™˜ ๊ฒฐ๊ณผ

๊ฐ ๋ชจ๋ธ์˜ FP32ยท์–‘์žํ™” ์›๋ณธ ๋ณต์‚ฌ๋ณธ, ๋ณ€ํ™˜ ONNX, ONNX Runtime ์ถœ๋ ฅ ๋น„๊ต, ONNX-MLIR ํŒŒ์ผ์„ ์ €์žฅํ•œ๋‹ค.

models/<task>/<model_id>/
โ”œโ”€โ”€ source/       ์›๋ณธ FP32ยท์–‘์žํ™” ๋ชจ๋ธ
โ”œโ”€โ”€ baseline/     ์›๋ณธ ์‹คํ–‰ ๊ฒ€์ฆ
โ”œโ”€โ”€ onnx/         ๋ณ€ํ™˜ ONNX์™€ ์ถœ๋ ฅ ๋น„๊ต
โ”œโ”€โ”€ mlir/         ONNX-MLIR ๋ณ€ํ™˜ ๊ฒฐ๊ณผ
โ”œโ”€โ”€ graphs/netron Netron ONNX PNG์™€ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ
โ””โ”€โ”€ run_result.json

์›๋ณธ ๋ชจ๋ธ์€ research/downloads/์— ์žˆ๊ณ  ๊ฒฝ๋กœ์™€ SHA-256์€ ../model_registry.csv์— ๊ธฐ๋ก๋˜์–ด ์žˆ๋‹ค.

๊ณต๊ฐœ ์–‘์žํ™” ๋ชจ๋ธ ์ •๋ฐ€๋„

์•„๋ž˜ ์ •๋ฐ€๋„๋Š” ์ƒˆ๋กœ ์–‘์žํ™”ํ•œ ๊ฒฐ๊ณผ๊ฐ€ ์•„๋‹ˆ๋ผ ํ™•๋ณดํ•œ ๊ณต๊ฐœ ์–‘์žํ™” artifact์˜ ๊ตฌ์กฐ๋ฅผ ๊ฒ€์‚ฌํ•œ ๊ฐ’์ด๋‹ค. INT8์€ signed 8-bit, UINT8์€ unsigned 8-bit์ด๋ฉฐ, ์–‘์žํ™” convolution๊ณผ dense ์—ฐ์‚ฐ์˜ ๋ˆ„์‚ฐ๊ธฐ๋Š” ์ผ๋ฐ˜์ ์œผ๋กœ INT32๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค. ๋‚ด๋ถ€ ์—ฐ์‚ฐ์ด 8-bit์—ฌ๋„ ๋ชจ๋ธ ์ž…์ถœ๋ ฅ ๊ฒฝ๊ณ„๋Š” FP32์ผ ์ˆ˜ ์žˆ๋‹ค.

๋ชจ๋ธ ์–‘์žํ™” ๋ฐฉ์‹ ๊ฐ€์ค‘์น˜ ์ •๋ฐ€๋„ activation/core ์ •๋ฐ€๋„ ๋ชจ๋ธ ์ž…๋ ฅ โ†’ ์ถœ๋ ฅ
AD01 Static PTQ, full integer INT8 INT8 INT8 โ†’ INT8
LM04 Dynamic ONNX quantization INT8/UINT8 ์„ ํƒ MatMul์€ dynamic UINT8, ๋‚˜๋จธ์ง€๋Š” FP32 INT64 tokens โ†’ FP32
OD06 Static full-integer core INT8 INT8 UINT8 โ†’ FP32 raw heads
OD07 Static full-integer core INT8 INT8 UINT8 โ†’ FP32 raw heads
SG06 Full integer UINT8 UINT8 UINT8 โ†’ INT64 class IDs
SG07 Full integer UINT8 UINT8 UINT8 โ†’ INT64 class IDs
SG08 Full integer UINT8 UINT8 UINT8 โ†’ INT64 class IDs
SP01 Static full integer INT8 INT8 INT8 โ†’ INT8
SP02 Full-integer public artifact INT8 INT8 INT8 โ†’ INT8
SP08 Full integer, legacy UINT8 artifact UINT8 UINT8 UINT8 โ†’ UINT8
SP09 Clustered model + static PTQ INT8 INT8 INT8 โ†’ INT8
VC01 Static full integer INT8 INT8 INT8 โ†’ INT8
VC02 Static full integer INT8 INT8 INT8 โ†’ INT8
VC03 QAT/FakeQuant + full integer UINT8 UINT8 UINT8 โ†’ UINT8
VC04 QAT/FakeQuant + full integer UINT8 UINT8 UINT8 โ†’ UINT8
VC05 Static INT8 INT8 INT8 UINT8 โ†’ FP32
VC06 Static Q/DQ INT8 ONNX INT8 INT8 core with FP32 boundaries FP32 โ†’ FP32
VC09 Static PTQ ONNX INT8 UINT8/INT8 core with FP32 boundaries FP32 โ†’ FP32
VC11 Static full integer INT8 INT8/UINT8 UINT8 โ†’ UINT8
VC12 Static PTQ ONNX INT8 UINT8/INT8 core with FP32 boundaries FP32 โ†’ FP32
VC13 Static PTQ, mixed QOperator/QDQ INT8 UINT8/INT8 core with FP32 boundaries FP32 โ†’ FP32

๋ชจ๋ธ๋ณ„ scale, zero point, ONNX ํ‘œํ˜„๊ณผ ONNX-MLIR ๋ณด์กด ๊ฒฐ๊ณผ๋Š” ../reports/conversion/quantization_preservation.csv์— ์žˆ๋‹ค.