ONNX
onnxruntime
onnx-mlir
quantization
fp32
File size: 2,932 Bytes
ed3aeeb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# ๋ชจ๋ธ๋ณ„ ๋ณ€ํ™˜ ๊ฒฐ๊ณผ

๊ฐ ๋ชจ๋ธ์˜ FP32ยท์–‘์žํ™” ์›๋ณธ ๋ณต์‚ฌ๋ณธ, ๋ณ€ํ™˜ ONNX, ONNX Runtime ์ถœ๋ ฅ ๋น„๊ต, ONNX-MLIR ํŒŒ์ผ์„ ์ €์žฅํ•œ๋‹ค.

```text
models/<task>/<model_id>/
โ”œโ”€โ”€ source/       ์›๋ณธ FP32ยท์–‘์žํ™” ๋ชจ๋ธ
โ”œโ”€โ”€ baseline/     ์›๋ณธ ์‹คํ–‰ ๊ฒ€์ฆ
โ”œโ”€โ”€ onnx/         ๋ณ€ํ™˜ ONNX์™€ ์ถœ๋ ฅ ๋น„๊ต
โ”œโ”€โ”€ mlir/         ONNX-MLIR ๋ณ€ํ™˜ ๊ฒฐ๊ณผ
โ”œโ”€โ”€ graphs/netron Netron ONNX PNG์™€ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ
โ””โ”€โ”€ run_result.json
```

์›๋ณธ ๋ชจ๋ธ์€ `research/downloads/`์— ์žˆ๊ณ  ๊ฒฝ๋กœ์™€ SHA-256์€ [`../model_registry.csv`](../model_registry.csv)์— ๊ธฐ๋ก๋˜์–ด ์žˆ๋‹ค.

## ๊ณต๊ฐœ ์–‘์žํ™” ๋ชจ๋ธ ์ •๋ฐ€๋„

์•„๋ž˜ ์ •๋ฐ€๋„๋Š” ์ƒˆ๋กœ ์–‘์žํ™”ํ•œ ๊ฒฐ๊ณผ๊ฐ€ ์•„๋‹ˆ๋ผ ํ™•๋ณดํ•œ ๊ณต๊ฐœ ์–‘์žํ™” artifact์˜ ๊ตฌ์กฐ๋ฅผ ๊ฒ€์‚ฌํ•œ ๊ฐ’์ด๋‹ค. `INT8`์€ signed 8-bit, `UINT8`์€ unsigned 8-bit์ด๋ฉฐ, ์–‘์žํ™” convolution๊ณผ dense ์—ฐ์‚ฐ์˜ ๋ˆ„์‚ฐ๊ธฐ๋Š” ์ผ๋ฐ˜์ ์œผ๋กœ INT32๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค. ๋‚ด๋ถ€ ์—ฐ์‚ฐ์ด 8-bit์—ฌ๋„ ๋ชจ๋ธ ์ž…์ถœ๋ ฅ ๊ฒฝ๊ณ„๋Š” FP32์ผ ์ˆ˜ ์žˆ๋‹ค.

| ๋ชจ๋ธ | ์–‘์žํ™” ๋ฐฉ์‹ | ๊ฐ€์ค‘์น˜ ์ •๋ฐ€๋„ | activation/core ์ •๋ฐ€๋„ | ๋ชจ๋ธ ์ž…๋ ฅ โ†’ ์ถœ๋ ฅ |
|---|---|---|---|---|
| AD01 | Static PTQ, full integer | INT8 | INT8 | INT8 โ†’ INT8 |
| LM04 | Dynamic ONNX quantization | INT8/UINT8 | ์„ ํƒ MatMul์€ dynamic UINT8, ๋‚˜๋จธ์ง€๋Š” FP32 | INT64 tokens โ†’ FP32 |
| OD06 | Static full-integer core | INT8 | INT8 | UINT8 โ†’ FP32 raw heads |
| OD07 | Static full-integer core | INT8 | INT8 | UINT8 โ†’ FP32 raw heads |
| SG06 | Full integer | UINT8 | UINT8 | UINT8 โ†’ INT64 class IDs |
| SG07 | Full integer | UINT8 | UINT8 | UINT8 โ†’ INT64 class IDs |
| SG08 | Full integer | UINT8 | UINT8 | UINT8 โ†’ INT64 class IDs |
| SP01 | Static full integer | INT8 | INT8 | INT8 โ†’ INT8 |
| SP02 | Full-integer public artifact | INT8 | INT8 | INT8 โ†’ INT8 |
| SP08 | Full integer, legacy UINT8 artifact | UINT8 | UINT8 | UINT8 โ†’ UINT8 |
| SP09 | Clustered model + static PTQ | INT8 | INT8 | INT8 โ†’ INT8 |
| VC01 | Static full integer | INT8 | INT8 | INT8 โ†’ INT8 |
| VC02 | Static full integer | INT8 | INT8 | INT8 โ†’ INT8 |
| VC03 | QAT/FakeQuant + full integer | UINT8 | UINT8 | UINT8 โ†’ UINT8 |
| VC04 | QAT/FakeQuant + full integer | UINT8 | UINT8 | UINT8 โ†’ UINT8 |
| VC05 | Static INT8 | INT8 | INT8 | UINT8 โ†’ FP32 |
| VC06 | Static Q/DQ INT8 ONNX | INT8 | INT8 core with FP32 boundaries | FP32 โ†’ FP32 |
| VC09 | Static PTQ ONNX | INT8 | UINT8/INT8 core with FP32 boundaries | FP32 โ†’ FP32 |
| VC11 | Static full integer | INT8 | INT8/UINT8 | UINT8 โ†’ UINT8 |
| VC12 | Static PTQ ONNX | INT8 | UINT8/INT8 core with FP32 boundaries | FP32 โ†’ FP32 |
| VC13 | Static PTQ, mixed QOperator/QDQ | INT8 | UINT8/INT8 core with FP32 boundaries | FP32 โ†’ FP32 |

๋ชจ๋ธ๋ณ„ scale, zero point, ONNX ํ‘œํ˜„๊ณผ ONNX-MLIR ๋ณด์กด ๊ฒฐ๊ณผ๋Š” [`../reports/conversion/quantization_preservation.csv`](../reports/conversion/quantization_preservation.csv)์— ์žˆ๋‹ค.