Xenova HF Staff commited on
Commit
40f0ca1
·
verified ·
1 Parent(s): 9e7cac1

sync 2e7068faf55e

Browse files
README.md CHANGED
@@ -1,3 +1,85 @@
1
  ---
 
2
  license: apache-2.0
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: kernels
3
  license: apache-2.0
4
+ tags:
5
+ - kernel
6
+ - webgpu
7
+ - wgsl
8
  ---
9
+ # ai.onnx.QuantizeLinear
10
+
11
+ `ai.onnx` · standard ONNX operator · ONNX opset ≥ 25
12
+
13
+ ## Description
14
+
15
+ Linearly quantizes a high-precision tensor to a lower-precision integer type using the formula `y = saturate((x / y_scale) + y_zero_point)`, with rounding to nearest even. Supports per-tensor, per-axis, and blocked quantization granularities determined by the shape of `y_scale`.
16
+
17
+ See the [ONNX `QuantizeLinear` spec](https://onnx.ai/onnx/operators/onnx__QuantizeLinear.html) for the reference semantics.
18
+
19
+ ## Inputs
20
+
21
+ | Name | Bind key | Logical dtype | Rank | Shape | Description | Presence |
22
+ | --- | --- | --- | --- | --- | --- | --- |
23
+ | `x` | `x` | `TX` | — | — | N-D full-precision input tensor to be quantized. | required |
24
+ | `y_scale` | `y_scale` | `TS` | — | — | Scale factor; scalar for per-tensor, 1-D for per-axis, or same rank as `x` (with one axis blocked) for blocked quantization. | required |
25
+ | `y_zero_point` | `y_zero_point` | `TQ` | — | — | Zero point for quantization; must have the same shape as `y_scale`. Defaults to zero if omitted. | optional |
26
+
27
+ ## Outputs
28
+
29
+ | Name | Bind key | Logical dtype | Rank | Shape | Description | Presence |
30
+ | --- | --- | --- | --- | --- | --- | --- |
31
+ | `y` | `y` | `TQ` | same as `x` | same as `x` | N-D quantized output tensor with the same shape as `x`. | required |
32
+
33
+ ## Attributes
34
+
35
+ Default values (overridable per request):
36
+
37
+ | Attribute | Default | Description |
38
+ | --- | --- | --- |
39
+ | `axis` | `1` | Axis of the quantization dimension in `x`, used for per-axis and blocked quantization; negative values count from the end. |
40
+ | `block_size` | `0` | Number of elements along `axis` that share a single scale value for blocked quantization; 0 means blocked quantization is not used. |
41
+ | `output_dtype` | `0` | ONNX TensorProto element-type code for `y`; 0 infers the type from `y_zero_point`, or uint8 when the zero point is omitted. |
42
+ | `precision` | `0` | ONNX TensorProto element-type code used for `x / y_scale`; `0` uses the dtype of `y_scale`, `1` selects FLOAT, and `10` selects FLOAT16. |
43
+ | `saturate` | `1` | Controls out-of-range conversion for float8 outputs. The implemented int8/uint8 subset accepts the ONNX default `1`. |
44
+
45
+ ## Type constraints
46
+
47
+ | Variable | Allowed dtypes |
48
+ | --- | --- |
49
+ | `TX` | `float32`, `float16` |
50
+ | `TS` | `float32`, `float16` |
51
+ | `TQ` | `uint8`, `int8` |
52
+
53
+ ## Files
54
+
55
+ - [`metadata.json`](build/webgpu/metadata.json) — kernel metadata (id, digests, provenance)
56
+ - [`manifest.json`](build/webgpu/manifest.json) — the op contract (source of truth)
57
+ - [`test.json`](build/webgpu/test.json) — correctness cases
58
+ - [`bench.json`](build/webgpu/bench.json) — benchmark + tuning cases
59
+ - [`quant-linear-blocked-axis.wgsl.jinja`](build/webgpu/quant-linear-blocked-axis.wgsl.jinja)
60
+ - [`quant-linear-scalar.wgsl.jinja`](build/webgpu/quant-linear-scalar.wgsl.jinja)
61
+ - [`quant-linear-vec4.wgsl.jinja`](build/webgpu/quant-linear-vec4.wgsl.jinja)
62
+
63
+ ## Use with `@huggingface/kernels`
64
+
65
+ The loader automatically allocates outputs whose metadata it can derive from the manifest contract and this call.
66
+
67
+ The explicit `outputs` entries provide shape and logical dtype metadata for the results listed below:
68
+
69
+ - `y`
70
+
71
+ Each entry either requests an optional result or supplies metadata that cannot be inferred from the inputs.
72
+
73
+ The `version: 1` option selects the published kernel contract; it is independent of any operator opset, contrib `since_version`, or model version.
74
+
75
+ Replace each `*Data` placeholder with a typed array containing the corresponding input data.
76
+
77
+ ```js
78
+ import { getKernel } from "@huggingface/kernels";
79
+
80
+ const kernel = await getKernel("webgpu-kernels/ai.onnx.QuantizeLinear", { version: 1 });
81
+ // Explicit destinations request optional results or supply metadata that cannot be inferred.
82
+ const { y } = await kernel({ x: { data: xData, shape: [] }, y_scale: { data: y_scaleData, shape: [] } }, {
83
+ outputs: { y: { shape: [], dtype: "uint8" } },
84
+ });
85
+ ```
build/webgpu/bench.json ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "op": "ai.onnx.QuantizeLinear",
3
+ "tunableSpace": { "WORKGROUP_SIZE": [64, 128, 256] },
4
+ "cases": [
5
+ {
6
+ "name": "f32_to_u8_1m_scalar",
7
+ "preset": "smoke",
8
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 1048576 },
9
+ "inputs": {
10
+ "x": {
11
+ "dtype": "float32",
12
+ "shape": [1024, 1024],
13
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
14
+ },
15
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.02] } },
16
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
17
+ },
18
+ "outputs": { "y": { "dtype": "uint8", "shape": [1024, 1024] } },
19
+ "bench": {
20
+ "primary": true,
21
+ "metrics": [
22
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
23
+ ]
24
+ }
25
+ },
26
+ {
27
+ "name": "f32_to_u8_1m_no_zero_point",
28
+ "preset": "smoke",
29
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 1048576 },
30
+ "inputs": {
31
+ "x": {
32
+ "dtype": "float32",
33
+ "shape": [1024, 1024],
34
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
35
+ },
36
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.02] } }
37
+ },
38
+ "outputs": { "y": { "dtype": "uint8", "shape": [1024, 1024] } },
39
+ "bench": {
40
+ "metrics": [
41
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
42
+ ]
43
+ }
44
+ },
45
+ {
46
+ "name": "f32_to_i8_1m_plus3_scalar",
47
+ "preset": "smoke",
48
+ "vars": { "inputDtype": "float32", "outputDtype": "int8", "count": 1048579 },
49
+ "inputs": {
50
+ "x": {
51
+ "dtype": "float32",
52
+ "shape": [1048579],
53
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
54
+ },
55
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.02] } },
56
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [0] } }
57
+ },
58
+ "outputs": { "y": { "dtype": "int8", "shape": [1048579] } },
59
+ "bench": {
60
+ "metrics": [
61
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
62
+ ]
63
+ }
64
+ },
65
+ {
66
+ "name": "f32_to_u8_1m_plus3_no_zero_point",
67
+ "preset": "smoke",
68
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 1048579 },
69
+ "inputs": {
70
+ "x": {
71
+ "dtype": "float32",
72
+ "shape": [1048579],
73
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
74
+ },
75
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.02] } }
76
+ },
77
+ "outputs": { "y": { "dtype": "uint8", "shape": [1048579] } },
78
+ "bench": {
79
+ "metrics": [
80
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
81
+ ]
82
+ }
83
+ },
84
+ {
85
+ "name": "f16_to_u8_1m_scalar",
86
+ "preset": "smoke",
87
+ "vars": { "inputDtype": "float16", "outputDtype": "uint8", "count": 1048576 },
88
+ "inputs": {
89
+ "x": { "dtype": "float16", "shape": [1024, 1024], "data": { "kind": "constant", "value": 1.25 } },
90
+ "y_scale": { "dtype": "float16", "shape": [1], "data": { "kind": "values", "values": [0.02] } },
91
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
92
+ },
93
+ "outputs": { "y": { "dtype": "uint8", "shape": [1024, 1024] } },
94
+ "bench": {
95
+ "metrics": [
96
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
97
+ ]
98
+ }
99
+ },
100
+ {
101
+ "name": "blocked_axis1_4096x4096_block32_with_zp",
102
+ "preset": "smoke",
103
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 16777216 },
104
+ "attrs": { "axis": 1, "block_size": 32 },
105
+ "inputs": {
106
+ "x": {
107
+ "dtype": "float32",
108
+ "shape": [4096, 4096],
109
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
110
+ },
111
+ "y_scale": {
112
+ "dtype": "float32",
113
+ "shape": [4096, 128],
114
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.01, "offset": 0.02 }
115
+ },
116
+ "y_zero_point": { "dtype": "uint8", "shape": [4096, 128], "data": { "kind": "constant", "value": 128 } }
117
+ },
118
+ "outputs": { "y": { "dtype": "uint8", "shape": [4096, 4096] } },
119
+ "bench": {
120
+ "primary": true,
121
+ "metrics": [
122
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
123
+ ]
124
+ }
125
+ },
126
+ {
127
+ "name": "pertensor_4096x4096_scalar_healthy_sibling",
128
+ "preset": "smoke",
129
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 16777216 },
130
+ "inputs": {
131
+ "x": {
132
+ "dtype": "float32",
133
+ "shape": [4096, 4096],
134
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
135
+ },
136
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.02] } },
137
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
138
+ },
139
+ "outputs": { "y": { "dtype": "uint8", "shape": [4096, 4096] } },
140
+ "bench": {
141
+ "metrics": [
142
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
143
+ ]
144
+ }
145
+ },
146
+ {
147
+ "name": "per_axis0_inner2049_scalar_fallback_alignment_miss",
148
+ "preset": "smoke",
149
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 1049088 },
150
+ "attrs": { "axis": 0 },
151
+ "inputs": {
152
+ "x": {
153
+ "dtype": "float32",
154
+ "shape": [512, 2049],
155
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
156
+ },
157
+ "y_scale": {
158
+ "dtype": "float32",
159
+ "shape": [512],
160
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.01, "offset": 0.02 }
161
+ },
162
+ "y_zero_point": { "dtype": "uint8", "shape": [512], "data": { "kind": "constant", "value": 128 } }
163
+ },
164
+ "outputs": { "y": { "dtype": "uint8", "shape": [512, 2049] } },
165
+ "bench": {
166
+ "metrics": [
167
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
168
+ ]
169
+ }
170
+ },
171
+ {
172
+ "name": "per_axis0_inner2048_vec4_healthy_sibling",
173
+ "preset": "smoke",
174
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 1048576 },
175
+ "attrs": { "axis": 0 },
176
+ "inputs": {
177
+ "x": {
178
+ "dtype": "float32",
179
+ "shape": [512, 2048],
180
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
181
+ },
182
+ "y_scale": {
183
+ "dtype": "float32",
184
+ "shape": [512],
185
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.01, "offset": 0.02 }
186
+ },
187
+ "y_zero_point": { "dtype": "uint8", "shape": [512], "data": { "kind": "constant", "value": 128 } }
188
+ },
189
+ "outputs": { "y": { "dtype": "uint8", "shape": [512, 2048] } },
190
+ "bench": {
191
+ "metrics": [
192
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
193
+ ]
194
+ }
195
+ },
196
+ {
197
+ "name": "f16_per_axis0_inner2048_vec4_unbenched",
198
+ "preset": "smoke",
199
+ "vars": { "inputDtype": "float16", "outputDtype": "uint8", "count": 1048576 },
200
+ "attrs": { "axis": 0 },
201
+ "inputs": {
202
+ "x": { "dtype": "float16", "shape": [512, 2048], "data": { "kind": "constant", "value": 1.25 } },
203
+ "y_scale": {
204
+ "dtype": "float16",
205
+ "shape": [512],
206
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.01, "offset": 0.02 }
207
+ },
208
+ "y_zero_point": { "dtype": "uint8", "shape": [512], "data": { "kind": "constant", "value": 128 } }
209
+ },
210
+ "outputs": { "y": { "dtype": "uint8", "shape": [512, 2048] } },
211
+ "bench": {
212
+ "metrics": [
213
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
214
+ ]
215
+ }
216
+ },
217
+ {
218
+ "name": "per_axis1_lastdim_inner1_generic_scalar_cliff",
219
+ "preset": "stress",
220
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 4194304 },
221
+ "attrs": { "axis": 1 },
222
+ "inputs": {
223
+ "x": {
224
+ "dtype": "float32",
225
+ "shape": [2048, 2048],
226
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
227
+ },
228
+ "y_scale": {
229
+ "dtype": "float32",
230
+ "shape": [2048],
231
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.01, "offset": 0.02 }
232
+ },
233
+ "y_zero_point": { "dtype": "uint8", "shape": [2048], "data": { "kind": "constant", "value": 128 } }
234
+ },
235
+ "outputs": { "y": { "dtype": "uint8", "shape": [2048, 2048] } },
236
+ "bench": {
237
+ "metrics": [
238
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
239
+ ]
240
+ }
241
+ },
242
+ {
243
+ "name": "per_axis1_lastdim_inner1_generic_scalar_no_zero_point_cliff",
244
+ "preset": "stress",
245
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 4194304 },
246
+ "attrs": { "axis": 1 },
247
+ "inputs": {
248
+ "x": {
249
+ "dtype": "float32",
250
+ "shape": [2048, 2048],
251
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
252
+ },
253
+ "y_scale": {
254
+ "dtype": "float32",
255
+ "shape": [2048],
256
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.01, "offset": 0.02 }
257
+ }
258
+ },
259
+ "outputs": { "y": { "dtype": "uint8", "shape": [2048, 2048] } },
260
+ "bench": {
261
+ "metrics": [
262
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
263
+ ]
264
+ }
265
+ },
266
+ {
267
+ "name": "blocked-rank4-axis1-bs32-no-zp-scalar-pathology",
268
+ "preset": "stress",
269
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 4194304 },
270
+ "attrs": { "axis": 1, "block_size": 32 },
271
+ "inputs": {
272
+ "x": {
273
+ "dtype": "float32",
274
+ "shape": [4, 512, 32, 64],
275
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
276
+ },
277
+ "y_scale": {
278
+ "dtype": "float32",
279
+ "shape": [4, 16, 32, 64],
280
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.005, "offset": 0.02 }
281
+ }
282
+ },
283
+ "outputs": { "y": { "dtype": "uint8", "shape": [4, 512, 32, 64], "dist": "empty" } },
284
+ "bench": {
285
+ "primary": true,
286
+ "metrics": [
287
+ { "type": "bandwidth", "value": "args.count * (dtypeBytes(args.inputDtype) + dtypeBytes(args.outputDtype))" }
288
+ ]
289
+ }
290
+ },
291
+ {
292
+ "name": "blocked-rank4-axis1-bs32-with-zp-scalar-pathology",
293
+ "preset": "stress",
294
+ "vars": { "inputDtype": "float32", "outputDtype": "uint8", "count": 4194304 },
295
+ "attrs": { "axis": 1, "block_size": 32 },
296
+ "inputs": {
297
+ "x": {
298
+ "dtype": "float32",
299
+ "shape": [4, 512, 32, 64],
300
+ "data": { "kind": "fillFloat32", "sinStep": 0.013, "cosStep": 0.017, "scale": 3.0 }
301
+ },
302
+ "y_scale": {
303
+ "dtype": "float32",
304
+ "shape": [4, 16, 32, 64],
305
+ "data": { "kind": "fillFloat32", "sinStep": 0.07, "cosStep": 0.05, "scale": 0.005, "offset": 0.02 }
306
+ },
307
+ "y_zero_point": {
308
+ "dtype": "uint8",
309
+ "shape": [4, 16, 32, 64],
310
+ "dist": "randint",
311
+ "seed": 204,
312
+ "min": 96,
313
+ "max": 160
314
+ }
315
+ },
316
+ "outputs": { "y": { "dtype": "uint8", "shape": [4, 512, 32, 64], "dist": "empty" } },
317
+ "bench": {
318
+ "primary": true,
319
+ "metrics": [
320
+ {
321
+ "type": "bandwidth",
322
+ "value": "dtypeBytes(args.inputDtype) * (numel(shapes.x) + numel(shapes.y_scale)) + dtypeBytes(args.outputDtype) * (numel(shapes.y_zero_point) + numel(shapes.y))"
323
+ }
324
+ ]
325
+ }
326
+ }
327
+ ]
328
+ }
build/webgpu/manifest.json ADDED
@@ -0,0 +1,839 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "domain": "ai.onnx",
3
+ "name": "QuantizeLinear",
4
+ "sinceVersion": 25,
5
+ "description": "Linearly quantizes a high-precision tensor to a lower-precision integer type using the formula `y = saturate((x / y_scale) + y_zero_point)`, with rounding to nearest even. Supports per-tensor, per-axis, and blocked quantization granularities determined by the shape of `y_scale`.",
6
+ "inputs": [
7
+ { "role": "x", "dtype": "TX", "description": "N-D full-precision input tensor to be quantized." },
8
+ {
9
+ "role": "y_scale",
10
+ "dtype": "TS",
11
+ "description": "Scale factor; scalar for per-tensor, 1-D for per-axis, or same rank as `x` (with one axis blocked) for blocked quantization."
12
+ },
13
+ {
14
+ "role": "y_zero_point",
15
+ "dtype": "TQ",
16
+ "optional": true,
17
+ "description": "Zero point for quantization; must have the same shape as `y_scale`. Defaults to zero if omitted."
18
+ }
19
+ ],
20
+ "outputs": [
21
+ {
22
+ "role": "y",
23
+ "dtype": "TQ",
24
+ "rank": "ranks.x",
25
+ "description": "N-D quantized output tensor with the same shape as `x`.",
26
+ "shape": "shapes.x"
27
+ }
28
+ ],
29
+ "attributes": { "axis": 1, "block_size": 0, "output_dtype": 0, "precision": 0, "saturate": 1 },
30
+ "attributeDescriptions": {
31
+ "axis": "Axis of the quantization dimension in `x`, used for per-axis and blocked quantization; negative values count from the end.",
32
+ "block_size": "Number of elements along `axis` that share a single scale value for blocked quantization; 0 means blocked quantization is not used.",
33
+ "output_dtype": "ONNX TensorProto element-type code for `y`; 0 infers the type from `y_zero_point`, or uint8 when the zero point is omitted.",
34
+ "precision": "ONNX TensorProto element-type code used for `x / y_scale`; `0` uses the dtype of `y_scale`, `1` selects FLOAT, and `10` selects FLOAT16.",
35
+ "saturate": "Controls out-of-range conversion for float8 outputs. The implemented int8/uint8 subset accepts the ONNX default `1`."
36
+ },
37
+ "attributeConstraints": { "precision": { "values": [0, 1, 10] }, "saturate": { "values": [1] } },
38
+ "typeConstraints": { "TX": ["float32", "float16"], "TS": ["float32", "float16"], "TQ": ["uint8", "int8"] },
39
+ "args": {
40
+ "x": { "kind": "tensor", "semantic": "x", "role": "input" },
41
+ "y_scale": { "kind": "tensor", "semantic": "y_scale", "role": "input" },
42
+ "y_zero_point": { "kind": "tensor", "semantic": "y_zero_point", "role": "input", "required": false },
43
+ "y": { "kind": "tensor", "semantic": "y", "role": "output" }
44
+ },
45
+ "tunables": { "WORKGROUP_SIZE": 256, "VEC4_TAIL_MIN_ELEMENTS": 4096 },
46
+ "derive": {
47
+ "deviceWorkgroupCap": "min(device.limits.maxComputeInvocationsPerWorkgroup, device.limits.maxComputeWorkgroupSizeX)",
48
+ "workgroupOk": "tunables.WORKGROUP_SIZE <= deviceWorkgroupCap",
49
+ "outputDispatchFits": "workgroupOk and ceilDiv(ceilDiv(numel(shapes.y), tunables.WORKGROUP_SIZE), device.limits.maxComputeWorkgroupsPerDimension) <= device.limits.maxComputeWorkgroupsPerDimension",
50
+ "outputDtypeOk": "(attrs.output_dtype == 0 and (present.y_zero_point or tensorDtypes.y == \"uint8\")) or attrs.output_dtype == onnxDtypeCode(logicalDtypes.TQ)",
51
+ "quantizeDivisionF16": "attrs.precision == onnxDtypeCode(\"float16\") or (attrs.precision == 0 and tensorDtypes.y_scale == \"float16\")",
52
+ "sameShapeOk": "ranks.y == ranks.x and numel(shapes.x) == numel(shapes.y) and outputDispatchFits and outputDtypeOk",
53
+ "quantDtypesOk": "(tensorDtypes.x != \"float16\" and tensorDtypes.y_scale != \"float16\" and not quantizeDivisionF16) or device.features.has(\"shader-f16\")",
54
+ "blockedScaleOk": "attrs.block_size > 0 and ranks.x >= 2 and ranks.y_scale == ranks.x",
55
+ "elementCount": "numel(shapes.y)",
56
+ "elementCount4": "floor(numel(shapes.y) / 4)",
57
+ "scaleSize": "1 if ranks.y_scale == 0 else dim(shapes.y_scale, 0)",
58
+ "scaleInner": "1 if ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1 else inner(shapes.x, attrs.axis)",
59
+ "blockedAxisDim": "dim(shapes.x, attrs.axis) if ranks.x >= 2 else 1",
60
+ "blockedScaleAxisDim": "dim(shapes.y_scale, attrs.axis) if ranks.y_scale >= 2 else 1",
61
+ "blockedInner": "inner(shapes.x, attrs.axis) if ranks.x >= 2 else 1"
62
+ },
63
+ "constants": {
64
+ "usesF16": "dtypes.TX == \"f16\" or dtypes.TS == \"f16\" or quantizeDivisionF16",
65
+ "xScalar": "dtypes.TX",
66
+ "xVec4": "\"vec4<\" ~ dtypes.TX ~ \">\"",
67
+ "scaleScalar": "dtypes.TS",
68
+ "scaleVec4": "\"vec4<\" ~ dtypes.TS ~ \">\"",
69
+ "yScalar": "dtypes.TQ",
70
+ "yVec4": "\"vec4<\" ~ dtypes.TQ ~ \">\"",
71
+ "yUnsigned": "tensorDtypes.y == \"uint8\"",
72
+ "qMin": "0 if tensorDtypes.y == \"uint8\" else 0 - 128",
73
+ "qMax": "255 if tensorDtypes.y == \"uint8\" else 127",
74
+ "divisionF16": "quantizeDivisionF16"
75
+ },
76
+ "variants": [
77
+ {
78
+ "id": "innermost_axis_vec4_with_zero_point",
79
+ "priority": 20,
80
+ "when": ["present.y_zero_point", "ranks.x >= 1", "attrs.axis == -1 or attrs.axis == ranks.x - 1", "ranks.y_scale == 1", "dim(shapes.y_scale, 0) == dim(shapes.x, ranks.x - 1)", "dim(shapes.y_scale, 0) % 4 == 0", "ranks.y_zero_point == 1", "dim(shapes.y_zero_point, 0) == dim(shapes.y_scale, 0)", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "sameShapeOk", "quantDtypesOk"],
81
+ "constants": { "hasZero": true },
82
+ "passes": [
83
+ {
84
+ "id": "main",
85
+ "name": "QuantizeLinear.InnermostAxisVec4",
86
+ "source": {
87
+ "shader": "quant-linear-vec4.wgsl.jinja",
88
+ "inputs": { "op": "\"quantize\"", "perAxis": true, "vectorParams": true }
89
+ },
90
+ "bindings": "linearInnermostVec4WithZero",
91
+ "dispatch": { "threads": "floor(numel(shapes.y) / 4)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
92
+ }
93
+ ]
94
+ },
95
+ {
96
+ "id": "innermost_axis_vec4_no_zero_point",
97
+ "priority": 20,
98
+ "when": ["not present.y_zero_point", "ranks.x >= 1", "attrs.axis == -1 or attrs.axis == ranks.x - 1", "ranks.y_scale == 1", "dim(shapes.y_scale, 0) == dim(shapes.x, ranks.x - 1)", "dim(shapes.y_scale, 0) % 4 == 0", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "sameShapeOk", "quantDtypesOk"],
99
+ "constants": { "hasZero": false },
100
+ "passes": [
101
+ {
102
+ "id": "main",
103
+ "name": "QuantizeLinear.InnermostAxisVec4NoZero",
104
+ "source": {
105
+ "shader": "quant-linear-vec4.wgsl.jinja",
106
+ "inputs": { "op": "\"quantize\"", "perAxis": true, "vectorParams": true }
107
+ },
108
+ "bindings": "linearInnermostVec4NoZero",
109
+ "dispatch": { "threads": "floor(numel(shapes.y) / 4)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
110
+ }
111
+ ]
112
+ },
113
+ {
114
+ "id": "vec4_cross_axis_with_zero_point",
115
+ "priority": 16,
116
+ "when": ["present.y_zero_point", "attrs.block_size == 0", "ranks.x >= 1", "attrs.axis >= 0", "attrs.axis < ranks.x", "ranks.y_scale == 1", "dim(shapes.y_scale, 0) == dim(shapes.x, attrs.axis)", "inner(shapes.x, attrs.axis) % 4 != 0", "ranks.y_zero_point == 1", "dim(shapes.y_zero_point, 0) == dim(shapes.y_scale, 0)", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "sameShapeOk", "quantDtypesOk"],
117
+ "constants": { "hasZero": true },
118
+ "passes": [
119
+ {
120
+ "id": "main",
121
+ "name": "QuantizeLinear.Vec4CrossAxis",
122
+ "source": {
123
+ "shader": "quant-linear-vec4.wgsl.jinja",
124
+ "inputs": { "op": "\"quantize\"", "perAxis": true, "crossingParams": true }
125
+ },
126
+ "bindings": "linearVec4WithZero",
127
+ "dispatch": { "threads": "floor(numel(shapes.y) / 4)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
128
+ }
129
+ ]
130
+ },
131
+ {
132
+ "id": "vec4_with_zero_point",
133
+ "priority": 15,
134
+ "when": ["present.y_zero_point", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "(ranks.y_scale == 0 or ranks.y_scale == 1)", "sameShapeOk", "(ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1 or (ranks.x >= 1 and dim(shapes.y_scale, 0) == dim(shapes.x, attrs.axis) and inner(shapes.x, attrs.axis) % 4 == 0))", "(ranks.y_zero_point == 0 or ranks.y_zero_point == 1)", "(ranks.y_zero_point == 0 or dim(shapes.y_zero_point, 0) == 1 or (ranks.x >= 1 and dim(shapes.y_zero_point, 0) == dim(shapes.x, attrs.axis)))", "quantDtypesOk"],
135
+ "constants": { "hasZero": true },
136
+ "passes": [
137
+ {
138
+ "id": "main",
139
+ "name": "QuantizeLinear.Vec4",
140
+ "source": {
141
+ "shader": "quant-linear-vec4.wgsl.jinja",
142
+ "inputs": { "op": "\"quantize\"", "perAxis": "not (ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1)" }
143
+ },
144
+ "bindings": "linearVec4WithZero",
145
+ "dispatch": { "threads": "floor(numel(shapes.y) / 4)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
146
+ }
147
+ ]
148
+ },
149
+ {
150
+ "id": "vec4_no_zero_point",
151
+ "priority": 15,
152
+ "when": ["not present.y_zero_point", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "(ranks.y_scale == 0 or ranks.y_scale == 1)", "sameShapeOk", "(ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1 or (ranks.x >= 1 and dim(shapes.y_scale, 0) == dim(shapes.x, attrs.axis) and inner(shapes.x, attrs.axis) % 4 == 0))", "quantDtypesOk"],
153
+ "constants": { "hasZero": false },
154
+ "passes": [
155
+ {
156
+ "id": "main",
157
+ "name": "QuantizeLinear.Vec4",
158
+ "source": {
159
+ "shader": "quant-linear-vec4.wgsl.jinja",
160
+ "inputs": { "op": "\"quantize\"", "perAxis": "not (ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1)" }
161
+ },
162
+ "bindings": "linearVec4NoZero",
163
+ "dispatch": { "threads": "numel(shapes.y) / 4", "workgroupSize": "tunables.WORKGROUP_SIZE" }
164
+ }
165
+ ]
166
+ },
167
+ {
168
+ "id": "vec4_tail_with_zero_point",
169
+ "priority": 12,
170
+ "when": ["present.y_zero_point", "numel(shapes.y) >= tunables.VEC4_TAIL_MIN_ELEMENTS", "numel(shapes.y) % 4 != 0", "(ranks.y_scale == 0 or (ranks.y_scale == 1 and dim(shapes.y_scale, 0) == 1))", "(ranks.y_zero_point == 0 or (ranks.y_zero_point == 1 and dim(shapes.y_zero_point, 0) == 1))", "sameShapeOk", "quantDtypesOk"],
171
+ "constants": { "hasZero": true },
172
+ "passes": [
173
+ {
174
+ "id": "bulk",
175
+ "name": "QuantizeLinear.Vec4Bulk",
176
+ "source": { "shader": "quant-linear-vec4.wgsl.jinja", "inputs": { "op": "\"quantize\"", "perAxis": false } },
177
+ "bindings": "linearVec4PerTensorWithZero",
178
+ "dispatch": { "threads": "floor(numel(shapes.y) / 4)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
179
+ },
180
+ {
181
+ "id": "tail",
182
+ "name": "QuantizeLinear.ScalarTail",
183
+ "source": {
184
+ "shader": "quant-linear-scalar.wgsl.jinja",
185
+ "inputs": { "op": "\"quantize\"", "x4": true, "perAxis": false }
186
+ },
187
+ "bindings": "linearScalarPerTensorWithZero",
188
+ "dispatch": { "threads": 1, "workgroupSize": "tunables.WORKGROUP_SIZE" }
189
+ }
190
+ ]
191
+ },
192
+ {
193
+ "id": "vec4_tail_no_zero_point",
194
+ "priority": 12,
195
+ "when": ["not present.y_zero_point", "numel(shapes.y) >= tunables.VEC4_TAIL_MIN_ELEMENTS", "numel(shapes.y) % 4 != 0", "(ranks.y_scale == 0 or (ranks.y_scale == 1 and dim(shapes.y_scale, 0) == 1))", "sameShapeOk", "quantDtypesOk"],
196
+ "constants": { "hasZero": false },
197
+ "passes": [
198
+ {
199
+ "id": "bulk",
200
+ "name": "QuantizeLinear.Vec4Bulk",
201
+ "source": { "shader": "quant-linear-vec4.wgsl.jinja", "inputs": { "op": "\"quantize\"", "perAxis": false } },
202
+ "bindings": "linearVec4PerTensorNoZero",
203
+ "dispatch": { "threads": "floor(numel(shapes.y) / 4)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
204
+ },
205
+ {
206
+ "id": "tail",
207
+ "name": "QuantizeLinear.ScalarTail",
208
+ "source": {
209
+ "shader": "quant-linear-scalar.wgsl.jinja",
210
+ "inputs": { "op": "\"quantize\"", "x4": true, "perAxis": false }
211
+ },
212
+ "bindings": "linearScalarPerTensorNoZero",
213
+ "dispatch": { "threads": 1, "workgroupSize": "tunables.WORKGROUP_SIZE" }
214
+ }
215
+ ]
216
+ },
217
+ {
218
+ "id": "blocked_last_axis_vec4_with_zero_point",
219
+ "priority": 24,
220
+ "when": ["present.y_zero_point", "blockedScaleOk", "attrs.block_size % 4 == 0", "ranks.y_zero_point == ranks.x", "sameShapeOk", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "inner(shapes.x, attrs.axis) == 1", "dim(shapes.x, attrs.axis) % attrs.block_size == 0", "dim(shapes.y_scale, attrs.axis) == dim(shapes.x, attrs.axis) / attrs.block_size", "numel(shapes.y_zero_point) == numel(shapes.y_scale)", "tensorDtypes.x == \"float32\"", "tensorDtypes.y_scale == \"float32\""],
221
+ "constants": { "hasZero": true, "blockVectors": "attrs.block_size / 4" },
222
+ "passes": [
223
+ {
224
+ "id": "main",
225
+ "name": "QuantizeLinear.BlockedLastAxisVec4",
226
+ "source": { "shader": "quant-linear-blocked-axis.wgsl.jinja", "inputs": { "lastAxisVectorized": true } },
227
+ "bindings": "blockedLastVec4WithZero",
228
+ "dispatch": { "threads": "numel(shapes.y) / 4", "workgroupSize": "tunables.WORKGROUP_SIZE" }
229
+ }
230
+ ]
231
+ },
232
+ {
233
+ "id": "blocked_last_axis_vec4_no_zero_point",
234
+ "priority": 23,
235
+ "when": ["not present.y_zero_point", "blockedScaleOk", "attrs.block_size % 4 == 0", "sameShapeOk", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "inner(shapes.x, attrs.axis) == 1", "dim(shapes.x, attrs.axis) % attrs.block_size == 0", "dim(shapes.y_scale, attrs.axis) == dim(shapes.x, attrs.axis) / attrs.block_size", "tensorDtypes.x == \"float32\"", "tensorDtypes.y_scale == \"float32\""],
236
+ "constants": { "hasZero": false, "blockVectors": "attrs.block_size / 4" },
237
+ "passes": [
238
+ {
239
+ "id": "main",
240
+ "name": "QuantizeLinear.BlockedLastAxisVec4",
241
+ "source": { "shader": "quant-linear-blocked-axis.wgsl.jinja", "inputs": { "lastAxisVectorized": true } },
242
+ "bindings": "blockedLastVec4NoZero",
243
+ "dispatch": { "threads": "numel(shapes.y) / 4", "workgroupSize": "tunables.WORKGROUP_SIZE" }
244
+ }
245
+ ]
246
+ },
247
+ {
248
+ "id": "blocked_vec4_with_zero_point",
249
+ "priority": 22,
250
+ "when": ["present.y_zero_point", "blockedScaleOk", "ranks.y_zero_point == ranks.x", "sameShapeOk", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "numel(shapes.y_scale) % 4 == 0", "numel(shapes.y_zero_point) == numel(shapes.y_scale)", "inner(shapes.x, attrs.axis) % 4 == 0", "dim(shapes.y_scale, attrs.axis) == ceil(dim(shapes.x, attrs.axis) / attrs.block_size)", "dim(shapes.y_zero_point, attrs.axis) == dim(shapes.y_scale, attrs.axis)", "tensorDtypes.x == \"float32\"", "tensorDtypes.y_scale == \"float32\""],
251
+ "constants": { "hasZero": true },
252
+ "passes": [
253
+ {
254
+ "id": "main",
255
+ "name": "QuantizeLinear.BlockedVec4WithZeroPoint",
256
+ "source": { "shader": "quant-linear-blocked-axis.wgsl.jinja", "inputs": { "vectorized": true } },
257
+ "bindings": "blockedVec4WithZero",
258
+ "dispatch": { "threads": "numel(shapes.y) / 4", "workgroupSize": "tunables.WORKGROUP_SIZE" }
259
+ }
260
+ ]
261
+ },
262
+ {
263
+ "id": "blocked_vec4_no_zero_point",
264
+ "priority": 21,
265
+ "when": ["not present.y_zero_point", "blockedScaleOk", "sameShapeOk", "numel(shapes.y) > 0", "numel(shapes.y) % 4 == 0", "numel(shapes.y_scale) % 4 == 0", "inner(shapes.x, attrs.axis) % 4 == 0", "dim(shapes.y_scale, attrs.axis) == ceil(dim(shapes.x, attrs.axis) / attrs.block_size)", "tensorDtypes.x == \"float32\"", "tensorDtypes.y_scale == \"float32\""],
266
+ "constants": { "hasZero": false },
267
+ "passes": [
268
+ {
269
+ "id": "main",
270
+ "name": "QuantizeLinear.BlockedVec4",
271
+ "source": { "shader": "quant-linear-blocked-axis.wgsl.jinja", "inputs": { "vectorized": true } },
272
+ "bindings": "blockedVec4NoZero",
273
+ "dispatch": { "threads": "numel(shapes.y) / 4", "workgroupSize": "tunables.WORKGROUP_SIZE" }
274
+ }
275
+ ]
276
+ },
277
+ {
278
+ "id": "blocked_with_zero_point",
279
+ "priority": 19,
280
+ "when": ["present.y_zero_point", "blockedScaleOk", "ranks.y_zero_point == ranks.x", "sameShapeOk", "dim(shapes.y_scale, attrs.axis) == ceil(dim(shapes.x, attrs.axis) / attrs.block_size)", "dim(shapes.y_zero_point, attrs.axis) == dim(shapes.y_scale, attrs.axis)", "quantDtypesOk"],
281
+ "constants": { "hasZero": true },
282
+ "passes": [
283
+ {
284
+ "id": "main",
285
+ "name": "QuantizeLinear.Blocked",
286
+ "shader": "quant-linear-blocked-axis.wgsl.jinja",
287
+ "bindings": "blockedScalarWithZero",
288
+ "dispatch": { "threads": "numel(shapes.y)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
289
+ }
290
+ ]
291
+ },
292
+ {
293
+ "id": "blocked_no_zero_point",
294
+ "priority": 19,
295
+ "when": ["not present.y_zero_point", "blockedScaleOk", "sameShapeOk", "dim(shapes.y_scale, attrs.axis) == ceil(dim(shapes.x, attrs.axis) / attrs.block_size)", "quantDtypesOk"],
296
+ "constants": { "hasZero": false },
297
+ "passes": [
298
+ {
299
+ "id": "main",
300
+ "name": "QuantizeLinear.BlockedNoZero",
301
+ "shader": "quant-linear-blocked-axis.wgsl.jinja",
302
+ "bindings": "blockedScalarNoZero",
303
+ "dispatch": { "threads": "numel(shapes.y)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
304
+ }
305
+ ]
306
+ },
307
+ {
308
+ "id": "with_zero_point",
309
+ "priority": 10,
310
+ "when": ["present.y_zero_point", "(ranks.y_scale == 0 or ranks.y_scale == 1)", "sameShapeOk", "(ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1 or (ranks.x >= 1 and dim(shapes.y_scale, 0) == dim(shapes.x, attrs.axis)))", "(ranks.y_zero_point == 0 or ranks.y_zero_point == 1)", "(ranks.y_zero_point == 0 or dim(shapes.y_zero_point, 0) == 1 or (ranks.x >= 1 and dim(shapes.y_zero_point, 0) == dim(shapes.x, attrs.axis)))", "quantDtypesOk"],
311
+ "constants": { "hasZero": true },
312
+ "passes": [
313
+ {
314
+ "id": "main",
315
+ "name": "QuantizeLinear",
316
+ "source": {
317
+ "shader": "quant-linear-scalar.wgsl.jinja",
318
+ "inputs": {
319
+ "op": "\"quantize\"",
320
+ "x4": "false",
321
+ "perAxis": "not (ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1)"
322
+ }
323
+ },
324
+ "bindings": "linearScalarWithZero",
325
+ "dispatch": { "threads": "numel(shapes.y)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
326
+ }
327
+ ]
328
+ },
329
+ {
330
+ "id": "no_zero_point",
331
+ "when": ["not present.y_zero_point", "(ranks.y_scale == 0 or ranks.y_scale == 1)", "sameShapeOk", "(ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1 or (ranks.x >= 1 and dim(shapes.y_scale, 0) == dim(shapes.x, attrs.axis)))", "quantDtypesOk"],
332
+ "constants": { "hasZero": false },
333
+ "passes": [
334
+ {
335
+ "id": "main",
336
+ "name": "QuantizeLinear",
337
+ "source": {
338
+ "shader": "quant-linear-scalar.wgsl.jinja",
339
+ "inputs": {
340
+ "op": "\"quantize\"",
341
+ "x4": "false",
342
+ "perAxis": "not (ranks.y_scale == 0 or dim(shapes.y_scale, 0) == 1)"
343
+ }
344
+ },
345
+ "bindings": "linearScalarNoZero",
346
+ "dispatch": { "threads": "numel(shapes.y)", "workgroupSize": "tunables.WORKGROUP_SIZE" }
347
+ }
348
+ ]
349
+ }
350
+ ],
351
+ "bindingSets": {
352
+ "linearVec4WithZero": [
353
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
354
+ {
355
+ "name": "y_scale",
356
+ "arg": "y_scale",
357
+ "semantic": "y_scale",
358
+ "buffer": { "type": "read-only-storage" },
359
+ "elementType": "$scaleScalar"
360
+ },
361
+ {
362
+ "name": "y_zero_point",
363
+ "arg": "y_zero_point",
364
+ "semantic": "y_zero_point",
365
+ "buffer": { "type": "read-only-storage" },
366
+ "elementType": "$yScalar"
367
+ },
368
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
369
+ {
370
+ "name": "params",
371
+ "semantic": "kernel.params",
372
+ "buffer": { "type": "uniform" },
373
+ "struct": {
374
+ "name": "Params",
375
+ "fields": [
376
+ { "name": "count4", "type": "u32", "value": "elementCount4" },
377
+ { "name": "scaleSize", "type": "u32", "value": "scaleSize" },
378
+ { "name": "inner", "type": "u32", "value": "scaleInner" }
379
+ ]
380
+ }
381
+ }
382
+ ],
383
+ "linearInnermostVec4WithZero": [
384
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
385
+ {
386
+ "name": "y_scale",
387
+ "arg": "y_scale",
388
+ "semantic": "y_scale",
389
+ "buffer": { "type": "read-only-storage" },
390
+ "elementType": "$scaleVec4"
391
+ },
392
+ {
393
+ "name": "y_zero_point",
394
+ "arg": "y_zero_point",
395
+ "semantic": "y_zero_point",
396
+ "buffer": { "type": "read-only-storage" },
397
+ "elementType": "$yVec4"
398
+ },
399
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
400
+ {
401
+ "name": "params",
402
+ "semantic": "kernel.params",
403
+ "buffer": { "type": "uniform" },
404
+ "struct": {
405
+ "name": "Params",
406
+ "fields": [
407
+ { "name": "count4", "type": "u32", "value": "elementCount4" },
408
+ { "name": "scaleSize", "type": "u32", "value": "scaleSize" }
409
+ ]
410
+ }
411
+ }
412
+ ],
413
+ "linearInnermostVec4NoZero": [
414
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
415
+ {
416
+ "name": "y_scale",
417
+ "arg": "y_scale",
418
+ "semantic": "y_scale",
419
+ "buffer": { "type": "read-only-storage" },
420
+ "elementType": "$scaleVec4"
421
+ },
422
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
423
+ {
424
+ "name": "params",
425
+ "semantic": "kernel.params",
426
+ "buffer": { "type": "uniform" },
427
+ "struct": {
428
+ "name": "Params",
429
+ "fields": [
430
+ { "name": "count4", "type": "u32", "value": "elementCount4" },
431
+ { "name": "scaleSize", "type": "u32", "value": "scaleSize" }
432
+ ]
433
+ }
434
+ }
435
+ ],
436
+ "linearVec4NoZero": [
437
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
438
+ {
439
+ "name": "y_scale",
440
+ "arg": "y_scale",
441
+ "semantic": "y_scale",
442
+ "buffer": { "type": "read-only-storage" },
443
+ "elementType": "$scaleScalar"
444
+ },
445
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
446
+ {
447
+ "name": "params",
448
+ "semantic": "kernel.params",
449
+ "buffer": { "type": "uniform" },
450
+ "struct": {
451
+ "name": "Params",
452
+ "fields": [
453
+ { "name": "count4", "type": "u32", "value": "elementCount4" },
454
+ { "name": "scaleSize", "type": "u32", "value": "scaleSize" },
455
+ { "name": "inner", "type": "u32", "value": "scaleInner" }
456
+ ]
457
+ }
458
+ }
459
+ ],
460
+ "linearScalarWithZero": [
461
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
462
+ {
463
+ "name": "y_scale",
464
+ "arg": "y_scale",
465
+ "semantic": "y_scale",
466
+ "buffer": { "type": "read-only-storage" },
467
+ "elementType": "$scaleScalar"
468
+ },
469
+ {
470
+ "name": "y_zero_point",
471
+ "arg": "y_zero_point",
472
+ "semantic": "y_zero_point",
473
+ "buffer": { "type": "read-only-storage" },
474
+ "elementType": "$yScalar"
475
+ },
476
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" },
477
+ {
478
+ "name": "params",
479
+ "semantic": "kernel.params",
480
+ "buffer": { "type": "uniform" },
481
+ "struct": {
482
+ "name": "Params",
483
+ "fields": [
484
+ { "name": "count", "type": "u32", "value": "elementCount" },
485
+ { "name": "scaleSize", "type": "u32", "value": "scaleSize" },
486
+ { "name": "inner", "type": "u32", "value": "scaleInner" }
487
+ ]
488
+ }
489
+ }
490
+ ],
491
+ "linearScalarNoZero": [
492
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
493
+ {
494
+ "name": "y_scale",
495
+ "arg": "y_scale",
496
+ "semantic": "y_scale",
497
+ "buffer": { "type": "read-only-storage" },
498
+ "elementType": "$scaleScalar"
499
+ },
500
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" },
501
+ {
502
+ "name": "params",
503
+ "semantic": "kernel.params",
504
+ "buffer": { "type": "uniform" },
505
+ "struct": {
506
+ "name": "Params",
507
+ "fields": [
508
+ { "name": "count", "type": "u32", "value": "elementCount" },
509
+ { "name": "scaleSize", "type": "u32", "value": "scaleSize" },
510
+ { "name": "inner", "type": "u32", "value": "scaleInner" }
511
+ ]
512
+ }
513
+ }
514
+ ],
515
+ "blockedLastVec4WithZero": [
516
+ {
517
+ "name": "x",
518
+ "arg": "x",
519
+ "semantic": "x",
520
+ "buffer": { "type": "read-only-storage" },
521
+ "elementType": "vec4<f32>"
522
+ },
523
+ {
524
+ "name": "y_scale",
525
+ "arg": "y_scale",
526
+ "semantic": "y_scale",
527
+ "buffer": { "type": "read-only-storage" },
528
+ "elementType": "f32"
529
+ },
530
+ {
531
+ "name": "y_zero_point",
532
+ "arg": "y_zero_point",
533
+ "semantic": "y_zero_point",
534
+ "buffer": { "type": "read-only-storage" },
535
+ "elementType": "$yScalar"
536
+ },
537
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
538
+ {
539
+ "name": "params",
540
+ "semantic": "kernel.params",
541
+ "buffer": { "type": "uniform" },
542
+ "struct": { "name": "Params", "fields": [{ "name": "count", "type": "u32", "value": "elementCount4" }] }
543
+ }
544
+ ],
545
+ "blockedLastVec4NoZero": [
546
+ {
547
+ "name": "x",
548
+ "arg": "x",
549
+ "semantic": "x",
550
+ "buffer": { "type": "read-only-storage" },
551
+ "elementType": "vec4<f32>"
552
+ },
553
+ {
554
+ "name": "y_scale",
555
+ "arg": "y_scale",
556
+ "semantic": "y_scale",
557
+ "buffer": { "type": "read-only-storage" },
558
+ "elementType": "f32"
559
+ },
560
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
561
+ {
562
+ "name": "params",
563
+ "semantic": "kernel.params",
564
+ "buffer": { "type": "uniform" },
565
+ "struct": { "name": "Params", "fields": [{ "name": "count", "type": "u32", "value": "elementCount4" }] }
566
+ }
567
+ ],
568
+ "blockedVec4WithZero": [
569
+ {
570
+ "name": "x",
571
+ "arg": "x",
572
+ "semantic": "x",
573
+ "buffer": { "type": "read-only-storage" },
574
+ "elementType": "vec4<f32>"
575
+ },
576
+ {
577
+ "name": "y_scale",
578
+ "arg": "y_scale",
579
+ "semantic": "y_scale",
580
+ "buffer": { "type": "read-only-storage" },
581
+ "elementType": "vec4<f32>"
582
+ },
583
+ {
584
+ "name": "y_zero_point",
585
+ "arg": "y_zero_point",
586
+ "semantic": "y_zero_point",
587
+ "buffer": { "type": "read-only-storage" },
588
+ "elementType": "$yVec4"
589
+ },
590
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
591
+ {
592
+ "name": "params",
593
+ "semantic": "kernel.params",
594
+ "buffer": { "type": "uniform" },
595
+ "struct": {
596
+ "name": "Params",
597
+ "fields": [
598
+ { "name": "count", "type": "u32", "value": "elementCount4" },
599
+ { "name": "axisDim", "type": "u32", "value": "blockedAxisDim" },
600
+ { "name": "scaleAxisDim", "type": "u32", "value": "blockedScaleAxisDim" },
601
+ { "name": "inner", "type": "u32", "value": "blockedInner" },
602
+ { "name": "blockSize", "type": "u32", "value": "attrs.block_size" }
603
+ ]
604
+ }
605
+ }
606
+ ],
607
+ "blockedVec4NoZero": [
608
+ {
609
+ "name": "x",
610
+ "arg": "x",
611
+ "semantic": "x",
612
+ "buffer": { "type": "read-only-storage" },
613
+ "elementType": "vec4<f32>"
614
+ },
615
+ {
616
+ "name": "y_scale",
617
+ "arg": "y_scale",
618
+ "semantic": "y_scale",
619
+ "buffer": { "type": "read-only-storage" },
620
+ "elementType": "vec4<f32>"
621
+ },
622
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
623
+ {
624
+ "name": "params",
625
+ "semantic": "kernel.params",
626
+ "buffer": { "type": "uniform" },
627
+ "struct": {
628
+ "name": "Params",
629
+ "fields": [
630
+ { "name": "count", "type": "u32", "value": "elementCount4" },
631
+ { "name": "axisDim", "type": "u32", "value": "blockedAxisDim" },
632
+ { "name": "scaleAxisDim", "type": "u32", "value": "blockedScaleAxisDim" },
633
+ { "name": "inner", "type": "u32", "value": "blockedInner" },
634
+ { "name": "blockSize", "type": "u32", "value": "attrs.block_size" }
635
+ ]
636
+ }
637
+ }
638
+ ],
639
+ "blockedScalarWithZero": [
640
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
641
+ {
642
+ "name": "y_scale",
643
+ "arg": "y_scale",
644
+ "semantic": "y_scale",
645
+ "buffer": { "type": "read-only-storage" },
646
+ "elementType": "$scaleScalar"
647
+ },
648
+ {
649
+ "name": "y_zero_point",
650
+ "arg": "y_zero_point",
651
+ "semantic": "y_zero_point",
652
+ "buffer": { "type": "read-only-storage" },
653
+ "elementType": "$yScalar"
654
+ },
655
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" },
656
+ {
657
+ "name": "params",
658
+ "semantic": "kernel.params",
659
+ "buffer": { "type": "uniform" },
660
+ "struct": {
661
+ "name": "Params",
662
+ "fields": [
663
+ { "name": "count", "type": "u32", "value": "elementCount" },
664
+ { "name": "axisDim", "type": "u32", "value": "blockedAxisDim" },
665
+ { "name": "scaleAxisDim", "type": "u32", "value": "blockedScaleAxisDim" },
666
+ { "name": "inner", "type": "u32", "value": "blockedInner" },
667
+ { "name": "blockSize", "type": "u32", "value": "attrs.block_size" }
668
+ ]
669
+ }
670
+ }
671
+ ],
672
+ "blockedScalarNoZero": [
673
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
674
+ {
675
+ "name": "y_scale",
676
+ "arg": "y_scale",
677
+ "semantic": "y_scale",
678
+ "buffer": { "type": "read-only-storage" },
679
+ "elementType": "$scaleScalar"
680
+ },
681
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" },
682
+ {
683
+ "name": "params",
684
+ "semantic": "kernel.params",
685
+ "buffer": { "type": "uniform" },
686
+ "struct": {
687
+ "name": "Params",
688
+ "fields": [
689
+ { "name": "count", "type": "u32", "value": "elementCount" },
690
+ { "name": "axisDim", "type": "u32", "value": "blockedAxisDim" },
691
+ { "name": "scaleAxisDim", "type": "u32", "value": "blockedScaleAxisDim" },
692
+ { "name": "inner", "type": "u32", "value": "blockedInner" },
693
+ { "name": "blockSize", "type": "u32", "value": "attrs.block_size" }
694
+ ]
695
+ }
696
+ }
697
+ ],
698
+ "linearVec4WithZeroIo": [
699
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
700
+ {
701
+ "name": "y_scale",
702
+ "arg": "y_scale",
703
+ "semantic": "y_scale",
704
+ "buffer": { "type": "read-only-storage" },
705
+ "elementType": "$scaleScalar"
706
+ },
707
+ {
708
+ "name": "y_zero_point",
709
+ "arg": "y_zero_point",
710
+ "semantic": "y_zero_point",
711
+ "buffer": { "type": "read-only-storage" },
712
+ "elementType": "$yScalar"
713
+ },
714
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" }
715
+ ],
716
+ "linearVec4NoZeroIo": [
717
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
718
+ {
719
+ "name": "y_scale",
720
+ "arg": "y_scale",
721
+ "semantic": "y_scale",
722
+ "buffer": { "type": "read-only-storage" },
723
+ "elementType": "$scaleScalar"
724
+ },
725
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" }
726
+ ],
727
+ "linearScalarWithZeroIo": [
728
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
729
+ {
730
+ "name": "y_scale",
731
+ "arg": "y_scale",
732
+ "semantic": "y_scale",
733
+ "buffer": { "type": "read-only-storage" },
734
+ "elementType": "$scaleScalar"
735
+ },
736
+ {
737
+ "name": "y_zero_point",
738
+ "arg": "y_zero_point",
739
+ "semantic": "y_zero_point",
740
+ "buffer": { "type": "read-only-storage" },
741
+ "elementType": "$yScalar"
742
+ },
743
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" }
744
+ ],
745
+ "linearScalarNoZeroIo": [
746
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
747
+ {
748
+ "name": "y_scale",
749
+ "arg": "y_scale",
750
+ "semantic": "y_scale",
751
+ "buffer": { "type": "read-only-storage" },
752
+ "elementType": "$scaleScalar"
753
+ },
754
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" }
755
+ ],
756
+ "linearVec4PerTensorWithZero": [
757
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
758
+ {
759
+ "name": "y_scale",
760
+ "arg": "y_scale",
761
+ "semantic": "y_scale",
762
+ "buffer": { "type": "read-only-storage" },
763
+ "elementType": "$scaleScalar"
764
+ },
765
+ {
766
+ "name": "y_zero_point",
767
+ "arg": "y_zero_point",
768
+ "semantic": "y_zero_point",
769
+ "buffer": { "type": "read-only-storage" },
770
+ "elementType": "$yScalar"
771
+ },
772
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
773
+ {
774
+ "name": "params",
775
+ "semantic": "kernel.params",
776
+ "buffer": { "type": "uniform" },
777
+ "struct": { "name": "Params", "fields": [{ "name": "count4", "type": "u32", "value": "elementCount4" }] }
778
+ }
779
+ ],
780
+ "linearVec4PerTensorNoZero": [
781
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xVec4" },
782
+ {
783
+ "name": "y_scale",
784
+ "arg": "y_scale",
785
+ "semantic": "y_scale",
786
+ "buffer": { "type": "read-only-storage" },
787
+ "elementType": "$scaleScalar"
788
+ },
789
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yVec4" },
790
+ {
791
+ "name": "params",
792
+ "semantic": "kernel.params",
793
+ "buffer": { "type": "uniform" },
794
+ "struct": { "name": "Params", "fields": [{ "name": "count4", "type": "u32", "value": "elementCount4" }] }
795
+ }
796
+ ],
797
+ "linearScalarPerTensorWithZero": [
798
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
799
+ {
800
+ "name": "y_scale",
801
+ "arg": "y_scale",
802
+ "semantic": "y_scale",
803
+ "buffer": { "type": "read-only-storage" },
804
+ "elementType": "$scaleScalar"
805
+ },
806
+ {
807
+ "name": "y_zero_point",
808
+ "arg": "y_zero_point",
809
+ "semantic": "y_zero_point",
810
+ "buffer": { "type": "read-only-storage" },
811
+ "elementType": "$yScalar"
812
+ },
813
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" },
814
+ {
815
+ "name": "params",
816
+ "semantic": "kernel.params",
817
+ "buffer": { "type": "uniform" },
818
+ "struct": { "name": "Params", "fields": [{ "name": "count", "type": "u32", "value": "elementCount" }] }
819
+ }
820
+ ],
821
+ "linearScalarPerTensorNoZero": [
822
+ { "name": "x", "arg": "x", "semantic": "x", "buffer": { "type": "read-only-storage" }, "elementType": "$xScalar" },
823
+ {
824
+ "name": "y_scale",
825
+ "arg": "y_scale",
826
+ "semantic": "y_scale",
827
+ "buffer": { "type": "read-only-storage" },
828
+ "elementType": "$scaleScalar"
829
+ },
830
+ { "name": "y", "arg": "y", "semantic": "y", "buffer": { "type": "storage" }, "elementType": "$yScalar" },
831
+ {
832
+ "name": "params",
833
+ "semantic": "kernel.params",
834
+ "buffer": { "type": "uniform" },
835
+ "struct": { "name": "Params", "fields": [{ "name": "count", "type": "u32", "value": "elementCount" }] }
836
+ }
837
+ ]
838
+ }
839
+ }
build/webgpu/metadata.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "ai.onnx.QuantizeLinear",
3
+ "id": "_ai_onnx_quantizelinear_webgpu_a6a6d0c",
4
+ "version": 1,
5
+ "license": "Apache-2.0",
6
+ "backend": { "type": "webgpu" },
7
+ "digest": {
8
+ "algorithm": "sha256",
9
+ "files": {
10
+ "bench.json": "dX5NKjv8Gz9VheT2QY/86ZiiTTguwW4MpDCtBw9kMHg=",
11
+ "manifest.json": "qDp2PlIs4E2AwjrcPIjItloZ2fWB900UI9LrGCs1PYg=",
12
+ "quant-linear-blocked-axis.wgsl.jinja": "964aieHZ1Jm6oeMgBxT1x9i3tEhFRUFJozYikdEIMwg=",
13
+ "quant-linear-scalar.wgsl.jinja": "191IHMqakZ9HaVXxPSwNl15B6ec70DLhx40LuSbeu30=",
14
+ "quant-linear-vec4.wgsl.jinja": "y48X/TVAaC+b/DD/AQT2ap35JVn7+KrdzlRrxrF8RDA=",
15
+ "test.json": "QV/it95sQgUj+dQu4OK/chmXHLuwBAsDJQc6f7RISfM="
16
+ }
17
+ },
18
+ "provenance": { "kernel": { "sha": "2e7068faf55e7f43df740015f6d1ee49391a41c5", "dirty": false } },
19
+ "webgpu": { "manifestSpec": "1.0", "specialized": true, "opPath": "ops/ai.onnx.QuantizeLinear" }
20
+ }
build/webgpu/quant-linear-blocked-axis.wgsl.jinja ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // Blocked-axis scale indexing for {{ source.op | default("quantize") }}.
2
+ {% set vectorized = source.vectorized if source.vectorized is defined else false %}
3
+ {% set lastAxisVectorized = source.lastAxisVectorized if source.lastAxisVectorized is defined else false %}
4
+ {% if usesF16 %}
5
+ enable f16;
6
+ {% endif %}
7
+ {{ env.wgsl.resourceDeclarations }}
8
+
9
+ {% if not vectorized and not lastAxisVectorized %}
10
+ fn read_zero({% if hasZero %}index: u32{% endif %}) -> i32 {
11
+ {% if hasZero %}
12
+ {% if yUnsigned %}
13
+ return i32(y_zero_point[index]);
14
+ {% else %}
15
+ return y_zero_point[index];
16
+ {% endif %}
17
+ {% else %}
18
+ return 0;
19
+ {% endif %}
20
+ }
21
+
22
+ fn write_quantized(index: u32, value: i32) {
23
+ {% if yUnsigned %}
24
+ y[index] = u32(value);
25
+ {% else %}
26
+ y[index] = value;
27
+ {% endif %}
28
+ }
29
+ {% endif %}
30
+
31
+ {% if not lastAxisVectorized %}
32
+ fn scale_index(index: u32) -> u32 {
33
+ let inner_index = index % params.inner;
34
+ let axis_index = (index / params.inner) % params.axisDim;
35
+ let outer_index = index / (params.axisDim * params.inner);
36
+ return (outer_index * params.scaleAxisDim + axis_index / params.blockSize) * params.inner + inner_index;
37
+ }
38
+
39
+ {% endif %}
40
+ {% set divisionF16 = divisionF16 is defined and divisionF16 %}
41
+ // Exact ONNX QuantizeLinear round-to-nearest-even, with identical handling of
42
+ // infinities, saturation, and halfway values across every kernel route.
43
+ {% if divisionF16 %}
44
+ fn round_quotient_half_to_even(quotient: f32) -> i32 {
45
+ let v = clamp(quotient, -2.0e9, 2.0e9);
46
+ let fl = floor(v);
47
+ let hi = fl + 1.0;
48
+ let fraction = v - fl;
49
+ if (fraction < 0.5) { return i32(fl); }
50
+ if (fraction > 0.5) { return i32(hi); }
51
+ let half = floor(fl * 0.5);
52
+ let is_even = (fl - half * 2.0) == 0.0;
53
+ return i32(select(hi, fl, is_even));
54
+ }
55
+
56
+ fn round_scaled_half_to_even(value: f32, scale: f32) -> i32 {
57
+ // ONNX precision=FLOAT16 (and an omitted precision with f16 y_scale) requires
58
+ // the division itself—not merely its operands—to round in f16.
59
+ return round_quotient_half_to_even(f32(f16(value) / f16(scale)));
60
+ }
61
+ {% else %}
62
+ fn round_scaled_half_to_even(value: f32, scale: f32) -> i32 {
63
+ // Clamp before the i32 cast so infinite and huge finite inputs saturate
64
+ // instead of invoking undefined conversion behavior. Compare distances in
65
+ // the input domain: doing the comparison on value / scale can move a value
66
+ // across a half-way boundary because GPU division is not correctly rounded.
67
+ let v = clamp(value / scale, -2.0e9, 2.0e9);
68
+ let fl = floor(v);
69
+ let lo = fl;
70
+ let hi = fl + 1.0;
71
+ let lo_dist = abs(value - lo * scale);
72
+ let hi_dist = abs(hi * scale - value);
73
+ if (lo_dist < hi_dist) {
74
+ return i32(lo);
75
+ }
76
+ if (hi_dist < lo_dist) {
77
+ return i32(hi);
78
+ }
79
+ let half = floor(fl * 0.5);
80
+ let is_even = (fl - half * 2.0) == 0.0;
81
+ return i32(select(hi, lo, is_even));
82
+ }
83
+ {% endif %}
84
+
85
+
86
+ {% if lastAxisVectorized %}
87
+ @compute @workgroup_size({{ tunables.WORKGROUP_SIZE }})
88
+ fn main(@builtin(global_invocation_id) gid: vec3<u32>, @builtin(num_workgroups) nwg: vec3<u32>) {
89
+ // A vec4 cannot cross a block boundary on this route. Map vector indices
90
+ // directly to their shared scale/zero-point entry.
91
+ let i4 = gid.x + gid.y * nwg.x * {{ tunables.WORKGROUP_SIZE }}u;
92
+ if (i4 >= params.count) { return; }
93
+ let scale_index = i4 / {{ blockVectors }}u;
94
+ let values = x[i4];
95
+ let scale = f32(y_scale[scale_index]);
96
+ {% if hasZero %}
97
+ {% if yUnsigned %}
98
+ let zero = i32(y_zero_point[scale_index]);
99
+ {% else %}
100
+ let zero = y_zero_point[scale_index];
101
+ {% endif %}
102
+ {% else %}
103
+ let zero = 0;
104
+ {% endif %}
105
+ let quantized = vec4<i32>(
106
+ clamp(round_scaled_half_to_even(f32(values.x), scale) + zero, {{ qMin }}, {{ qMax }}),
107
+ clamp(round_scaled_half_to_even(f32(values.y), scale) + zero, {{ qMin }}, {{ qMax }}),
108
+ clamp(round_scaled_half_to_even(f32(values.z), scale) + zero, {{ qMin }}, {{ qMax }}),
109
+ clamp(round_scaled_half_to_even(f32(values.w), scale) + zero, {{ qMin }}, {{ qMax }})
110
+ );
111
+ {% if yUnsigned %}
112
+ y[i4] = vec4<u32>(quantized);
113
+ {% else %}
114
+ y[i4] = quantized;
115
+ {% endif %}
116
+ }
117
+ {% elif vectorized %}
118
+ @compute @workgroup_size({{ tunables.WORKGROUP_SIZE }})
119
+ fn main(@builtin(global_invocation_id) gid: vec3<u32>, @builtin(num_workgroups) nwg: vec3<u32>) {
120
+ // Four adjacent inner elements map to four adjacent scale entries.
121
+ let i4 = gid.x + gid.y * nwg.x * {{ tunables.WORKGROUP_SIZE }}u;
122
+ if (i4 >= params.count) { return; }
123
+ let scale4 = scale_index(i4 * 4u) / 4u;
124
+ let values = x[i4];
125
+ let scales = y_scale[scale4];
126
+ {% if hasZero %}
127
+ {% if yUnsigned %}
128
+ let zero = vec4<i32>(y_zero_point[scale4]);
129
+ {% else %}
130
+ let zero = y_zero_point[scale4];
131
+ {% endif %}
132
+ {% endif %}
133
+ let quantized = vec4<i32>(
134
+ clamp(round_scaled_half_to_even(f32(values.x), f32(scales.x)){% if hasZero %} + zero.x{% endif %}, {{ qMin }}, {{ qMax }}),
135
+ clamp(round_scaled_half_to_even(f32(values.y), f32(scales.y)){% if hasZero %} + zero.y{% endif %}, {{ qMin }}, {{ qMax }}),
136
+ clamp(round_scaled_half_to_even(f32(values.z), f32(scales.z)){% if hasZero %} + zero.z{% endif %}, {{ qMin }}, {{ qMax }}),
137
+ clamp(round_scaled_half_to_even(f32(values.w), f32(scales.w)){% if hasZero %} + zero.w{% endif %}, {{ qMin }}, {{ qMax }})
138
+ );
139
+ {% if yUnsigned %}
140
+ y[i4] = vec4<u32>(quantized);
141
+ {% else %}
142
+ y[i4] = quantized;
143
+ {% endif %}
144
+ }
145
+ {% else %}
146
+ @compute @workgroup_size({{ tunables.WORKGROUP_SIZE }})
147
+ fn main(@builtin(global_invocation_id) gid: vec3<u32>, @builtin(num_workgroups) nwg: vec3<u32>) {
148
+ // The flat dispatch is folded across x/y at the device's per-axis workgroup
149
+ // limit; gid.y carries the high portion of the element index.
150
+ let index = gid.x + gid.y * nwg.x * {{ tunables.WORKGROUP_SIZE }}u;
151
+ if (index >= params.count) { return; }
152
+ let scaleIndex = scale_index(index);
153
+ let rounded = round_scaled_half_to_even(f32(x[index]), f32(y_scale[scaleIndex])) + read_zero({% if hasZero %}scaleIndex{% endif %});
154
+ write_quantized(index, clamp(rounded, {{ qMin }}, {{ qMax }}));
155
+ }
156
+ {% endif %}
build/webgpu/quant-linear-scalar.wgsl.jinja ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // The x4 route handles widened int8/uint8 storage with scalar scale parameters;
2
+ // the linear route supports both per-tensor and per-axis quantization.
3
+ {% if usesF16 %}
4
+ enable f16;
5
+ {% endif %}
6
+ {{ env.wgsl.resourceDeclarations }}
7
+
8
+ {% set divisionF16 = divisionF16 is defined and divisionF16 %}
9
+ // Exact ONNX QuantizeLinear round-to-nearest-even, with identical handling of
10
+ // infinities, saturation, and halfway values across every kernel route.
11
+ {% if divisionF16 %}
12
+ fn round_quotient_half_to_even(quotient: f32) -> i32 {
13
+ let v = clamp(quotient, -2.0e9, 2.0e9);
14
+ let fl = floor(v);
15
+ let hi = fl + 1.0;
16
+ let fraction = v - fl;
17
+ if (fraction < 0.5) { return i32(fl); }
18
+ if (fraction > 0.5) { return i32(hi); }
19
+ let half = floor(fl * 0.5);
20
+ let is_even = (fl - half * 2.0) == 0.0;
21
+ return i32(select(hi, fl, is_even));
22
+ }
23
+
24
+ fn round_scaled_half_to_even(value: f32, scale: f32) -> i32 {
25
+ // ONNX precision=FLOAT16 (and an omitted precision with f16 y_scale) requires
26
+ // the division itself—not merely its operands—to round in f16.
27
+ return round_quotient_half_to_even(f32(f16(value) / f16(scale)));
28
+ }
29
+ {% else %}
30
+ fn round_scaled_half_to_even(value: f32, scale: f32) -> i32 {
31
+ // Clamp before the i32 cast so infinite and huge finite inputs saturate
32
+ // instead of invoking undefined conversion behavior. Compare distances in
33
+ // the input domain: doing the comparison on value / scale can move a value
34
+ // across a half-way boundary because GPU division is not correctly rounded.
35
+ let v = clamp(value / scale, -2.0e9, 2.0e9);
36
+ let fl = floor(v);
37
+ let lo = fl;
38
+ let hi = fl + 1.0;
39
+ let lo_dist = abs(value - lo * scale);
40
+ let hi_dist = abs(hi * scale - value);
41
+ if (lo_dist < hi_dist) {
42
+ return i32(lo);
43
+ }
44
+ if (hi_dist < lo_dist) {
45
+ return i32(hi);
46
+ }
47
+ let half = floor(fl * 0.5);
48
+ let is_even = (fl - half * 2.0) == 0.0;
49
+ return i32(select(hi, lo, is_even));
50
+ }
51
+ {% endif %}
52
+
53
+
54
+ fn read_zero({% if hasZero %}index: u32{% endif %}) -> i32 {
55
+ {% if hasZero %}
56
+ {% if yUnsigned %}
57
+ return i32(y_zero_point[index]);
58
+ {% else %}
59
+ return y_zero_point[index];
60
+ {% endif %}
61
+ {% else %}
62
+ return 0;
63
+ {% endif %}
64
+ }
65
+
66
+ fn write_value(index: u32, value: i32) {
67
+ {% if yUnsigned %}
68
+ y[index] = u32(value);
69
+ {% else %}
70
+ y[index] = value;
71
+ {% endif %}
72
+ }
73
+
74
+ fn transform_one(index: u32, scale: f32, zero_point: i32) {
75
+ let rounded = round_scaled_half_to_even(f32(x[index]), scale) + zero_point;
76
+ write_value(index, clamp(rounded, {{ qMin }}, {{ qMax }}));
77
+ }
78
+
79
+ @compute @workgroup_size({{ tunables.WORKGROUP_SIZE }})
80
+ fn main(@builtin(global_invocation_id) gid: vec3<u32>, @builtin(num_workgroups) nwg: vec3<u32>) {
81
+ // Fold oversized flat dispatches into two dimensions; gid.y carries work
82
+ // beyond the device-capped x dimension.
83
+ let invocation = gid.x + gid.y * nwg.x * {{ tunables.WORKGROUP_SIZE }}u;
84
+ {% if source.x4 %}
85
+ if (invocation != 0u) {
86
+ return;
87
+ }
88
+ let base = params.count - params.count % 4u;
89
+ if (base >= params.count) {
90
+ return;
91
+ }
92
+ let scale = f32(y_scale[0]);
93
+ let zero_point = read_zero({% if hasZero %}0u{% endif %});
94
+ transform_one(base, scale, zero_point);
95
+ if (base + 1u < params.count) { transform_one(base + 1u, scale, zero_point); }
96
+ if (base + 2u < params.count) { transform_one(base + 2u, scale, zero_point); }
97
+ if (base + 3u < params.count) { transform_one(base + 3u, scale, zero_point); }
98
+ {% else %}
99
+ if (invocation >= params.count) {
100
+ return;
101
+ }
102
+ var scale_index = 0u;
103
+ {% if source.perAxis %}
104
+ scale_index = (invocation / params.inner) % params.scaleSize;
105
+ {% endif %}
106
+ let scale = f32(y_scale[scale_index]);
107
+ transform_one(invocation, scale, read_zero({% if hasZero %}scale_index{% endif %}));
108
+ {% endif %}
109
+ }
build/webgpu/quant-linear-vec4.wgsl.jinja ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // int8/uint8 tensors are stored widened (one u32/i32 per element), so the vec4
2
+ // binding gives 128-bit loads/stores of four elements. Per-component arithmetic
3
+ // remains identical to the scalar quantize/dequantize paths.
4
+ {% if usesF16 %}
5
+ enable f16;
6
+ {% endif %}
7
+ {{ env.wgsl.resourceDeclarations }}
8
+
9
+ {% set divisionF16 = divisionF16 is defined and divisionF16 %}
10
+ // Exact ONNX QuantizeLinear round-to-nearest-even, with identical handling of
11
+ // infinities, saturation, and halfway values across every kernel route.
12
+ {% if divisionF16 %}
13
+ fn round_quotient_half_to_even(quotient: f32) -> i32 {
14
+ let v = clamp(quotient, -2.0e9, 2.0e9);
15
+ let fl = floor(v);
16
+ let hi = fl + 1.0;
17
+ let fraction = v - fl;
18
+ if (fraction < 0.5) { return i32(fl); }
19
+ if (fraction > 0.5) { return i32(hi); }
20
+ let half = floor(fl * 0.5);
21
+ let is_even = (fl - half * 2.0) == 0.0;
22
+ return i32(select(hi, fl, is_even));
23
+ }
24
+
25
+ fn round_scaled_half_to_even(value: f32, scale: f32) -> i32 {
26
+ // ONNX precision=FLOAT16 (and an omitted precision with f16 y_scale) requires
27
+ // the division itself—not merely its operands—to round in f16.
28
+ return round_quotient_half_to_even(f32(f16(value) / f16(scale)));
29
+ }
30
+ {% else %}
31
+ fn round_scaled_half_to_even(value: f32, scale: f32) -> i32 {
32
+ // Clamp before the i32 cast so infinite and huge finite inputs saturate
33
+ // instead of invoking undefined conversion behavior. Compare distances in
34
+ // the input domain: doing the comparison on value / scale can move a value
35
+ // across a half-way boundary because GPU division is not correctly rounded.
36
+ let v = clamp(value / scale, -2.0e9, 2.0e9);
37
+ let fl = floor(v);
38
+ let lo = fl;
39
+ let hi = fl + 1.0;
40
+ let lo_dist = abs(value - lo * scale);
41
+ let hi_dist = abs(hi * scale - value);
42
+ if (lo_dist < hi_dist) {
43
+ return i32(lo);
44
+ }
45
+ if (hi_dist < lo_dist) {
46
+ return i32(hi);
47
+ }
48
+ let half = floor(fl * 0.5);
49
+ let is_even = (fl - half * 2.0) == 0.0;
50
+ return i32(select(hi, lo, is_even));
51
+ }
52
+ {% endif %}
53
+
54
+
55
+ {% if source.vectorParams is defined and source.vectorParams %}
56
+ fn read_zero4({% if hasZero %}index: u32{% endif %}) -> vec4<i32> {
57
+ {% if hasZero %}
58
+ {% if yUnsigned %}
59
+ return vec4<i32>(y_zero_point[index]);
60
+ {% else %}
61
+ return y_zero_point[index];
62
+ {% endif %}
63
+ {% else %}
64
+ return vec4<i32>(0);
65
+ {% endif %}
66
+ }
67
+ {% else %}
68
+ fn read_zero({% if hasZero %}index: u32{% endif %}) -> i32 {
69
+ {% if hasZero %}
70
+ {% if yUnsigned %}
71
+ return i32(y_zero_point[index]);
72
+ {% else %}
73
+ return y_zero_point[index];
74
+ {% endif %}
75
+ {% else %}
76
+ return 0;
77
+ {% endif %}
78
+ }
79
+ {% endif %}
80
+
81
+ @compute @workgroup_size({{ tunables.WORKGROUP_SIZE }})
82
+ fn main(@builtin(global_invocation_id) gid: vec3<u32>, @builtin(num_workgroups) nwg: vec3<u32>) {
83
+ // The flat dispatch is folded across x/y at the device's per-axis workgroup
84
+ // limit; gid.y carries the high portion of the vector index.
85
+ let i = gid.x + gid.y * nwg.x * {{ tunables.WORKGROUP_SIZE }}u;
86
+ if (i >= params.count4) {
87
+ return;
88
+ }
89
+ {% if source.vectorParams is defined and source.vectorParams %}
90
+ let scale_index = i % (params.scaleSize / 4u);
91
+ {% elif source.crossingParams is defined and source.crossingParams %}
92
+ let base_index = i * 4u;
93
+ let scale_index0 = (base_index / params.inner) % params.scaleSize;
94
+ let scale_index1 = ((base_index + 1u) / params.inner) % params.scaleSize;
95
+ let scale_index2 = ((base_index + 2u) / params.inner) % params.scaleSize;
96
+ let scale_index3 = ((base_index + 3u) / params.inner) % params.scaleSize;
97
+ {% elif source.perAxis %}
98
+ // inner % 4 == 0, so all 4 lanes share one scale index.
99
+ let scale_index = ((i * 4u) / params.inner) % params.scaleSize;
100
+ {% else %}
101
+ let scale_index = 0u;
102
+ {% endif %}
103
+ let xv = x[i];
104
+ {% if source.vectorParams is defined and source.vectorParams %}
105
+ let zp4 = read_zero4({% if hasZero %}scale_index{% endif %});
106
+ {% elif source.crossingParams is defined and source.crossingParams %}
107
+ let zp0 = read_zero(scale_index0);
108
+ let zp1 = read_zero(scale_index1);
109
+ let zp2 = read_zero(scale_index2);
110
+ let zp3 = read_zero(scale_index3);
111
+ {% else %}
112
+ let zp = read_zero({% if hasZero %}scale_index{% endif %});
113
+ {% endif %}
114
+ {% if source.vectorParams is defined and source.vectorParams %}
115
+ let scale = vec4<f32>(y_scale[scale_index]);
116
+ let q0 = clamp(round_scaled_half_to_even(f32(xv.x), scale.x) + zp4.x, {{ qMin }}, {{ qMax }});
117
+ let q1 = clamp(round_scaled_half_to_even(f32(xv.y), scale.y) + zp4.y, {{ qMin }}, {{ qMax }});
118
+ let q2 = clamp(round_scaled_half_to_even(f32(xv.z), scale.z) + zp4.z, {{ qMin }}, {{ qMax }});
119
+ let q3 = clamp(round_scaled_half_to_even(f32(xv.w), scale.w) + zp4.w, {{ qMin }}, {{ qMax }});
120
+ {% elif source.crossingParams is defined and source.crossingParams %}
121
+ let q0 = clamp(round_scaled_half_to_even(f32(xv.x), f32(y_scale[scale_index0])) + zp0, {{ qMin }}, {{ qMax }});
122
+ let q1 = clamp(round_scaled_half_to_even(f32(xv.y), f32(y_scale[scale_index1])) + zp1, {{ qMin }}, {{ qMax }});
123
+ let q2 = clamp(round_scaled_half_to_even(f32(xv.z), f32(y_scale[scale_index2])) + zp2, {{ qMin }}, {{ qMax }});
124
+ let q3 = clamp(round_scaled_half_to_even(f32(xv.w), f32(y_scale[scale_index3])) + zp3, {{ qMin }}, {{ qMax }});
125
+ {% else %}
126
+ let scale = f32(y_scale[scale_index]);
127
+ let q0 = clamp(round_scaled_half_to_even(f32(xv.x), scale) + zp, {{ qMin }}, {{ qMax }});
128
+ let q1 = clamp(round_scaled_half_to_even(f32(xv.y), scale) + zp, {{ qMin }}, {{ qMax }});
129
+ let q2 = clamp(round_scaled_half_to_even(f32(xv.z), scale) + zp, {{ qMin }}, {{ qMax }});
130
+ let q3 = clamp(round_scaled_half_to_even(f32(xv.w), scale) + zp, {{ qMin }}, {{ qMax }});
131
+ {% endif %}
132
+ {% if yUnsigned %}
133
+ y[i] = vec4<u32>(u32(q0), u32(q1), u32(q2), u32(q3));
134
+ {% else %}
135
+ y[i] = vec4<i32>(q0, q1, q2, q3);
136
+ {% endif %}
137
+ }
build/webgpu/test.json ADDED
@@ -0,0 +1,1327 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "op": "ai.onnx.QuantizeLinear",
3
+ "fixtureArrays": {
4
+ "uint8_axis1_rank4_onnx_backend_input_x": [-162, 10, -100, 232, -20, -50, -76, 0, 0, 252, 32, -44, 245, -485, -960, -270, -375, -470]
5
+ },
6
+ "cases": [
7
+ {
8
+ "name": "dispatch_cliff_vec4_no_zero_point",
9
+ "attrs": { "axis": 1 },
10
+ "inputs": {
11
+ "x": { "dtype": "float32", "shape": [2, 33554432], "data": { "kind": "linspace", "start": -64.0, "end": 64.0 } },
12
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.5] } }
13
+ },
14
+ "outputs": { "y": { "dtype": "uint8", "shape": [2, 33554432], "tolerance": 0 } }
15
+ },
16
+ {
17
+ "name": "uint8_scalar_saturate_round_even",
18
+ "inputs": {
19
+ "x": {
20
+ "dtype": "float32",
21
+ "shape": [7],
22
+ "data": { "kind": "values", "values": [-100.0, -0.25, 0.25, 0.75, 1.25, 63.75, 200.0] }
23
+ },
24
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.5] } },
25
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
26
+ },
27
+ "outputs": { "y": { "dtype": "uint8", "shape": [7] } }
28
+ },
29
+ {
30
+ "name": "uint8_subnormal_scale_vec4_gpu_gap",
31
+ "skipGpu": {
32
+ "category": "permanent",
33
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
34
+ },
35
+ "provenance": {
36
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
37
+ "test": "QuantizeLinearOpTest.Uint8",
38
+ "notes": "Valid positive subnormal scale: quantization should preserve one-LSB steps instead of flushing the scale to zero."
39
+ },
40
+ "inputs": {
41
+ "x": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [0.0, 1e-40, 2e-40, 3e-40] } },
42
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1e-40] } },
43
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [0] } }
44
+ },
45
+ "outputs": { "y": { "dtype": "uint8", "shape": [4], "tolerance": 0 } }
46
+ },
47
+ {
48
+ "name": "uint8_subnormal_scale_nonzero_zero_point_gpu_gap",
49
+ "skipGpu": {
50
+ "category": "permanent",
51
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
52
+ },
53
+ "provenance": {
54
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
55
+ "test": "QuantizeLinearOpTest.Uint8",
56
+ "notes": "Subnormal per-tensor scale with a nonzero zero point should preserve one-LSB signed offsets around the zero point."
57
+ },
58
+ "inputs": {
59
+ "x": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [-1e-40, 0.0, 1e-40, 2e-40] } },
60
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1e-40] } },
61
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
62
+ },
63
+ "outputs": { "y": { "dtype": "uint8", "shape": [4], "tolerance": 0 } }
64
+ },
65
+ {
66
+ "name": "int8_subnormal_scale_preserves_signed_steps_gpu_gap",
67
+ "skipGpu": {
68
+ "category": "permanent",
69
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
70
+ },
71
+ "provenance": {
72
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
73
+ "test": "QuantizeLinearOpTest.Int8",
74
+ "notes": "Signed-output subnormal scale companion: distinct one-LSB signed steps should survive instead of collapsing through zero-scale flushing."
75
+ },
76
+ "inputs": {
77
+ "x": {
78
+ "dtype": "float32",
79
+ "shape": [5],
80
+ "data": { "kind": "values", "values": [-2e-40, -1e-40, 0.0, 1e-40, 2e-40] }
81
+ },
82
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1e-40] } },
83
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [0] } }
84
+ },
85
+ "outputs": {
86
+ "y": {
87
+ "dtype": "int8",
88
+ "shape": [5],
89
+ "tolerance": 0,
90
+ "data": { "kind": "values", "values": [-2, -1, 0, 1, 2] }
91
+ }
92
+ }
93
+ },
94
+ {
95
+ "name": "int8_subnormal_scale_vec4_preserves_signed_steps_gpu_gap",
96
+ "skipGpu": {
97
+ "category": "permanent",
98
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
99
+ },
100
+ "provenance": {
101
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
102
+ "test": "QuantizeLinearOpTest.Int8",
103
+ "notes": "Vec4 signed-output companion for subnormal QuantizeLinear scale handling."
104
+ },
105
+ "inputs": {
106
+ "x": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [-1e-40, 0.0, 1e-40, 2e-40] } },
107
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1e-40] } },
108
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [0] } }
109
+ },
110
+ "outputs": {
111
+ "y": { "dtype": "int8", "shape": [4], "tolerance": 0, "data": { "kind": "values", "values": [-1, 0, 1, 2] } }
112
+ }
113
+ },
114
+ {
115
+ "name": "uint8_axis1_per_channel_subnormal_scale_gpu_gap",
116
+ "skipGpu": {
117
+ "category": "permanent",
118
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
119
+ },
120
+ "provenance": {
121
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
122
+ "test": "QuantizeLinearOpTest.Per_Channel_Axis_Default",
123
+ "notes": "Per-axis companion for subnormal QuantizeLinear scale handling: each channel uses a valid subnormal scale and nonzero zero point."
124
+ },
125
+ "attrs": { "axis": 1 },
126
+ "inputs": {
127
+ "x": {
128
+ "dtype": "float32",
129
+ "shape": [2, 4],
130
+ "data": { "kind": "values", "values": [0.0, 1e-40, -1e-40, 2e-40, 2e-40, -2e-40, 0.0, 1e-40] }
131
+ },
132
+ "y_scale": {
133
+ "dtype": "float32",
134
+ "shape": [4],
135
+ "data": { "kind": "values", "values": [1e-40, 1e-40, 1e-40, 1e-40] }
136
+ },
137
+ "y_zero_point": { "dtype": "uint8", "shape": [4], "data": { "kind": "values", "values": [10, 20, 30, 40] } }
138
+ },
139
+ "outputs": {
140
+ "y": {
141
+ "dtype": "uint8",
142
+ "shape": [2, 4],
143
+ "tolerance": 0,
144
+ "data": { "kind": "values", "values": [10, 21, 29, 42, 12, 18, 30, 41] }
145
+ }
146
+ }
147
+ },
148
+ {
149
+ "name": "uint8_axis0_per_channel_subnormal_scale_vec4_gpu_gap",
150
+ "skipGpu": {
151
+ "category": "permanent",
152
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
153
+ },
154
+ "provenance": {
155
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
156
+ "test": "QuantizeLinearOpTest.Per_Channel_Axis_0",
157
+ "notes": "Vec4 per-axis companion for subnormal QuantizeLinear scale handling: axis=0 has inner size 4, so the vectorized channel path must preserve tiny one-LSB steps."
158
+ },
159
+ "attrs": { "axis": 0 },
160
+ "inputs": {
161
+ "x": {
162
+ "dtype": "float32",
163
+ "shape": [2, 4],
164
+ "data": { "kind": "values", "values": [0.0, 1e-40, 2e-40, 3e-40, 2e-40, -2e-40, 0.0, 1e-40] }
165
+ },
166
+ "y_scale": { "dtype": "float32", "shape": [2], "data": { "kind": "values", "values": [1e-40, 1e-40] } },
167
+ "y_zero_point": { "dtype": "uint8", "shape": [2], "data": { "kind": "values", "values": [10, 20] } }
168
+ },
169
+ "outputs": {
170
+ "y": {
171
+ "dtype": "uint8",
172
+ "shape": [2, 4],
173
+ "tolerance": 0,
174
+ "data": { "kind": "values", "values": [10, 11, 12, 13, 22, 18, 20, 21] }
175
+ }
176
+ }
177
+ },
178
+ {
179
+ "name": "uint8_axis1_per_channel_subnormal_scale_no_zero_point_gpu_gap",
180
+ "skipGpu": {
181
+ "category": "permanent",
182
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
183
+ },
184
+ "provenance": {
185
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
186
+ "test": "QuantizeLinearOpTest.QuantizeLinear_Without_Zero_Point_Opset13",
187
+ "notes": "Per-axis extension of omitted-zero-point coverage: default uint8 zero point is zero, but each channel still uses a valid subnormal scale."
188
+ },
189
+ "attrs": { "axis": 1 },
190
+ "inputs": {
191
+ "x": {
192
+ "dtype": "float32",
193
+ "shape": [2, 4],
194
+ "data": { "kind": "values", "values": [0.0, 1e-40, 2e-40, 3e-40, 2e-40, 0.0, 1e-40, -1e-40] }
195
+ },
196
+ "y_scale": {
197
+ "dtype": "float32",
198
+ "shape": [4],
199
+ "data": { "kind": "values", "values": [1e-40, 1e-40, 1e-40, 1e-40] }
200
+ }
201
+ },
202
+ "outputs": {
203
+ "y": {
204
+ "dtype": "uint8",
205
+ "shape": [2, 4],
206
+ "tolerance": 0,
207
+ "data": { "kind": "values", "values": [0, 1, 2, 3, 2, 0, 1, 0] }
208
+ }
209
+ }
210
+ },
211
+ {
212
+ "name": "uint8_axis0_per_channel_subnormal_scale_no_zero_point_vec4_gpu_gap",
213
+ "skipGpu": {
214
+ "category": "permanent",
215
+ "reason": "Portable WGSL floating-point semantics do not guarantee preservation of the subnormal scale values required by this fixture. Backend evidence: Metal flushes denormals in floating-point division (1e-40/1e-40 -> NaN); the CPU reference preserves the subnormal scale, so subnormal y_scale cases remain CPU-reference-only."
216
+ },
217
+ "provenance": {
218
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
219
+ "test": "QuantizeLinearOpTest.QuantizeLinear_Without_Zero_Point_Opset13",
220
+ "notes": "Vec4 per-axis extension of omitted-zero-point coverage with valid subnormal scales."
221
+ },
222
+ "attrs": { "axis": 0 },
223
+ "inputs": {
224
+ "x": {
225
+ "dtype": "float32",
226
+ "shape": [2, 4],
227
+ "data": { "kind": "values", "values": [0.0, 1e-40, 2e-40, 3e-40, 2e-40, 0.0, -1e-40, 1e-40] }
228
+ },
229
+ "y_scale": { "dtype": "float32", "shape": [2], "data": { "kind": "values", "values": [1e-40, 1e-40] } }
230
+ },
231
+ "outputs": {
232
+ "y": {
233
+ "dtype": "uint8",
234
+ "shape": [2, 4],
235
+ "tolerance": 0,
236
+ "data": { "kind": "values", "values": [0, 1, 2, 3, 2, 0, 0, 1] }
237
+ }
238
+ }
239
+ },
240
+ {
241
+ "name": "uint8_huge_finite_saturates_before_i32_overflow",
242
+ "provenance": {
243
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
244
+ "test": "QuantizeLinearOpTest.Uint8",
245
+ "notes": "Finite values far outside the uint8 range should saturate; kernels must avoid converting an out-of-range rounded float to i32 before clamping."
246
+ },
247
+ "inputs": {
248
+ "x": {
249
+ "dtype": "float32",
250
+ "shape": [2],
251
+ "data": { "kind": "values", "values": [100000000000000000000.0, -100000000000000000000.0] }
252
+ },
253
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1.0] } },
254
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
255
+ },
256
+ "outputs": {
257
+ "y": { "dtype": "uint8", "shape": [2], "tolerance": 0, "data": { "kind": "values", "values": [255, 0] } }
258
+ }
259
+ },
260
+ {
261
+ "name": "uint8_infinities_saturate_before_i32_conversion",
262
+ "provenance": {
263
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
264
+ "test": "QuantizeLinearOpTest.Uint8",
265
+ "notes": "Infinite inputs are valid float tensor values and should saturate to quantized bounds without first converting infinity to i32."
266
+ },
267
+ "inputs": {
268
+ "x": {
269
+ "dtype": "float32",
270
+ "shape": [3],
271
+ "data": { "kind": "values", "values": ["Infinity", "-Infinity", 0.0] }
272
+ },
273
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1.0] } },
274
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
275
+ },
276
+ "outputs": {
277
+ "y": { "dtype": "uint8", "shape": [3], "tolerance": 0, "data": { "kind": "values", "values": [255, 0, 128] } }
278
+ }
279
+ },
280
+ {
281
+ "name": "uint8_infinities_saturate_vec4_before_i32_conversion",
282
+ "provenance": {
283
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
284
+ "test": "QuantizeLinearOpTest.Uint8",
285
+ "notes": "Vec4 zero-point path: infinite inputs should saturate to uint8 bounds without converting infinity to i32 first."
286
+ },
287
+ "inputs": {
288
+ "x": {
289
+ "dtype": "float32",
290
+ "shape": [4],
291
+ "data": { "kind": "values", "values": ["Infinity", "-Infinity", 0.0, 1.0] }
292
+ },
293
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1.0] } },
294
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
295
+ },
296
+ "outputs": {
297
+ "y": {
298
+ "dtype": "uint8",
299
+ "shape": [4],
300
+ "tolerance": 0,
301
+ "data": { "kind": "values", "values": [255, 0, 128, 129] }
302
+ }
303
+ }
304
+ },
305
+ {
306
+ "name": "int8_axis1_f16",
307
+ "attrs": { "axis": 1 },
308
+ "inputs": {
309
+ "x": {
310
+ "dtype": "float16",
311
+ "shape": [2, 4],
312
+ "data": { "kind": "values", "values": [-10.0, -0.5, 0.5, 1.5, 2.5, 4.0, 8.0, 64.0] }
313
+ },
314
+ "y_scale": { "dtype": "float16", "shape": [4], "data": { "kind": "values", "values": [0.5, 1.0, 2.0, 0.25] } },
315
+ "y_zero_point": { "dtype": "int8", "shape": [4], "data": { "kind": "values", "values": [0, -2, 1, 3] } }
316
+ },
317
+ "outputs": { "y": { "dtype": "int8", "shape": [2, 4] } }
318
+ },
319
+ {
320
+ "name": "uint8_no_zero_negative_axis",
321
+ "attrs": { "axis": -1 },
322
+ "inputs": {
323
+ "x": { "dtype": "float32", "shape": [2, 2], "data": { "kind": "values", "values": [0.0, 0.5, 2.0, 3.0] } },
324
+ "y_scale": { "dtype": "float32", "shape": [2], "data": { "kind": "values", "values": [0.5, 1.0] } }
325
+ },
326
+ "outputs": { "y": { "dtype": "uint8", "shape": [2, 2] } }
327
+ },
328
+ {
329
+ "name": "ort_int16_round_even_and_saturate_gpu_gap",
330
+ "skipGpu": {
331
+ "category": "todo",
332
+ "reason": "The QuantizeLinear kernels only implement uint8/int8 output clamps; int16 needs round-to-even plus saturation to [-32768, 32767] before the standard route can be enabled."
333
+ },
334
+ "provenance": {
335
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
336
+ "test": "QuantizeLinearOpTest.Int16"
337
+ },
338
+ "inputs": {
339
+ "x": {
340
+ "dtype": "float32",
341
+ "shape": [16],
342
+ "data": {
343
+ "kind": "values",
344
+ "values": [0.0, -514.0, 3.0, -3.0, 2.9, -2.9, 3.1, -3.1, 65022.0, -66046.0, 65023.0, -66047.0, 65024.0, -66048.0, 70000.0, -70000.0]
345
+ }
346
+ },
347
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.0] } },
348
+ "y_zero_point": { "dtype": "int16", "shape": [], "data": { "kind": "values", "values": [256] } }
349
+ },
350
+ "outputs": { "y": { "dtype": "int16", "shape": [16], "tolerance": 0 } }
351
+ },
352
+ {
353
+ "name": "uint8_scalar_onnx_backend",
354
+ "provenance": {
355
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
356
+ "test": "QuantizeLinearOpTest.Uint8"
357
+ },
358
+ "inputs": {
359
+ "x": {
360
+ "dtype": "float32",
361
+ "shape": [6],
362
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 1000.0, -254.0, -1000.0] }
363
+ },
364
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [2.0] } },
365
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
366
+ },
367
+ "outputs": { "y": { "dtype": "uint8", "shape": [6] } }
368
+ },
369
+ {
370
+ "name": "ort_uint8_rank0_scale_zero_point",
371
+ "provenance": {
372
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
373
+ "test": "QuantizeLinearOpTest.Uint8"
374
+ },
375
+ "inputs": {
376
+ "x": {
377
+ "dtype": "float32",
378
+ "shape": [6],
379
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 1000.0, -254.0, -1000.0] }
380
+ },
381
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.0] } },
382
+ "y_zero_point": { "dtype": "uint8", "shape": [], "data": { "kind": "values", "values": [128] } }
383
+ },
384
+ "outputs": { "y": { "dtype": "uint8", "shape": [6] } }
385
+ },
386
+ {
387
+ "name": "ort_int8_rank0_scale_zero_point",
388
+ "provenance": {
389
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
390
+ "test": "QuantizeLinearOpTest.Int8"
391
+ },
392
+ "inputs": {
393
+ "x": {
394
+ "dtype": "float32",
395
+ "shape": [6],
396
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 5.0, -2.0, -5.0] }
397
+ },
398
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [0.039215686] } },
399
+ "y_zero_point": { "dtype": "int8", "shape": [], "data": { "kind": "values", "values": [0] } }
400
+ },
401
+ "outputs": { "y": { "dtype": "int8", "shape": [6] } }
402
+ },
403
+ {
404
+ "name": "ort_scalar_input_rank0_with_zero_point",
405
+ "provenance": {
406
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
407
+ "test": "QuantizeLinearOpTest.Scalar"
408
+ },
409
+ "inputs": {
410
+ "x": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [3.0] } },
411
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.0] } },
412
+ "y_zero_point": { "dtype": "uint8", "shape": [], "data": { "kind": "values", "values": [128] } }
413
+ },
414
+ "outputs": { "y": { "dtype": "uint8", "shape": [] } }
415
+ },
416
+ {
417
+ "name": "ort_scalar_input_rank0_no_zero_point",
418
+ "provenance": {
419
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
420
+ "test": "QuantizeLinearOpTest.QuantizeLinear_Without_Zero_Point_Opset13"
421
+ },
422
+ "inputs": {
423
+ "x": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [3.0] } },
424
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.0] } }
425
+ },
426
+ "outputs": { "y": { "dtype": "uint8", "shape": [] } }
427
+ },
428
+ {
429
+ "name": "ort_scalar_input_rank0_zero_point_zero",
430
+ "provenance": {
431
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
432
+ "test": "QuantizeLinearOpTest.QuantizeLinear_With_Zero_Point0"
433
+ },
434
+ "inputs": {
435
+ "x": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [3.0] } },
436
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.0] } },
437
+ "y_zero_point": { "dtype": "uint8", "shape": [], "data": { "kind": "values", "values": [0] } }
438
+ },
439
+ "outputs": { "y": { "dtype": "uint8", "shape": [] } }
440
+ },
441
+ {
442
+ "name": "ort_singleton_input_no_zero_point",
443
+ "provenance": {
444
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
445
+ "test": "QuantizeLinearOpTest.QuantizeLinear_With_Zero_Dim1"
446
+ },
447
+ "inputs": {
448
+ "x": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [3.0] } },
449
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [2.0] } }
450
+ },
451
+ "outputs": { "y": { "dtype": "uint8", "shape": [1] } }
452
+ },
453
+ {
454
+ "name": "scalar_x4_int8_with_zero_tail",
455
+ "inputs": {
456
+ "x": {
457
+ "dtype": "float32",
458
+ "shape": [17],
459
+ "data": {
460
+ "kind": "values",
461
+ "values": [-64.0, -32.0, -16.0, -8.0, -4.0, -2.0, -1.0, 0.0, 1.0, 2.0, 4.0, 8.0, 16.0, 32.0, 48.0, 64.0, 96.0]
462
+ }
463
+ },
464
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.5] } },
465
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [3] } }
466
+ },
467
+ "outputs": { "y": { "dtype": "int8", "shape": [17] } }
468
+ },
469
+ {
470
+ "name": "scalar_x4_uint8_no_zero_tail",
471
+ "inputs": {
472
+ "x": {
473
+ "dtype": "float32",
474
+ "shape": [17],
475
+ "data": {
476
+ "kind": "values",
477
+ "values": [0.0, 0.25, 0.5, 0.75, 1.0, 1.25, 2.0, 4.0, 8.0, 16.0, 32.0, 64.0, 128.0, 256.0, 384.0, 512.0, 1024.0]
478
+ }
479
+ },
480
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [2.0] } }
481
+ },
482
+ "outputs": { "y": { "dtype": "uint8", "shape": [17] } }
483
+ },
484
+ {
485
+ "name": "vec4_tail_int8_with_zero_4097",
486
+ "inputs": {
487
+ "x": { "dtype": "float32", "shape": [4097], "data": { "kind": "constant", "value": 1.25 } },
488
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.5] } },
489
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [3] } }
490
+ },
491
+ "outputs": { "y": { "dtype": "int8", "shape": [4097] } },
492
+ "provenance": { "notes": "Covers the packed bulk plus scalar tail with a scalar zero point." }
493
+ },
494
+ {
495
+ "name": "vec4_tail_uint8_no_zero_4097",
496
+ "inputs": {
497
+ "x": { "dtype": "float32", "shape": [4097], "data": { "kind": "constant", "value": 4.0 } },
498
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [2.0] } }
499
+ },
500
+ "outputs": { "y": { "dtype": "uint8", "shape": [4097] } },
501
+ "provenance": { "notes": "Covers the packed bulk plus scalar tail without a zero point." }
502
+ },
503
+ {
504
+ "name": "uint8_axis1_rank4_onnx_backend",
505
+ "attrs": { "axis": 1 },
506
+ "inputs": {
507
+ "x": {
508
+ "dtype": "float32",
509
+ "shape": [1, 3, 3, 2],
510
+ "data": { "kind": "values", "values": { "$ref": "#/fixtureArrays/uint8_axis1_rank4_onnx_backend_input_x" } }
511
+ },
512
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [2.0, 4.0, 5.0] } },
513
+ "y_zero_point": { "dtype": "uint8", "shape": [3], "data": { "kind": "values", "values": [84, 24, 196] } }
514
+ },
515
+ "outputs": { "y": { "dtype": "uint8", "shape": [1, 3, 3, 2] } }
516
+ },
517
+ {
518
+ "name": "int8_scalar_zero_point_saturate_round_even",
519
+ "inputs": {
520
+ "x": {
521
+ "dtype": "float32",
522
+ "shape": [9],
523
+ "data": { "kind": "values", "values": [-100.0, -64.0, -63.5, -0.5, 0.0, 0.5, 63.5, 64.0, 100.0] }
524
+ },
525
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.5] } },
526
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [0] } }
527
+ },
528
+ "outputs": { "y": { "dtype": "int8", "shape": [9] } }
529
+ },
530
+ {
531
+ "name": "ort_int8_negative_zero_point_formulation",
532
+ "provenance": {
533
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
534
+ "test": "QuantizeLinearOpTest.Int8_NegativeZeroPoint"
535
+ },
536
+ "inputs": {
537
+ "x": {
538
+ "dtype": "float32",
539
+ "shape": [8],
540
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 5.0, 6.0, -2.0, -5.0, -6.0] }
541
+ },
542
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.039215686] } },
543
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [-23] } }
544
+ },
545
+ "outputs": { "y": { "dtype": "int8", "shape": [8] } }
546
+ },
547
+ {
548
+ "name": "ort_int8_positive_zero_point_formulation",
549
+ "provenance": {
550
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
551
+ "test": "QuantizeLinearOpTest.Int8_PositiveZeroPoint"
552
+ },
553
+ "inputs": {
554
+ "x": {
555
+ "dtype": "float32",
556
+ "shape": [8],
557
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 5.0, 6.0, -2.0, -5.0, -6.0] }
558
+ },
559
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.039215686] } },
560
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [23] } }
561
+ },
562
+ "outputs": { "y": { "dtype": "int8", "shape": [8] } }
563
+ },
564
+ {
565
+ "name": "uint8_axis0_rank2_per_row",
566
+ "attrs": { "axis": 0 },
567
+ "inputs": {
568
+ "x": {
569
+ "dtype": "float32",
570
+ "shape": [3, 2],
571
+ "data": { "kind": "values", "values": [-2.0, 2.0, 20.0, 21.5, -20.0, 200.0] }
572
+ },
573
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [0.5, 1.0, 2.0] } },
574
+ "y_zero_point": { "dtype": "uint8", "shape": [3], "data": { "kind": "values", "values": [10, 20, 30] } }
575
+ },
576
+ "outputs": { "y": { "dtype": "uint8", "shape": [3, 2] } }
577
+ },
578
+ {
579
+ "name": "int8_negative_axis_rank3_per_last_dim",
580
+ "attrs": { "axis": -1 },
581
+ "inputs": {
582
+ "x": {
583
+ "dtype": "float32",
584
+ "shape": [2, 2, 3],
585
+ "data": { "kind": "values", "values": [-4.0, -1.0, 0.0, 1.0, 2.5, 4.0, 8.0, -8.0, 0.5, -0.5, 63.0, -300.0] }
586
+ },
587
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [1.0, 0.5, 2.0] } },
588
+ "y_zero_point": { "dtype": "int8", "shape": [3], "data": { "kind": "values", "values": [-1, 0, 1] } }
589
+ },
590
+ "outputs": { "y": { "dtype": "int8", "shape": [2, 2, 3] } }
591
+ },
592
+ {
593
+ "name": "ort_f16_uint8_scalar",
594
+ "provenance": {
595
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
596
+ "test": "QuantizeLinearOpMLFloat16Test.Uint8"
597
+ },
598
+ "inputs": {
599
+ "x": {
600
+ "dtype": "float16",
601
+ "shape": [6],
602
+ "data": { "kind": "values", "values": [0.0, 2.0, 4.0, 1000.0, -254.0, -1000.0] }
603
+ },
604
+ "y_scale": { "dtype": "float16", "shape": [1], "data": { "kind": "values", "values": [2.0] } },
605
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
606
+ },
607
+ "outputs": { "y": { "dtype": "uint8", "shape": [6] } }
608
+ },
609
+ {
610
+ "name": "ort_opset25_f16_int8_axis1",
611
+ "provenance": {
612
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
613
+ "test": "QuantizeLinearOpMLFloat16Test.Opset25_PerAxisInt8_Cuda"
614
+ },
615
+ "attrs": { "axis": 1 },
616
+ "inputs": {
617
+ "x": {
618
+ "dtype": "float16",
619
+ "shape": [2, 4],
620
+ "data": { "kind": "values", "values": [-4.0, -2.0, 0.0, 2.0, 4.0, 6.0, 8.0, 10.0] }
621
+ },
622
+ "y_scale": { "dtype": "float16", "shape": [4], "data": { "kind": "values", "values": [2.0, 2.0, 4.0, 4.0] } },
623
+ "y_zero_point": { "dtype": "int8", "shape": [4], "data": { "kind": "values", "values": [0, 0, 0, 0] } }
624
+ },
625
+ "outputs": { "y": { "dtype": "int8", "shape": [2, 4], "tolerance": 0 } }
626
+ },
627
+ {
628
+ "name": "ort_int8_5d_per_tensor",
629
+ "provenance": {
630
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
631
+ "test": "QuantizeLinearOpTest.Int8_5D_DML_TypeMismatch"
632
+ },
633
+ "inputs": {
634
+ "x": {
635
+ "dtype": "float32",
636
+ "shape": [6, 1, 1, 1, 1],
637
+ "data": { "kind": "values", "values": [1.0, 2.0, 3.0, 4.0, 5.0, 6.0] }
638
+ },
639
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1.0] } },
640
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [0] } }
641
+ },
642
+ "outputs": { "y": { "dtype": "int8", "shape": [6, 1, 1, 1, 1] } }
643
+ },
644
+ {
645
+ "name": "ort_int8_5d_rank0_zero_point_opset21",
646
+ "provenance": {
647
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
648
+ "test": "QuantizeLinearOpTest.Int8_5D_WithZeroPoint_Opset21_DML"
649
+ },
650
+ "inputs": {
651
+ "x": {
652
+ "dtype": "float32",
653
+ "shape": [6, 1, 1, 1, 1],
654
+ "data": { "kind": "values", "values": [1.0, 2.0, 3.0, 4.0, 5.0, 6.0] }
655
+ },
656
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [1.0] } },
657
+ "y_zero_point": { "dtype": "int8", "shape": [], "data": { "kind": "values", "values": [0] } }
658
+ },
659
+ "outputs": { "y": { "dtype": "int8", "shape": [6, 1, 1, 1, 1], "tolerance": 0 } }
660
+ },
661
+ {
662
+ "name": "ort_int8_5d_per_axis_axis0",
663
+ "provenance": {
664
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
665
+ "test": "QuantizeLinearOpTest.Int8_5D_PerAxis_DML_TypeMismatch"
666
+ },
667
+ "attrs": { "axis": 0 },
668
+ "inputs": {
669
+ "x": {
670
+ "dtype": "float32",
671
+ "shape": [6, 1, 1, 1, 1],
672
+ "data": { "kind": "values", "values": [1.0, 2.0, 3.0, 4.0, 5.0, 6.0] }
673
+ },
674
+ "y_scale": {
675
+ "dtype": "float32",
676
+ "shape": [6],
677
+ "data": { "kind": "values", "values": [1.0, 1.0, 1.0, 1.0, 1.0, 1.0] }
678
+ },
679
+ "y_zero_point": { "dtype": "int8", "shape": [6], "data": { "kind": "values", "values": [0, 0, 0, 0, 0, 0] } }
680
+ },
681
+ "outputs": { "y": { "dtype": "int8", "shape": [6, 1, 1, 1, 1] } }
682
+ },
683
+ {
684
+ "name": "ort_uint8_5d_no_zero_point",
685
+ "provenance": {
686
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
687
+ "test": "QuantizeLinearOpTest.Uint8_5D_NoZeroPoint_Opset21_DML"
688
+ },
689
+ "inputs": {
690
+ "x": {
691
+ "dtype": "float32",
692
+ "shape": [6, 1, 1, 1, 1],
693
+ "data": { "kind": "values", "values": [0.0, 51.0, 102.0, 153.0, 204.0, 255.0] }
694
+ },
695
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [1.0] } }
696
+ },
697
+ "outputs": { "y": { "dtype": "uint8", "shape": [6, 1, 1, 1, 1] } }
698
+ },
699
+ {
700
+ "name": "ort_uint8_2d_scalar_quantization",
701
+ "provenance": {
702
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
703
+ "test": "QuantizeLinearOpTest.2D"
704
+ },
705
+ "inputs": {
706
+ "x": {
707
+ "dtype": "float32",
708
+ "shape": [3, 4],
709
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 1000.0, 0.0, 2.0, 3.0, 1000.0, 0.0, 2.0, 3.0, 1000.0] }
710
+ },
711
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [4.0] } },
712
+ "y_zero_point": { "dtype": "uint8", "shape": [], "data": { "kind": "values", "values": [0] } }
713
+ },
714
+ "outputs": { "y": { "dtype": "uint8", "shape": [3, 4] } }
715
+ },
716
+ {
717
+ "name": "ort_uint8_per_channel_default_axis",
718
+ "provenance": {
719
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
720
+ "test": "QuantizeLinearOpTest.Per_Channel_Axis_Default"
721
+ },
722
+ "inputs": {
723
+ "x": {
724
+ "dtype": "float32",
725
+ "shape": [3, 4],
726
+ "data": { "kind": "values", "values": [0.0, 2.0, 1.0, 1001.0, 1.0, 1.0, 2.0, 1100.0, 2.0, 4.2, 3.0, 1200.0] }
727
+ },
728
+ "y_scale": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [1.0, 2.0, 3.0, 20.0] } },
729
+ "y_zero_point": { "dtype": "uint8", "shape": [4], "data": { "kind": "values", "values": [64, 100, 127, 127] } }
730
+ },
731
+ "outputs": { "y": { "dtype": "uint8", "shape": [3, 4] } }
732
+ },
733
+ {
734
+ "name": "uint8_axis1_per_channel_innermost_vec4_no_zero_point",
735
+ "provenance": {
736
+ "notes": "Per-axis scale on the innermost axis with the zero point omitted, so the vec4-bound scale path runs with the implicit zero. Each channel uses a different scale and the first column ties on .5 in both directions, so a lane reading the wrong channel or rounding half-away-from-zero changes the output."
737
+ },
738
+ "attrs": { "axis": 1 },
739
+ "inputs": {
740
+ "x": {
741
+ "dtype": "float32",
742
+ "shape": [3, 4],
743
+ "data": {
744
+ "kind": "values",
745
+ "values": [1.25, 6.0, 20.0, 3.0, 3.75, 10.0, 36.0, 7.25, 50.0, 90.0, 172.0, 15.75]
746
+ }
747
+ },
748
+ "y_scale": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [0.5, 2.0, 4.0, 0.25] } }
749
+ },
750
+ "outputs": { "y": { "dtype": "uint8", "shape": [3, 4], "tolerance": 0 } }
751
+ },
752
+ {
753
+ "name": "ort_uint8_per_channel_axis0",
754
+ "provenance": {
755
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
756
+ "test": "QuantizeLinearOpTest.Per_Channel_Axis_0"
757
+ },
758
+ "attrs": { "axis": 0 },
759
+ "inputs": {
760
+ "x": {
761
+ "dtype": "float32",
762
+ "shape": [3, 4],
763
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 1000.0, 0.0, 2.0, 3.0, 1000.0, 0.0, 2.0, 3.0, 1000.0] }
764
+ },
765
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [1.0, 2.0, 4.0] } },
766
+ "y_zero_point": { "dtype": "uint8", "shape": [3], "data": { "kind": "values", "values": [0, 0, 0] } }
767
+ },
768
+ "outputs": { "y": { "dtype": "uint8", "shape": [3, 4] } }
769
+ },
770
+ {
771
+ "name": "ort_uint8_per_channel_negative_axis_minus2",
772
+ "provenance": {
773
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
774
+ "test": "QuantizeLinearOpTest.Per_Channel_Axis_neg"
775
+ },
776
+ "attrs": { "axis": -2 },
777
+ "inputs": {
778
+ "x": {
779
+ "dtype": "float32",
780
+ "shape": [3, 4],
781
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 1000.0, 0.0, 2.0, 3.0, 1000.0, 0.0, 2.0, 3.0, 1000.0] }
782
+ },
783
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [1.0, 2.0, 4.0] } },
784
+ "y_zero_point": { "dtype": "uint8", "shape": [3], "data": { "kind": "values", "values": [0, 0, 0] } }
785
+ },
786
+ "outputs": { "y": { "dtype": "uint8", "shape": [3, 4] } }
787
+ },
788
+ {
789
+ "name": "onnx_backend_quantizelinear",
790
+ "provenance": { "source": "cmake/external/onnx/onnx/backend/test/data/node/test_quantizelinear" },
791
+ "inputs": {
792
+ "x": {
793
+ "dtype": "float32",
794
+ "shape": [6],
795
+ "data": { "kind": "values", "values": [0.0, 2.0, 3.0, 1000.0, -254.0, -1000.0] }
796
+ },
797
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.0] } },
798
+ "y_zero_point": { "dtype": "uint8", "shape": [], "data": { "kind": "values", "values": [128] } }
799
+ },
800
+ "outputs": { "y": { "dtype": "uint8", "shape": [6], "tolerance": 0 } }
801
+ },
802
+ {
803
+ "name": "onnx_backend_quantizelinear_axis",
804
+ "provenance": { "source": "cmake/external/onnx/onnx/backend/test/data/node/test_quantizelinear_axis" },
805
+ "inputs": {
806
+ "x": {
807
+ "dtype": "float32",
808
+ "shape": [1, 3, 3, 2],
809
+ "data": { "kind": "values", "values": { "$ref": "#/fixtureArrays/uint8_axis1_rank4_onnx_backend_input_x" } }
810
+ },
811
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [2.0, 4.0, 5.0] } },
812
+ "y_zero_point": { "dtype": "uint8", "shape": [3], "data": { "kind": "values", "values": [84, 24, 196] } }
813
+ },
814
+ "outputs": { "y": { "dtype": "uint8", "shape": [1, 3, 3, 2], "tolerance": 0 } }
815
+ },
816
+ {
817
+ "name": "onnx_backend_quantizelinear_blocked_asymmetric",
818
+ "provenance": {
819
+ "source": "cmake/external/onnx/onnx/backend/test/data/node/test_quantizelinear_blocked_asymmetric"
820
+ },
821
+ "attrs": { "axis": 1, "block_size": 2 },
822
+ "inputs": {
823
+ "x": {
824
+ "dtype": "float32",
825
+ "shape": [3, 4],
826
+ "data": { "kind": "values", "values": [6.0, 12.0, 50.0, 5.0, 1.0, 8.0, 4.0, 5.0, 0.0, 20.0, 10.0, 4.0] }
827
+ },
828
+ "y_scale": {
829
+ "dtype": "float32",
830
+ "shape": [3, 2],
831
+ "data": {
832
+ "kind": "values",
833
+ "values": [1.5, 2.5, 3.0, 4.900000095367432, 5.099999904632568, 6.900000095367432]
834
+ }
835
+ },
836
+ "y_zero_point": {
837
+ "dtype": "uint8",
838
+ "shape": [3, 2],
839
+ "data": { "kind": "values", "values": [0, 1, 1, 0, 2, 3] }
840
+ }
841
+ },
842
+ "outputs": { "y": { "dtype": "uint8", "shape": [3, 4], "tolerance": 0 } }
843
+ },
844
+ {
845
+ "name": "vec4_uint8_no_zero_point_4x8",
846
+ "inputs": {
847
+ "x": {
848
+ "dtype": "float32",
849
+ "shape": [4, 8],
850
+ "data": {
851
+ "kind": "values",
852
+ "values": [0.0, 2.0, 3.0, 6.0, 10.0, 14.0, 18.0, 22.0, 1.999, 2.001, 5.999, 6.001, 100.0, 250.0, 500.0, 750.0, 1000.0, 1016.0, 1018.0, 1020.0, 1022.0, 1024.0, 2000.0, 5000.0, -1.0, -100.0, 0.5, 1.5, 2.5, 3.5, 511.0, 513.0]
853
+ }
854
+ },
855
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [4.0] } }
856
+ },
857
+ "outputs": { "y": { "dtype": "uint8", "shape": [4, 8] } }
858
+ },
859
+ {
860
+ "name": "vec4_uint8_no_zero_point_per_axis0",
861
+ "attrs": { "axis": 0 },
862
+ "inputs": {
863
+ "x": {
864
+ "dtype": "float32",
865
+ "shape": [2, 2, 4],
866
+ "data": {
867
+ "kind": "values",
868
+ "values": [0.0, 0.25, 0.5, 0.75, 1.25, 63.75, 127.5, 200.0, 0.0, 2.0, 3.0, 6.0, 10.0, 250.0, 510.0, 1000.0]
869
+ }
870
+ },
871
+ "y_scale": { "dtype": "float32", "shape": [2], "data": { "kind": "values", "values": [0.5, 2.0] } }
872
+ },
873
+ "outputs": { "y": { "dtype": "uint8", "shape": [2, 2, 4] } }
874
+ },
875
+ {
876
+ "name": "ort_uint8_f16_scalar_scale_zero_point",
877
+ "provenance": {
878
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
879
+ "test": "QuantizeLinearOpMLFloat16Test.Uint8"
880
+ },
881
+ "inputs": {
882
+ "x": {
883
+ "dtype": "float16",
884
+ "shape": [6],
885
+ "data": { "kind": "values", "values": [0.0, 2.0, 4.0, 1000.0, -254.0, -1000.0] }
886
+ },
887
+ "y_scale": { "dtype": "float16", "shape": [], "data": { "kind": "values", "values": [2.0] } },
888
+ "y_zero_point": { "dtype": "uint8", "shape": [], "data": { "kind": "values", "values": [128] } }
889
+ },
890
+ "outputs": { "y": { "dtype": "uint8", "shape": [6], "tolerance": 0 } }
891
+ },
892
+ {
893
+ "name": "ort_uint8_5d_scalar_no_zero_point",
894
+ "provenance": {
895
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
896
+ "test": "QuantizeLinearOpTest.Uint8_5D_NoZeroPoint_Opset21_DML"
897
+ },
898
+ "inputs": {
899
+ "x": {
900
+ "dtype": "float32",
901
+ "shape": [6, 1, 1, 1, 1],
902
+ "data": { "kind": "values", "values": [0.0, 51.0, 102.0, 153.0, 204.0, 255.0] }
903
+ },
904
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [1.0] } }
905
+ },
906
+ "outputs": { "y": { "dtype": "uint8", "shape": [6, 1, 1, 1, 1], "tolerance": 0 } }
907
+ },
908
+ {
909
+ "name": "ort_int8_5d_per_axis0_with_zero_point",
910
+ "provenance": {
911
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
912
+ "test": "QuantizeLinearOpTest.Int8_5D_PerAxis_DML_TypeMismatch"
913
+ },
914
+ "attrs": { "axis": 0 },
915
+ "inputs": {
916
+ "x": {
917
+ "dtype": "float32",
918
+ "shape": [6, 1, 1, 1, 1],
919
+ "data": { "kind": "values", "values": [1.0, 2.0, 3.0, 4.0, 5.0, 6.0] }
920
+ },
921
+ "y_scale": {
922
+ "dtype": "float32",
923
+ "shape": [6],
924
+ "data": { "kind": "values", "values": [1.0, 1.0, 1.0, 1.0, 1.0, 1.0] }
925
+ },
926
+ "y_zero_point": { "dtype": "int8", "shape": [6], "data": { "kind": "values", "values": [0, 0, 0, 0, 0, 0] } }
927
+ },
928
+ "outputs": { "y": { "dtype": "int8", "shape": [6, 1, 1, 1, 1], "tolerance": 0 } }
929
+ },
930
+ {
931
+ "name": "ort_blocked_uint8_with_zero_point_rank3_axis2",
932
+ "provenance": {
933
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
934
+ "test": "QuantizeLinearOp21BlockedTest.UnsignedInt_UseZeroPoint_LastAxis",
935
+ "notes": "Valid blocked quantization with per-block zero-points on the last axis."
936
+ },
937
+ "attrs": { "axis": 2, "block_size": 4 },
938
+ "inputs": {
939
+ "x": {
940
+ "dtype": "float32",
941
+ "shape": [2, 4, 8],
942
+ "data": {
943
+ "kind": "values",
944
+ "values": [4.0, 2.0, 4.0, 2.0, -8.0, -12.0, -8.0, -12.0, 4.0, 2.0, 4.0, 2.0, -8.0, -12.0, -8.0, -12.0, 10.5, 14.0, 10.5, 14.0, -3.0, -2.0, -3.0, -2.0, 10.5, 14.0, 10.5, 14.0, -3.0, -2.0, -3.0, -2.0, -10.0, -8.0, -10.0, -8.0, 20.0, 24.0, 20.0, 24.0, -10.0, -8.0, -10.0, -8.0, 20.0, 24.0, 20.0, 24.0, -3.5, -7.0, -3.5, -7.0, -8.0, -9.0, -8.0, -9.0, -3.5, -7.0, -3.5, -7.0, -8.0, -9.0, -8.0, -9.0]
945
+ }
946
+ },
947
+ "y_scale": {
948
+ "dtype": "float32",
949
+ "shape": [2, 4, 2],
950
+ "data": {
951
+ "kind": "values",
952
+ "values": [-2.0, -4.0, -2.0, -4.0, 3.5, 1.0, 3.5, 1.0, 2.0, 4.0, 2.0, 4.0, -3.5, -1.0, -3.5, -1.0]
953
+ }
954
+ },
955
+ "y_zero_point": {
956
+ "dtype": "uint8",
957
+ "shape": [2, 4, 2],
958
+ "data": { "kind": "values", "values": [2, 0, 2, 0, 1, 9, 1, 9, 13, 5, 13, 5, 11, 6, 11, 6] }
959
+ }
960
+ },
961
+ "outputs": { "y": { "dtype": "uint8", "shape": [2, 4, 8], "tolerance": 0 } }
962
+ },
963
+ {
964
+ "name": "ort_blocked_uint8_no_zero_point_rank3_axis1",
965
+ "provenance": {
966
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
967
+ "test": "QuantizeLinearOp21BlockedTest.UnsignedInt_NoZeroPoint_MiddleAxis"
968
+ },
969
+ "attrs": { "axis": 1, "block_size": 2 },
970
+ "inputs": {
971
+ "x": {
972
+ "dtype": "float32",
973
+ "shape": [2, 4, 2],
974
+ "data": {
975
+ "kind": "values",
976
+ "values": [0.0, 1.0, 4.0, 5.0, 16.0, 18.0, 30.0, 33.0, 2.0, 3.0, 8.0, 10.0, 40.0, 42.0, 70.0, 72.0]
977
+ }
978
+ },
979
+ "y_scale": {
980
+ "dtype": "float32",
981
+ "shape": [2, 2, 2],
982
+ "data": { "kind": "values", "values": [1.0, 0.5, 2.0, 3.0, 4.0, 5.0, 10.0, 16.0] }
983
+ }
984
+ },
985
+ "outputs": { "y": { "dtype": "uint8", "shape": [2, 4, 2], "tolerance": 0 } }
986
+ },
987
+ {
988
+ "name": "blocked_uint8_saturates_extreme_finite_inputs",
989
+ "provenance": {
990
+ "source": "onnxruntime/test/providers/cpu/tensor/quantize_linear_test.cc",
991
+ "test": "QuantizeLinearOp21BlockedTest.UnsignedInt_UseZeroPoint_MiddleAxis",
992
+ "notes": "Blocked quantization saturates large finite scaled values after round-to-even, matching the non-blocked QuantizeLinear path (the blocked kernel now clamps value/scale into i32 range before the cast)."
993
+ },
994
+ "attrs": { "axis": 1, "block_size": 2 },
995
+ "inputs": {
996
+ "x": {
997
+ "dtype": "float32",
998
+ "shape": [1, 4],
999
+ "data": { "kind": "values", "values": [3.4028234663852886e+38, -3.4028234663852886e+38, 0.5, -0.5] }
1000
+ },
1001
+ "y_scale": { "dtype": "float32", "shape": [1, 2], "data": { "kind": "values", "values": [0.01, 0.01] } },
1002
+ "y_zero_point": { "dtype": "uint8", "shape": [1, 2], "data": { "kind": "values", "values": [128, 128] } }
1003
+ },
1004
+ "outputs": {
1005
+ "y": {
1006
+ "dtype": "uint8",
1007
+ "shape": [1, 4],
1008
+ "data": { "kind": "values", "values": [255, 0, 178, 78] },
1009
+ "tolerance": 0
1010
+ }
1011
+ }
1012
+ },
1013
+ {
1014
+ "name": "empty_zero_dim",
1015
+ "inputs": {
1016
+ "x": { "dtype": "float32", "shape": [0, 3], "data": { "kind": "values", "values": [] } },
1017
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.5] } },
1018
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
1019
+ },
1020
+ "outputs": { "y": { "dtype": "uint8", "shape": [0, 3], "tolerance": 0 } }
1021
+ },
1022
+ {
1023
+ "name": "empty_zero_dim_f16",
1024
+ "inputs": {
1025
+ "x": { "dtype": "float16", "shape": [0, 3], "data": { "kind": "values", "values": [] } },
1026
+ "y_scale": { "dtype": "float16", "shape": [1], "data": { "kind": "values", "values": [0.5] } },
1027
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [0] } }
1028
+ },
1029
+ "outputs": { "y": { "dtype": "int8", "shape": [0, 3], "tolerance": 0 } }
1030
+ },
1031
+ {
1032
+ "name": "blocked_uint8_no_zero_point_rank2_axis1",
1033
+ "attrs": { "axis": 1, "block_size": 2 },
1034
+ "inputs": {
1035
+ "x": {
1036
+ "dtype": "float32",
1037
+ "shape": [2, 6],
1038
+ "data": { "kind": "values", "values": [0.0, 1.0, 4.0, 6.0, 16.0, 20.0, 2.0, 3.0, 10.0, 12.0, 40.0, 44.0] }
1039
+ },
1040
+ "y_scale": {
1041
+ "dtype": "float32",
1042
+ "shape": [2, 3],
1043
+ "data": { "kind": "values", "values": [1.0, 0.5, 2.0, 4.0, 5.0, 8.0] }
1044
+ }
1045
+ },
1046
+ "outputs": { "y": { "dtype": "uint8", "shape": [2, 6], "tolerance": 0 } }
1047
+ },
1048
+ {
1049
+ "name": "blocked_uint8_partial_last_block_with_zp_axis1",
1050
+ "attrs": { "axis": 1, "block_size": 4 },
1051
+ "inputs": {
1052
+ "x": {
1053
+ "dtype": "float32",
1054
+ "shape": [2, 6],
1055
+ "data": { "kind": "values", "values": [0.0, 4.0, 8.0, 12.0, 20.0, 24.0, 2.0, 6.0, 10.0, 14.0, 30.0, 34.0] }
1056
+ },
1057
+ "y_scale": { "dtype": "float32", "shape": [2, 2], "data": { "kind": "values", "values": [2.0, 4.0, 1.0, 3.0] } },
1058
+ "y_zero_point": { "dtype": "uint8", "shape": [2, 2], "data": { "kind": "values", "values": [1, 2, 0, 5] } }
1059
+ },
1060
+ "outputs": { "y": { "dtype": "uint8", "shape": [2, 6], "tolerance": 0 } }
1061
+ },
1062
+ {
1063
+ "name": "int8_per_axis1_rank4_inner_not_mult4_scalar_fallback",
1064
+ "attrs": { "axis": 1 },
1065
+ "inputs": {
1066
+ "x": {
1067
+ "dtype": "float32",
1068
+ "shape": [1, 3, 3, 1],
1069
+ "data": { "kind": "values", "values": [-4.0, -2.0, 0.0, 2.0, 4.0, 6.0, -8.0, 8.0, 16.0] }
1070
+ },
1071
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [2.0, 4.0, 8.0] } },
1072
+ "y_zero_point": { "dtype": "int8", "shape": [3], "data": { "kind": "values", "values": [0, -1, 2] } }
1073
+ },
1074
+ "outputs": { "y": { "dtype": "int8", "shape": [1, 3, 3, 1], "tolerance": 0 } }
1075
+ },
1076
+ {
1077
+ "name": "per_axis1_scalar_kernel_dispatch_fold_over_16m",
1078
+ "attrs": { "axis": 1 },
1079
+ "inputs": {
1080
+ "x": { "dtype": "float32", "shape": [8388610, 3], "data": { "kind": "linspace", "start": -64.0, "end": 64.0 } },
1081
+ "y_scale": { "dtype": "float32", "shape": [3], "data": { "kind": "values", "values": [0.5, 1.0, 2.0] } },
1082
+ "y_zero_point": { "dtype": "uint8", "shape": [3], "data": { "kind": "values", "values": [10, 20, 30] } }
1083
+ },
1084
+ "outputs": { "y": { "dtype": "uint8", "shape": [8388610, 3], "tolerance": 0 } }
1085
+ },
1086
+ {
1087
+ "name": "empty_zero_dim_per_axis_scale_nonempty",
1088
+ "attrs": { "axis": 1 },
1089
+ "inputs": {
1090
+ "x": { "dtype": "float32", "shape": [0, 4], "data": { "kind": "values", "values": [] } },
1091
+ "y_scale": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [0.5, 1.0, 2.0, 4.0] } },
1092
+ "y_zero_point": { "dtype": "uint8", "shape": [4], "data": { "kind": "values", "values": [128, 100, 64, 0] } }
1093
+ },
1094
+ "outputs": { "y": { "dtype": "uint8", "shape": [0, 4], "tolerance": 0 } }
1095
+ },
1096
+ {
1097
+ "name": "f16_huge_finite_tiny_scale_saturates_not_inf",
1098
+ "inputs": {
1099
+ "x": {
1100
+ "dtype": "float16",
1101
+ "shape": [4],
1102
+ "data": { "kind": "values", "values": [60000.0, -60000.0, 0.0, 32.0] }
1103
+ },
1104
+ "y_scale": { "dtype": "float16", "shape": [1], "data": { "kind": "values", "values": [0.001] } },
1105
+ "y_zero_point": { "dtype": "uint8", "shape": [1], "data": { "kind": "values", "values": [128] } }
1106
+ },
1107
+ "outputs": {
1108
+ "y": {
1109
+ "dtype": "uint8",
1110
+ "shape": [4],
1111
+ "tolerance": 0,
1112
+ "data": { "kind": "values", "values": [255, 0, 128, 255] }
1113
+ }
1114
+ }
1115
+ },
1116
+ {
1117
+ "name": "int8_negative_zero_point_saturation_at_bounds",
1118
+ "inputs": {
1119
+ "x": {
1120
+ "dtype": "float32",
1121
+ "shape": [6],
1122
+ "data": { "kind": "values", "values": [-64.0, -62.5, 63.5, 64.0, 65.0, -100.0] }
1123
+ },
1124
+ "y_scale": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [0.5] } },
1125
+ "y_zero_point": { "dtype": "int8", "shape": [1], "data": { "kind": "values", "values": [-3] } }
1126
+ },
1127
+ "outputs": { "y": { "dtype": "int8", "shape": [6], "tolerance": 0 } }
1128
+ },
1129
+ {
1130
+ "name": "blocked_vec4_int8_negative_axis_partial_block",
1131
+ "provenance": {
1132
+ "notes": "Route lock for blocked vec4 quantization: negative axis -3 normalizes to axis 1, block_size=2 leaves a partial fifth-axis entry, inner=8 keeps each four-lane x/scale access aligned, and signed int8 output exercises both saturation bounds."
1133
+ },
1134
+ "attrs": { "axis": -3, "block_size": 2, "output_dtype": 3 },
1135
+ "inputs": {
1136
+ "x": {
1137
+ "dtype": "float32",
1138
+ "shape": [1, 5, 2, 4],
1139
+ "data": { "kind": "fillFloat32", "sinStep": 0.31, "cosStep": 0.17, "scale": 40.0 }
1140
+ },
1141
+ "y_scale": {
1142
+ "dtype": "float32",
1143
+ "shape": [1, 3, 2, 4],
1144
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.19, "scale": 0.05, "offset": 0.2 }
1145
+ }
1146
+ },
1147
+ "outputs": { "y": { "dtype": "int8", "shape": [1, 5, 2, 4], "tolerance": 0 } }
1148
+ },
1149
+ {
1150
+ "name": "blocked_with_zp_int8_negative_axis_partial_block_vec4_candidate",
1151
+ "provenance": {
1152
+ "notes": "Asymmetric blocked-quantization lock adjacent to the no-zero-point vec4 route: negative axis -3, a partial final axis block, inner=8 alignment, lane-varying signed zero points, and saturating inputs distinguish signed ZP conversion from the symmetric path."
1153
+ },
1154
+ "attrs": { "axis": -3, "block_size": 2 },
1155
+ "inputs": {
1156
+ "x": {
1157
+ "dtype": "float32",
1158
+ "shape": [1, 5, 2, 4],
1159
+ "data": { "kind": "fillFloat32", "sinStep": 0.31, "cosStep": 0.17, "scale": 40.0 }
1160
+ },
1161
+ "y_scale": {
1162
+ "dtype": "float32",
1163
+ "shape": [1, 3, 2, 4],
1164
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.19, "scale": 0.05, "offset": 0.2 }
1165
+ },
1166
+ "y_zero_point": {
1167
+ "dtype": "int8",
1168
+ "shape": [1, 3, 2, 4],
1169
+ "data": { "kind": "cycle", "values": [-128, -17, -1, 0, 1, 23, 64, 127] }
1170
+ }
1171
+ },
1172
+ "outputs": { "y": { "dtype": "int8", "shape": [1, 5, 2, 4], "tolerance": 0 } }
1173
+ },
1174
+ {
1175
+ "name": "blocked_last_axis_vec4_no_zero_point_exact_blocks",
1176
+ "provenance": {
1177
+ "notes": "Route lock for the scalar-scale last-axis vec4 path: each aligned group of four values shares one scale, and signed output covers negative rounding and saturation semantics."
1178
+ },
1179
+ "attrs": { "axis": 1, "block_size": 4, "output_dtype": 3 },
1180
+ "inputs": {
1181
+ "x": {
1182
+ "dtype": "float32",
1183
+ "shape": [2, 8],
1184
+ "data": {
1185
+ "kind": "values",
1186
+ "values": [0.0, 1.0, 2.0, 3.0, 8.0, 12.0, 16.0, 20.0, 10.0, 20.0, 30.0, 40.0, -8.0, -4.0, 0.0, 4.0]
1187
+ }
1188
+ },
1189
+ "y_scale": {
1190
+ "dtype": "float32",
1191
+ "shape": [2, 2],
1192
+ "data": { "kind": "values", "values": [1.0, 4.0, 10.0, 2.0] }
1193
+ }
1194
+ },
1195
+ "outputs": {
1196
+ "y": {
1197
+ "dtype": "int8",
1198
+ "shape": [2, 8],
1199
+ "tolerance": 0,
1200
+ "data": { "kind": "values", "values": [0, 1, 2, 3, 2, 3, 4, 5, 1, 2, 3, 4, -4, -2, 0, 2] }
1201
+ }
1202
+ }
1203
+ },
1204
+ {
1205
+ "name": "blocked_last_axis_vec4_int8_with_zero_point_exact_blocks",
1206
+ "provenance": {
1207
+ "notes": "The blocked last-axis vec4 path uses an int8 zero point, checking the signed zero-point read independently of the uint8 form."
1208
+ },
1209
+ "attrs": { "axis": 1, "block_size": 4, "output_dtype": 3 },
1210
+ "inputs": {
1211
+ "x": {
1212
+ "dtype": "float32",
1213
+ "shape": [2, 8],
1214
+ "data": { "kind": "fillFloat32", "sinStep": 0.23, "cosStep": 0.11, "scale": 2.0 }
1215
+ },
1216
+ "y_scale": {
1217
+ "dtype": "float32",
1218
+ "shape": [2, 2],
1219
+ "data": { "kind": "values", "values": [0.25, 0.5, 0.125, 0.0625] }
1220
+ },
1221
+ "y_zero_point": { "dtype": "int8", "shape": [2, 2], "data": { "kind": "values", "values": [-8, 4, 0, -32] } }
1222
+ },
1223
+ "outputs": { "y": { "dtype": "int8", "shape": [2, 8], "tolerance": 0 } }
1224
+ },
1225
+ {
1226
+ "name": "vec4_cross_axis_rows_width6_with_zero_point",
1227
+ "provenance": {
1228
+ "notes": "A vec4 crosses each six-element row boundary, so its four lanes may use two different per-axis scale and zero-point entries. Locks the lane-specific parameter-index path."
1229
+ },
1230
+ "attrs": { "axis": 0 },
1231
+ "inputs": {
1232
+ "x": {
1233
+ "dtype": "float32",
1234
+ "shape": [4, 6],
1235
+ "data": { "kind": "cycle", "values": [-3.2, -1.1, 0.0, 0.9, 2.4, 7.8] }
1236
+ },
1237
+ "y_scale": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [0.25, 0.5, 1.0, 2.0] } },
1238
+ "y_zero_point": { "dtype": "uint8", "shape": [4], "data": { "kind": "values", "values": [128, 120, 100, 80] } }
1239
+ },
1240
+ "outputs": { "y": { "dtype": "uint8", "shape": [4, 6], "tolerance": 0 } }
1241
+ },
1242
+ {
1243
+ "name": "vec4_cross_axis_rows_width6_no_zero_point",
1244
+ "provenance": {
1245
+ "notes": "Symmetric per-channel quantization over six-element rows with y_zero_point omitted (ONNX makes it optional and defaults it to 0), so a four-lane vector still crosses each row boundary and may need two different per-axis scale entries while no zero-point binding exists. Twin of vec4_cross_axis_rows_width6_with_zero_point; signed int8 output keeps the negative half of each row representable at zero point 0, and the seven-value input cycle is coprime with the six-wide row so no two rows repeat the same lane pattern. No quotient lands on a .5 tie, so the result is exact for either rounding of a tie."
1246
+ },
1247
+ "attrs": { "axis": 0, "output_dtype": 3 },
1248
+ "inputs": {
1249
+ "x": {
1250
+ "dtype": "float32",
1251
+ "shape": [4, 6],
1252
+ "data": { "kind": "cycle", "values": [-3.2, -1.1, 0.0, 0.9, 2.4, 7.8, -5.6] }
1253
+ },
1254
+ "y_scale": { "dtype": "float32", "shape": [4], "data": { "kind": "values", "values": [0.25, 0.5, 1.0, 2.0] } }
1255
+ },
1256
+ "outputs": { "y": { "dtype": "int8", "shape": [4, 6], "tolerance": 0 } }
1257
+ },
1258
+ {
1259
+ "name": "f16_default_precision_rounds_division_in_f16",
1260
+ "provenance": {
1261
+ "notes": "Exact opset-25 precision witness: with f16 y_scale and omitted precision, x/y_scale is evaluated in f16. The f16 quotient rounds above 13.5 and then rounds to integer 14; an f32 division incorrectly produces 13."
1262
+ },
1263
+ "inputs": {
1264
+ "x": { "dtype": "float16", "shape": [1], "data": { "kind": "values", "values": [35.25] } },
1265
+ "y_scale": { "dtype": "float16", "shape": [], "data": { "kind": "values", "values": [2.611328125] } }
1266
+ },
1267
+ "outputs": {
1268
+ "y": { "dtype": "uint8", "shape": [1], "tolerance": 0, "data": { "kind": "values", "values": [14] } }
1269
+ }
1270
+ },
1271
+ {
1272
+ "name": "f16_inputs_explicit_float32_precision",
1273
+ "provenance": {
1274
+ "notes": "Twin of the default-precision witness with precision=FLOAT (TensorProto code 1), proving that an explicit f32 division remains distinct and rounds to 13."
1275
+ },
1276
+ "attrs": { "precision": 1 },
1277
+ "inputs": {
1278
+ "x": { "dtype": "float16", "shape": [1], "data": { "kind": "values", "values": [35.25] } },
1279
+ "y_scale": { "dtype": "float16", "shape": [], "data": { "kind": "values", "values": [2.611328125] } }
1280
+ },
1281
+ "outputs": {
1282
+ "y": { "dtype": "uint8", "shape": [1], "tolerance": 0, "data": { "kind": "values", "values": [13] } }
1283
+ }
1284
+ },
1285
+ {
1286
+ "name": "f32_x_f16_scale_default_precision",
1287
+ "provenance": {
1288
+ "notes": "Independent T1/T2 type-variable witness: float32 x and float16 y_scale are a standard mixed route. With precision omitted, the scale type selects f16 division and the quotient rounds to integer 14."
1289
+ },
1290
+ "inputs": {
1291
+ "x": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [35.25] } },
1292
+ "y_scale": { "dtype": "float16", "shape": [], "data": { "kind": "values", "values": [2.611328125] } }
1293
+ },
1294
+ "outputs": {
1295
+ "y": { "dtype": "uint8", "shape": [1], "tolerance": 0, "data": { "kind": "values", "values": [14] } }
1296
+ }
1297
+ },
1298
+ {
1299
+ "name": "f16_x_f32_scale_default_precision",
1300
+ "provenance": {
1301
+ "notes": "Independent T1/T2 type-variable witness: float16 x and float32 y_scale are a standard mixed route. With precision omitted, the scale type selects f32 division and the quotient rounds to integer 13."
1302
+ },
1303
+ "inputs": {
1304
+ "x": { "dtype": "float16", "shape": [1], "data": { "kind": "values", "values": [35.25] } },
1305
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.611328125] } }
1306
+ },
1307
+ "outputs": {
1308
+ "y": { "dtype": "uint8", "shape": [1], "tolerance": 0, "data": { "kind": "values", "values": [13] } }
1309
+ }
1310
+ },
1311
+ {
1312
+ "name": "f32_inputs_explicit_float16_precision",
1313
+ "provenance": {
1314
+ "notes": "Exact opset-25 precision=FLOAT16 witness with float32 x and y_scale. The division is explicitly evaluated in f16, where the quotient rounds above 13.5 and then rounds to integer 14; ignoring precision and dividing in f32 incorrectly produces 13."
1315
+ },
1316
+ "requires": { "features": ["shader-f16"] },
1317
+ "attrs": { "precision": 10 },
1318
+ "inputs": {
1319
+ "x": { "dtype": "float32", "shape": [1], "data": { "kind": "values", "values": [35.25] } },
1320
+ "y_scale": { "dtype": "float32", "shape": [], "data": { "kind": "values", "values": [2.611328125] } }
1321
+ },
1322
+ "outputs": {
1323
+ "y": { "dtype": "uint8", "shape": [1], "tolerance": 0, "data": { "kind": "values", "values": [14] } }
1324
+ }
1325
+ }
1326
+ ]
1327
+ }