Hugging Face's logo

Kernels:
webgpu-kernels
/
com.microsoft.GatedRelativePositionBias

Kernel card Files
xet
 Community
Xenova HF Staff commited on
Commit
81aba80
·
verified ·
1 Parent(s): b81405f

sync 6fdf6301e2bb

Browse files
Files changed (6) hide show
  1. README.md +94 -0
  2. build/webgpu/bench.json +1621 -0
  3. build/webgpu/gated-relative-position-bias.wgsl.jinja +195 -0
  4. build/webgpu/manifest.json +125 -0
  5. build/webgpu/metadata.json +25 -0
  6. build/webgpu/test.json +2478 -0
README.md CHANGED
@@ -1,3 +1,97 @@
1
  ---
 
2
  license: apache-2.0
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: kernels
3
  license: apache-2.0
4
+ tags:
5
+ - kernel
6
+ - webgpu
7
+ - wgsl
8
  ---
9
+ # com.microsoft.GatedRelativePositionBias
10
+
11
+ `com.microsoft` · ONNX Runtime contrib operator · contrib since_version 1
12
+
13
+ ## Description
14
+
15
+ Scales a precomputed relative-position bias by a per-query gate. The biased query row for each head is projected through `weight` and `bias` to `D` values; the first `D/2` sum to one gate and the last `D/2` to another, both through a logistic, and `output = (gate_u * (gate_r * eco_a - 1) + 2) * rel_pos`. `D` must be even and at most 32. All gate arithmetic is float32 whatever the tensor type, and only the store narrows.
16
+
17
+ See the [ONNX Runtime `GatedRelativePositionBias` contrib-operator spec](https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#com.microsoft.GatedRelativePositionBias) for the reference semantics.
18
+
19
+ ## Inputs
20
+
21
+ | Name | Upstream name | Logical dtype | Rank | Shape | Description | Presence |
22
+ | --- | --- | --- | --- | --- | --- | --- |
23
+ | `queryLayerT` | `query_layer` | `T` | `3` | — | Query activations with shape `(batch_size, seq_len, num_heads * head_size)`. | required |
24
+ | `queryBiasT` | `query_bias` | `T` | `1` | — | Bias added to `query_layer` before the projection, with shape `(num_heads * head_size)`. | required |
25
+ | `relPosT` | `rel_pos` | `T` | `4` | — | Relative position bias with shape `(1, num_heads, seq_len, seq_len)`. Its single batch is broadcast over the output's batches. | required |
26
+ | `weightT` | `weight` | `T` | `2` | — | Projection weight for the gate, with shape `(head_size, D)` where `D` is even. | required |
27
+ | `biasT` | `bias` | `T` | `1` | — | Projection bias for the gate, with shape `(D)`. | required |
28
+ | `ecoAT` | `eco_a` | `T` | `4` | — | Per-head coefficient with shape `(1, num_heads, 1, 1)`. | required |
29
+
30
+ ## Outputs
31
+
32
+ | Name | Upstream name | Logical dtype | Rank | Shape | Description | Presence |
33
+ | --- | --- | --- | --- | --- | --- | --- |
34
+ | `outputT` | `output` | `T` | `4` | `[queryLayerT[0], num_heads, queryLayerT[1], queryLayerT[1]]` | Gated relative position bias with shape `(batch_size, num_heads, seq_len, seq_len)`. | required |
35
+
36
+ ## Attributes
37
+
38
+ Attributes and default values (overridable per request):
39
+
40
+ | Attribute | Default | Description |
41
+ | --- | --- | --- |
42
+ | `num_heads` | — | Number of attention heads. The head size is the query hidden size divided by this count. |
43
+
44
+ ## Type constraints
45
+
46
+ | Variable | Allowed dtypes |
47
+ | --- | --- |
48
+ | `T` | `float32`, `float16` |
49
+
50
+ ## Implementation variants
51
+
52
+ One implementation is selected per call from the device capabilities, the request shapes and the dtypes; these notes say what each one covers.
53
+
54
+ - `row` — One workgroup per output row, streaming the bias row as `vec4` words when the sequence length is a multiple of four and as scalars otherwise; the route every device can select. The gate is computed once into workgroup memory, so its cost is amortized over the whole row rather than repeated per element.
55
+ - `row_head_tree` — Projects head elements in parallel with a tree reduction of both float32 gate sums. Used when the head dimension fills at least four head lanes per projection lane or, on adapters with a variable or unreported subgroup range, for long vec4-aligned rows with a head-to-projection ratio of at least eight. Subgroup leaders combine sums in workgroup memory.
56
+ - `row_head_subgroups` — Projects head elements in parallel with a subgroups reduction of both float32 gate sums. Used when the head dimension fills at least four head lanes per projection lane or, on adapters with a variable or unreported subgroup range, for long vec4-aligned rows with a head-to-projection ratio of at least eight. Subgroup leaders combine sums in workgroup memory.
57
+
58
+ ## Device requirements
59
+
60
+ Some implementation variants require `subgroups`. These are route-specific capabilities, not package-wide requirements; availability also depends on the request shape and dtype.
61
+
62
+ ## Files
63
+
64
+ - [`metadata.json`](build/webgpu/metadata.json) — kernel metadata (id, digests, per-variant templates, provenance)
65
+ - [`manifest.json`](build/webgpu/manifest.json) — the op contract (source of truth)
66
+ - [`test.json`](build/webgpu/test.json) — correctness cases
67
+ - [`bench.json`](build/webgpu/bench.json) — benchmark cases
68
+ - [`gated-relative-position-bias.wgsl.jinja`](build/webgpu/gated-relative-position-bias.wgsl.jinja)
69
+
70
+ ## Use with `@huggingface/kernels`
71
+
72
+ ```sh
73
+ npm install --save-exact @huggingface/kernels@0.0.1-preview.3
74
+ ```
75
+
76
+ Required output shapes and logical data types are inferred from the supplied inputs and attributes; result tensors are allocated automatically.
77
+
78
+ The `version: 1` option selects the published kernel contract; it is independent of any operator opset, contrib `since_version`, or model version.
79
+ It follows the `v1` branch as fixes land. To pin exact artifact bytes, pass a 40-character commit `revision` instead of `version`.
80
+
81
+ Replace each `*Data` placeholder with a typed array containing the corresponding input data.
82
+
83
+ ```js
84
+ import { getKernel } from "@huggingface/kernels";
85
+
86
+ const kernel = await getKernel("webgpu-kernels/com.microsoft.GatedRelativePositionBias", { version: 1 });
87
+ const { outputT } = await kernel({
88
+ queryLayerT: { data: queryLayerTData, shape: [2, 1, 3] },
89
+ queryBiasT: { data: queryBiasTData, shape: [3] },
90
+ relPosT: { data: relPosTData, shape: [1, 3, 1, 1] },
91
+ weightT: { data: weightTData, shape: [1, 2] },
92
+ biasT: { data: biasTData, shape: [2] },
93
+ ecoAT: { data: ecoATData, shape: [1, 3, 1, 1] },
94
+ }, {
95
+ attrs: { num_heads: 3 },
96
+ });
97
+ ```
build/webgpu/bench.json ADDED
@@ -0,0 +1,1621 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "tunableSpace": { "WORKGROUP_SIZE": [64, 128, 256, 512] },
3
+ "cases": [
4
+ {
5
+ "name": "grpb-b1-n12-s512-h64-d8-f32",
6
+ "preset": "smoke",
7
+ "provenance": {
8
+ "notes": "Measures GatedRelativePositionBias over a 512-position sequence with 12 heads, producing 3,145,728 output elements in float32."
9
+ },
10
+ "vars": { "dtype": "float32", "count": 3145728, "relCount": 3145728 },
11
+ "inputs": {
12
+ "queryLayerT": { "shape": [1, 512, 768], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
13
+ "queryBiasT": { "shape": [768], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
14
+ "relPosT": { "shape": [1, 12, 512, 512], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
15
+ "weightT": { "shape": [64, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
16
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
17
+ "ecoAT": { "shape": [1, 12, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
18
+ },
19
+ "outputs": { "outputT": { "shape": [1, 12, 512, 512], "dtype": "float32" } },
20
+ "bench": {
21
+ "metrics": [
22
+ {
23
+ "type": "bandwidth",
24
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
25
+ },
26
+ {
27
+ "type": "gflops",
28
+ "name": "projection_and_row_scale",
29
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
30
+ }
31
+ ]
32
+ },
33
+ "attrs": { "num_heads": 12 }
34
+ },
35
+ {
36
+ "name": "grpb-b1-n12-s512-h64-d8-f16",
37
+ "preset": "smoke",
38
+ "provenance": { "notes": "The float16 counterpart of the same utterance." },
39
+ "vars": { "dtype": "float16", "count": 3145728, "relCount": 3145728 },
40
+ "inputs": {
41
+ "queryLayerT": { "shape": [1, 512, 768], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
42
+ "queryBiasT": { "shape": [768], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
43
+ "relPosT": { "shape": [1, 12, 512, 512], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
44
+ "weightT": { "shape": [64, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
45
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
46
+ "ecoAT": { "shape": [1, 12, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
47
+ },
48
+ "outputs": { "outputT": { "shape": [1, 12, 512, 512], "dtype": "float16" } },
49
+ "bench": {
50
+ "metrics": [
51
+ {
52
+ "type": "bandwidth",
53
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
54
+ },
55
+ {
56
+ "type": "gflops",
57
+ "name": "projection_and_row_scale",
58
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
59
+ }
60
+ ]
61
+ },
62
+ "attrs": { "num_heads": 12 }
63
+ },
64
+ {
65
+ "name": "grpb-b4-n8-s256-h64-d8-f16",
66
+ "preset": "smoke",
67
+ "provenance": {
68
+ "notes": "Batch 4 at a shorter length, where rel_pos is read once per batch and the batch stride is live."
69
+ },
70
+ "vars": { "dtype": "float16", "count": 2097152, "relCount": 524288 },
71
+ "inputs": {
72
+ "queryLayerT": { "shape": [4, 256, 512], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
73
+ "queryBiasT": { "shape": [512], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
74
+ "relPosT": { "shape": [1, 8, 256, 256], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
75
+ "weightT": { "shape": [64, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
76
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
77
+ "ecoAT": { "shape": [1, 8, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
78
+ },
79
+ "outputs": { "outputT": { "shape": [4, 8, 256, 256], "dtype": "float16" } },
80
+ "bench": {
81
+ "metrics": [
82
+ {
83
+ "type": "bandwidth",
84
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
85
+ },
86
+ {
87
+ "type": "gflops",
88
+ "name": "projection_and_row_scale",
89
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
90
+ }
91
+ ]
92
+ },
93
+ "attrs": { "num_heads": 8 }
94
+ },
95
+ {
96
+ "name": "grpb-b1-n12-s1002-h64-d8-f32",
97
+ "preset": "smoke",
98
+ "provenance": {
99
+ "notes": "A length that is not a multiple of four, so the scalar route runs at a realistic size."
100
+ },
101
+ "vars": { "dtype": "float32", "count": 12048048, "relCount": 12048048 },
102
+ "inputs": {
103
+ "queryLayerT": { "shape": [1, 1002, 768], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
104
+ "queryBiasT": { "shape": [768], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
105
+ "relPosT": { "shape": [1, 12, 1002, 1002], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
106
+ "weightT": { "shape": [64, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
107
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
108
+ "ecoAT": { "shape": [1, 12, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
109
+ },
110
+ "outputs": { "outputT": { "shape": [1, 12, 1002, 1002], "dtype": "float32" } },
111
+ "bench": {
112
+ "metrics": [
113
+ {
114
+ "type": "bandwidth",
115
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
116
+ },
117
+ {
118
+ "type": "gflops",
119
+ "name": "projection_and_row_scale",
120
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
121
+ }
122
+ ]
123
+ },
124
+ "attrs": { "num_heads": 12 }
125
+ },
126
+ {
127
+ "name": "grpb-b1-n12-s1024-h64-d8-f16",
128
+ "preset": "model",
129
+ "provenance": { "notes": "A 1024-position utterance: 12.6M output elements, a working set past cache." },
130
+ "vars": { "dtype": "float16", "count": 12582912, "relCount": 12582912 },
131
+ "inputs": {
132
+ "queryLayerT": { "shape": [1, 1024, 768], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
133
+ "queryBiasT": { "shape": [768], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
134
+ "relPosT": { "shape": [1, 12, 1024, 1024], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
135
+ "weightT": { "shape": [64, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
136
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
137
+ "ecoAT": { "shape": [1, 12, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
138
+ },
139
+ "outputs": { "outputT": { "shape": [1, 12, 1024, 1024], "dtype": "float16" } },
140
+ "bench": {
141
+ "metrics": [
142
+ {
143
+ "type": "bandwidth",
144
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
145
+ },
146
+ {
147
+ "type": "gflops",
148
+ "name": "projection_and_row_scale",
149
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
150
+ }
151
+ ]
152
+ },
153
+ "attrs": { "num_heads": 12 }
154
+ },
155
+ {
156
+ "name": "grpb-b1-n12-s1024-h64-d8-f32",
157
+ "preset": "model",
158
+ "provenance": { "notes": "The float32 counterpart at twice the bytes." },
159
+ "vars": { "dtype": "float32", "count": 12582912, "relCount": 12582912 },
160
+ "inputs": {
161
+ "queryLayerT": { "shape": [1, 1024, 768], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
162
+ "queryBiasT": { "shape": [768], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
163
+ "relPosT": { "shape": [1, 12, 1024, 1024], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
164
+ "weightT": { "shape": [64, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
165
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
166
+ "ecoAT": { "shape": [1, 12, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
167
+ },
168
+ "outputs": { "outputT": { "shape": [1, 12, 1024, 1024], "dtype": "float32" } },
169
+ "bench": {
170
+ "metrics": [
171
+ {
172
+ "type": "bandwidth",
173
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
174
+ },
175
+ {
176
+ "type": "gflops",
177
+ "name": "projection_and_row_scale",
178
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
179
+ }
180
+ ]
181
+ },
182
+ "attrs": { "num_heads": 12 }
183
+ },
184
+ {
185
+ "name": "grpb-boundary-b2-n3-s128-h31-d8-f32",
186
+ "preset": "stress",
187
+ "provenance": {
188
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
189
+ },
190
+ "attrs": { "num_heads": 3 },
191
+ "vars": { "dtype": "float32", "count": 98304, "relCount": 49152 },
192
+ "inputs": {
193
+ "queryLayerT": { "shape": [2, 128, 93], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
194
+ "queryBiasT": { "shape": [93], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
195
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
196
+ "weightT": { "shape": [31, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
197
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
198
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
199
+ },
200
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float32" } },
201
+ "bench": {
202
+ "metrics": [
203
+ {
204
+ "type": "bandwidth",
205
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
206
+ },
207
+ {
208
+ "type": "gflops",
209
+ "name": "projection_and_row_scale",
210
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
211
+ }
212
+ ]
213
+ }
214
+ },
215
+ {
216
+ "name": "grpb-boundary-b2-n3-s128-h32-d8-f32",
217
+ "preset": "stress",
218
+ "provenance": {
219
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
220
+ },
221
+ "attrs": { "num_heads": 3 },
222
+ "vars": { "dtype": "float32", "count": 98304, "relCount": 49152 },
223
+ "inputs": {
224
+ "queryLayerT": { "shape": [2, 128, 96], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
225
+ "queryBiasT": { "shape": [96], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
226
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
227
+ "weightT": { "shape": [32, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
228
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
229
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
230
+ },
231
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float32" } },
232
+ "bench": {
233
+ "metrics": [
234
+ {
235
+ "type": "bandwidth",
236
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
237
+ },
238
+ {
239
+ "type": "gflops",
240
+ "name": "projection_and_row_scale",
241
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
242
+ }
243
+ ]
244
+ }
245
+ },
246
+ {
247
+ "name": "grpb-boundary-b2-n3-s127-h23-d6-f32",
248
+ "preset": "stress",
249
+ "provenance": {
250
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
251
+ },
252
+ "attrs": { "num_heads": 3 },
253
+ "vars": { "dtype": "float32", "count": 96774, "relCount": 48387 },
254
+ "inputs": {
255
+ "queryLayerT": { "shape": [2, 127, 69], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
256
+ "queryBiasT": { "shape": [69], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
257
+ "relPosT": { "shape": [1, 3, 127, 127], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
258
+ "weightT": { "shape": [23, 6], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
259
+ "biasT": { "shape": [6], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
260
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
261
+ },
262
+ "outputs": { "outputT": { "shape": [2, 3, 127, 127], "dtype": "float32" } },
263
+ "bench": {
264
+ "metrics": [
265
+ {
266
+ "type": "bandwidth",
267
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
268
+ },
269
+ {
270
+ "type": "gflops",
271
+ "name": "projection_and_row_scale",
272
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
273
+ }
274
+ ]
275
+ }
276
+ },
277
+ {
278
+ "name": "grpb-boundary-b2-n3-s127-h24-d6-f32",
279
+ "preset": "stress",
280
+ "provenance": {
281
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
282
+ },
283
+ "attrs": { "num_heads": 3 },
284
+ "vars": { "dtype": "float32", "count": 96774, "relCount": 48387 },
285
+ "inputs": {
286
+ "queryLayerT": { "shape": [2, 127, 72], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
287
+ "queryBiasT": { "shape": [72], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
288
+ "relPosT": { "shape": [1, 3, 127, 127], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
289
+ "weightT": { "shape": [24, 6], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
290
+ "biasT": { "shape": [6], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
291
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
292
+ },
293
+ "outputs": { "outputT": { "shape": [2, 3, 127, 127], "dtype": "float32" } },
294
+ "bench": {
295
+ "metrics": [
296
+ {
297
+ "type": "bandwidth",
298
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
299
+ },
300
+ {
301
+ "type": "gflops",
302
+ "name": "projection_and_row_scale",
303
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
304
+ }
305
+ ]
306
+ }
307
+ },
308
+ {
309
+ "name": "grpb-boundary-b2-n3-s128-h64-d10-f32",
310
+ "preset": "stress",
311
+ "provenance": {
312
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
313
+ },
314
+ "attrs": { "num_heads": 3 },
315
+ "vars": { "dtype": "float32", "count": 98304, "relCount": 49152 },
316
+ "inputs": {
317
+ "queryLayerT": { "shape": [2, 128, 192], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
318
+ "queryBiasT": { "shape": [192], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
319
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
320
+ "weightT": { "shape": [64, 10], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
321
+ "biasT": { "shape": [10], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
322
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
323
+ },
324
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float32" } },
325
+ "bench": {
326
+ "metrics": [
327
+ {
328
+ "type": "bandwidth",
329
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
330
+ },
331
+ {
332
+ "type": "gflops",
333
+ "name": "projection_and_row_scale",
334
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
335
+ }
336
+ ]
337
+ }
338
+ },
339
+ {
340
+ "name": "grpb-general-b2-n3-s1-h1-d2-f32",
341
+ "preset": "stress",
342
+ "provenance": {
343
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
344
+ },
345
+ "attrs": { "num_heads": 3 },
346
+ "vars": { "dtype": "float32", "count": 6, "relCount": 3 },
347
+ "inputs": {
348
+ "queryLayerT": { "shape": [2, 1, 3], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
349
+ "queryBiasT": { "shape": [3], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
350
+ "relPosT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
351
+ "weightT": { "shape": [1, 2], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
352
+ "biasT": { "shape": [2], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
353
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
354
+ },
355
+ "outputs": { "outputT": { "shape": [2, 3, 1, 1], "dtype": "float32" } },
356
+ "bench": {
357
+ "metrics": [
358
+ {
359
+ "type": "bandwidth",
360
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
361
+ },
362
+ {
363
+ "type": "gflops",
364
+ "name": "projection_and_row_scale",
365
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
366
+ }
367
+ ]
368
+ }
369
+ },
370
+ {
371
+ "name": "grpb-general-b2-n3-s5-h3-d32-f32",
372
+ "preset": "stress",
373
+ "provenance": {
374
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
375
+ },
376
+ "attrs": { "num_heads": 3 },
377
+ "vars": { "dtype": "float32", "count": 150, "relCount": 75 },
378
+ "inputs": {
379
+ "queryLayerT": { "shape": [2, 5, 9], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
380
+ "queryBiasT": { "shape": [9], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
381
+ "relPosT": { "shape": [1, 3, 5, 5], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
382
+ "weightT": { "shape": [3, 32], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
383
+ "biasT": { "shape": [32], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
384
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
385
+ },
386
+ "outputs": { "outputT": { "shape": [2, 3, 5, 5], "dtype": "float32" } },
387
+ "bench": {
388
+ "metrics": [
389
+ {
390
+ "type": "bandwidth",
391
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
392
+ },
393
+ {
394
+ "type": "gflops",
395
+ "name": "projection_and_row_scale",
396
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
397
+ }
398
+ ]
399
+ }
400
+ },
401
+ {
402
+ "name": "grpb-general-b2-n3-s17-h8-d16-f32",
403
+ "preset": "stress",
404
+ "provenance": {
405
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
406
+ },
407
+ "attrs": { "num_heads": 3 },
408
+ "vars": { "dtype": "float32", "count": 1734, "relCount": 867 },
409
+ "inputs": {
410
+ "queryLayerT": { "shape": [2, 17, 24], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
411
+ "queryBiasT": { "shape": [24], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
412
+ "relPosT": { "shape": [1, 3, 17, 17], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
413
+ "weightT": { "shape": [8, 16], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
414
+ "biasT": { "shape": [16], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
415
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
416
+ },
417
+ "outputs": { "outputT": { "shape": [2, 3, 17, 17], "dtype": "float32" } },
418
+ "bench": {
419
+ "metrics": [
420
+ {
421
+ "type": "bandwidth",
422
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
423
+ },
424
+ {
425
+ "type": "gflops",
426
+ "name": "projection_and_row_scale",
427
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
428
+ }
429
+ ]
430
+ }
431
+ },
432
+ {
433
+ "name": "grpb-general-b2-n3-s31-h31-d6-f32",
434
+ "preset": "stress",
435
+ "provenance": {
436
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
437
+ },
438
+ "attrs": { "num_heads": 3 },
439
+ "vars": { "dtype": "float32", "count": 5766, "relCount": 2883 },
440
+ "inputs": {
441
+ "queryLayerT": { "shape": [2, 31, 93], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
442
+ "queryBiasT": { "shape": [93], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
443
+ "relPosT": { "shape": [1, 3, 31, 31], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
444
+ "weightT": { "shape": [31, 6], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
445
+ "biasT": { "shape": [6], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
446
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
447
+ },
448
+ "outputs": { "outputT": { "shape": [2, 3, 31, 31], "dtype": "float32" } },
449
+ "bench": {
450
+ "metrics": [
451
+ {
452
+ "type": "bandwidth",
453
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
454
+ },
455
+ {
456
+ "type": "gflops",
457
+ "name": "projection_and_row_scale",
458
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
459
+ }
460
+ ]
461
+ }
462
+ },
463
+ {
464
+ "name": "grpb-general-b2-n3-s64-h32-d32-f32",
465
+ "preset": "stress",
466
+ "provenance": {
467
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
468
+ },
469
+ "attrs": { "num_heads": 3 },
470
+ "vars": { "dtype": "float32", "count": 24576, "relCount": 12288 },
471
+ "inputs": {
472
+ "queryLayerT": { "shape": [2, 64, 96], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
473
+ "queryBiasT": { "shape": [96], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
474
+ "relPosT": { "shape": [1, 3, 64, 64], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
475
+ "weightT": { "shape": [32, 32], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
476
+ "biasT": { "shape": [32], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
477
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
478
+ },
479
+ "outputs": { "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32" } },
480
+ "bench": {
481
+ "metrics": [
482
+ {
483
+ "type": "bandwidth",
484
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
485
+ },
486
+ {
487
+ "type": "gflops",
488
+ "name": "projection_and_row_scale",
489
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
490
+ }
491
+ ]
492
+ }
493
+ },
494
+ {
495
+ "name": "grpb-general-b2-n3-s127-h64-d2-f32",
496
+ "preset": "stress",
497
+ "provenance": {
498
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
499
+ },
500
+ "attrs": { "num_heads": 3 },
501
+ "vars": { "dtype": "float32", "count": 96774, "relCount": 48387 },
502
+ "inputs": {
503
+ "queryLayerT": { "shape": [2, 127, 192], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
504
+ "queryBiasT": { "shape": [192], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
505
+ "relPosT": { "shape": [1, 3, 127, 127], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
506
+ "weightT": { "shape": [64, 2], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
507
+ "biasT": { "shape": [2], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
508
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
509
+ },
510
+ "outputs": { "outputT": { "shape": [2, 3, 127, 127], "dtype": "float32" } },
511
+ "bench": {
512
+ "metrics": [
513
+ {
514
+ "type": "bandwidth",
515
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
516
+ },
517
+ {
518
+ "type": "gflops",
519
+ "name": "projection_and_row_scale",
520
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
521
+ }
522
+ ]
523
+ }
524
+ },
525
+ {
526
+ "name": "grpb-general-b2-n3-s128-h63-d8-f32",
527
+ "preset": "stress",
528
+ "provenance": {
529
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
530
+ },
531
+ "attrs": { "num_heads": 3 },
532
+ "vars": { "dtype": "float32", "count": 98304, "relCount": 49152 },
533
+ "inputs": {
534
+ "queryLayerT": { "shape": [2, 128, 189], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
535
+ "queryBiasT": { "shape": [189], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
536
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
537
+ "weightT": { "shape": [63, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
538
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
539
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
540
+ },
541
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float32" } },
542
+ "bench": {
543
+ "metrics": [
544
+ {
545
+ "type": "bandwidth",
546
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
547
+ },
548
+ {
549
+ "type": "gflops",
550
+ "name": "projection_and_row_scale",
551
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
552
+ }
553
+ ]
554
+ }
555
+ },
556
+ {
557
+ "name": "grpb-general-b2-n3-s257-h127-d6-f32",
558
+ "preset": "stress",
559
+ "provenance": {
560
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
561
+ },
562
+ "attrs": { "num_heads": 3 },
563
+ "vars": { "dtype": "float32", "count": 396294, "relCount": 198147 },
564
+ "inputs": {
565
+ "queryLayerT": { "shape": [2, 257, 381], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
566
+ "queryBiasT": { "shape": [381], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
567
+ "relPosT": { "shape": [1, 3, 257, 257], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
568
+ "weightT": { "shape": [127, 6], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
569
+ "biasT": { "shape": [6], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
570
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
571
+ },
572
+ "outputs": { "outputT": { "shape": [2, 3, 257, 257], "dtype": "float32" } },
573
+ "bench": {
574
+ "metrics": [
575
+ {
576
+ "type": "bandwidth",
577
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
578
+ },
579
+ {
580
+ "type": "gflops",
581
+ "name": "projection_and_row_scale",
582
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
583
+ }
584
+ ]
585
+ }
586
+ },
587
+ {
588
+ "name": "grpb-general-b2-n3-s256-h128-d32-f32",
589
+ "preset": "stress",
590
+ "provenance": {
591
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
592
+ },
593
+ "attrs": { "num_heads": 3 },
594
+ "vars": { "dtype": "float32", "count": 393216, "relCount": 196608 },
595
+ "inputs": {
596
+ "queryLayerT": { "shape": [2, 256, 384], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
597
+ "queryBiasT": { "shape": [384], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
598
+ "relPosT": { "shape": [1, 3, 256, 256], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
599
+ "weightT": { "shape": [128, 32], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
600
+ "biasT": { "shape": [32], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
601
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
602
+ },
603
+ "outputs": { "outputT": { "shape": [2, 3, 256, 256], "dtype": "float32" } },
604
+ "bench": {
605
+ "metrics": [
606
+ {
607
+ "type": "bandwidth",
608
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
609
+ },
610
+ {
611
+ "type": "gflops",
612
+ "name": "projection_and_row_scale",
613
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
614
+ }
615
+ ]
616
+ }
617
+ },
618
+ {
619
+ "name": "grpb-general-b1-n4-s512-h256-d16-f32",
620
+ "preset": "stress",
621
+ "provenance": {
622
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
623
+ },
624
+ "attrs": { "num_heads": 4 },
625
+ "vars": { "dtype": "float32", "count": 1048576, "relCount": 1048576 },
626
+ "inputs": {
627
+ "queryLayerT": { "shape": [1, 512, 1024], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
628
+ "queryBiasT": { "shape": [1024], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
629
+ "relPosT": { "shape": [1, 4, 512, 512], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
630
+ "weightT": { "shape": [256, 16], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
631
+ "biasT": { "shape": [16], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
632
+ "ecoAT": { "shape": [1, 4, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
633
+ },
634
+ "outputs": { "outputT": { "shape": [1, 4, 512, 512], "dtype": "float32" } },
635
+ "bench": {
636
+ "metrics": [
637
+ {
638
+ "type": "bandwidth",
639
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
640
+ },
641
+ {
642
+ "type": "gflops",
643
+ "name": "projection_and_row_scale",
644
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
645
+ }
646
+ ]
647
+ }
648
+ },
649
+ {
650
+ "name": "grpb-general-b1-n2-s63-h512-d8-f32",
651
+ "preset": "stress",
652
+ "provenance": {
653
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
654
+ },
655
+ "attrs": { "num_heads": 2 },
656
+ "vars": { "dtype": "float32", "count": 7938, "relCount": 7938 },
657
+ "inputs": {
658
+ "queryLayerT": { "shape": [1, 63, 1024], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
659
+ "queryBiasT": { "shape": [1024], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
660
+ "relPosT": { "shape": [1, 2, 63, 63], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
661
+ "weightT": { "shape": [512, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
662
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
663
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
664
+ },
665
+ "outputs": { "outputT": { "shape": [1, 2, 63, 63], "dtype": "float32" } },
666
+ "bench": {
667
+ "metrics": [
668
+ {
669
+ "type": "bandwidth",
670
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
671
+ },
672
+ {
673
+ "type": "gflops",
674
+ "name": "projection_and_row_scale",
675
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
676
+ }
677
+ ]
678
+ }
679
+ },
680
+ {
681
+ "name": "grpb-general-b1-n2-s32-h1024-d32-f32",
682
+ "preset": "stress",
683
+ "provenance": {
684
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
685
+ },
686
+ "attrs": { "num_heads": 2 },
687
+ "vars": { "dtype": "float32", "count": 2048, "relCount": 2048 },
688
+ "inputs": {
689
+ "queryLayerT": { "shape": [1, 32, 2048], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
690
+ "queryBiasT": { "shape": [2048], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
691
+ "relPosT": { "shape": [1, 2, 32, 32], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
692
+ "weightT": { "shape": [1024, 32], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
693
+ "biasT": { "shape": [32], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
694
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
695
+ },
696
+ "outputs": { "outputT": { "shape": [1, 2, 32, 32], "dtype": "float32" } },
697
+ "bench": {
698
+ "metrics": [
699
+ {
700
+ "type": "bandwidth",
701
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
702
+ },
703
+ {
704
+ "type": "gflops",
705
+ "name": "projection_and_row_scale",
706
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
707
+ }
708
+ ]
709
+ }
710
+ },
711
+ {
712
+ "name": "grpb-general-b1-n2-s2053-h64-d8-f32",
713
+ "preset": "stress",
714
+ "provenance": {
715
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
716
+ },
717
+ "attrs": { "num_heads": 2 },
718
+ "vars": { "dtype": "float32", "count": 8429618, "relCount": 8429618 },
719
+ "inputs": {
720
+ "queryLayerT": { "shape": [1, 2053, 128], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
721
+ "queryBiasT": { "shape": [128], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
722
+ "relPosT": { "shape": [1, 2, 2053, 2053], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
723
+ "weightT": { "shape": [64, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
724
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
725
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
726
+ },
727
+ "outputs": { "outputT": { "shape": [1, 2, 2053, 2053], "dtype": "float32" } },
728
+ "bench": {
729
+ "metrics": [
730
+ {
731
+ "type": "bandwidth",
732
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
733
+ },
734
+ {
735
+ "type": "gflops",
736
+ "name": "projection_and_row_scale",
737
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
738
+ }
739
+ ]
740
+ }
741
+ },
742
+ {
743
+ "name": "grpb-stream-b2-n3-s258-h31-d6-f32",
744
+ "preset": "stress",
745
+ "provenance": {
746
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
747
+ },
748
+ "attrs": { "num_heads": 3 },
749
+ "tunables": { "WORKGROUP_SIZE": 32 },
750
+ "vars": { "dtype": "float32", "count": 399384, "relCount": 199692 },
751
+ "inputs": {
752
+ "queryLayerT": { "shape": [2, 258, 93], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
753
+ "queryBiasT": { "shape": [93], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
754
+ "relPosT": { "shape": [1, 3, 258, 258], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
755
+ "weightT": { "shape": [31, 6], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
756
+ "biasT": { "shape": [6], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
757
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
758
+ },
759
+ "outputs": { "outputT": { "shape": [2, 3, 258, 258], "dtype": "float32" } },
760
+ "bench": {
761
+ "metrics": [
762
+ {
763
+ "type": "bandwidth",
764
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
765
+ },
766
+ {
767
+ "type": "gflops",
768
+ "name": "projection_and_row_scale",
769
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
770
+ }
771
+ ]
772
+ }
773
+ },
774
+ {
775
+ "name": "grpb-wg64-b2-n3-s257-h127-d8-f32",
776
+ "preset": "stress",
777
+ "provenance": {
778
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
779
+ },
780
+ "attrs": { "num_heads": 3 },
781
+ "tunables": { "WORKGROUP_SIZE": 64, "LANE_WORDS": 1 },
782
+ "vars": { "dtype": "float32", "count": 396294, "relCount": 198147 },
783
+ "inputs": {
784
+ "queryLayerT": { "shape": [2, 257, 381], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
785
+ "queryBiasT": { "shape": [381], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
786
+ "relPosT": { "shape": [1, 3, 257, 257], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
787
+ "weightT": { "shape": [127, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
788
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
789
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
790
+ },
791
+ "outputs": { "outputT": { "shape": [2, 3, 257, 257], "dtype": "float32" } },
792
+ "bench": {
793
+ "metrics": [
794
+ {
795
+ "type": "bandwidth",
796
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
797
+ },
798
+ {
799
+ "type": "gflops",
800
+ "name": "projection_and_row_scale",
801
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
802
+ }
803
+ ]
804
+ }
805
+ },
806
+ {
807
+ "name": "grpb-wg256-b2-n3-s256-h128-d32-f32",
808
+ "preset": "stress",
809
+ "provenance": {
810
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
811
+ },
812
+ "attrs": { "num_heads": 3 },
813
+ "tunables": { "WORKGROUP_SIZE": 256, "LANE_WORDS": 1 },
814
+ "vars": { "dtype": "float32", "count": 393216, "relCount": 196608 },
815
+ "inputs": {
816
+ "queryLayerT": { "shape": [2, 256, 384], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
817
+ "queryBiasT": { "shape": [384], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
818
+ "relPosT": { "shape": [1, 3, 256, 256], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
819
+ "weightT": { "shape": [128, 32], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
820
+ "biasT": { "shape": [32], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
821
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
822
+ },
823
+ "outputs": { "outputT": { "shape": [2, 3, 256, 256], "dtype": "float32" } },
824
+ "bench": {
825
+ "metrics": [
826
+ {
827
+ "type": "bandwidth",
828
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
829
+ },
830
+ {
831
+ "type": "gflops",
832
+ "name": "projection_and_row_scale",
833
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
834
+ }
835
+ ]
836
+ }
837
+ },
838
+ {
839
+ "name": "grpb-wg512-b2-n3-s128-h127-d32-f32",
840
+ "preset": "stress",
841
+ "provenance": {
842
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
843
+ },
844
+ "attrs": { "num_heads": 3 },
845
+ "tunables": { "WORKGROUP_SIZE": 512, "LANE_WORDS": 8 },
846
+ "vars": { "dtype": "float32", "count": 98304, "relCount": 49152 },
847
+ "inputs": {
848
+ "queryLayerT": { "shape": [2, 128, 381], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
849
+ "queryBiasT": { "shape": [381], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
850
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
851
+ "weightT": { "shape": [127, 32], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
852
+ "biasT": { "shape": [32], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
853
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
854
+ },
855
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float32" } },
856
+ "bench": {
857
+ "metrics": [
858
+ {
859
+ "type": "bandwidth",
860
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
861
+ },
862
+ {
863
+ "type": "gflops",
864
+ "name": "projection_and_row_scale",
865
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
866
+ }
867
+ ]
868
+ }
869
+ },
870
+ {
871
+ "name": "grpb-boundary-b2-n3-s128-h31-d8-f16",
872
+ "preset": "stress",
873
+ "provenance": {
874
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
875
+ },
876
+ "attrs": { "num_heads": 3 },
877
+ "vars": { "dtype": "float16", "count": 98304, "relCount": 49152 },
878
+ "inputs": {
879
+ "queryLayerT": { "shape": [2, 128, 93], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
880
+ "queryBiasT": { "shape": [93], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
881
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
882
+ "weightT": { "shape": [31, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
883
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
884
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
885
+ },
886
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float16" } },
887
+ "bench": {
888
+ "metrics": [
889
+ {
890
+ "type": "bandwidth",
891
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
892
+ },
893
+ {
894
+ "type": "gflops",
895
+ "name": "projection_and_row_scale",
896
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
897
+ }
898
+ ]
899
+ }
900
+ },
901
+ {
902
+ "name": "grpb-boundary-b2-n3-s128-h32-d8-f16",
903
+ "preset": "stress",
904
+ "provenance": {
905
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
906
+ },
907
+ "attrs": { "num_heads": 3 },
908
+ "vars": { "dtype": "float16", "count": 98304, "relCount": 49152 },
909
+ "inputs": {
910
+ "queryLayerT": { "shape": [2, 128, 96], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
911
+ "queryBiasT": { "shape": [96], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
912
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
913
+ "weightT": { "shape": [32, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
914
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
915
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
916
+ },
917
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float16" } },
918
+ "bench": {
919
+ "metrics": [
920
+ {
921
+ "type": "bandwidth",
922
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
923
+ },
924
+ {
925
+ "type": "gflops",
926
+ "name": "projection_and_row_scale",
927
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
928
+ }
929
+ ]
930
+ }
931
+ },
932
+ {
933
+ "name": "grpb-boundary-b2-n3-s127-h23-d6-f16",
934
+ "preset": "stress",
935
+ "provenance": {
936
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
937
+ },
938
+ "attrs": { "num_heads": 3 },
939
+ "vars": { "dtype": "float16", "count": 96774, "relCount": 48387 },
940
+ "inputs": {
941
+ "queryLayerT": { "shape": [2, 127, 69], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
942
+ "queryBiasT": { "shape": [69], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
943
+ "relPosT": { "shape": [1, 3, 127, 127], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
944
+ "weightT": { "shape": [23, 6], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
945
+ "biasT": { "shape": [6], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
946
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
947
+ },
948
+ "outputs": { "outputT": { "shape": [2, 3, 127, 127], "dtype": "float16" } },
949
+ "bench": {
950
+ "metrics": [
951
+ {
952
+ "type": "bandwidth",
953
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
954
+ },
955
+ {
956
+ "type": "gflops",
957
+ "name": "projection_and_row_scale",
958
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
959
+ }
960
+ ]
961
+ }
962
+ },
963
+ {
964
+ "name": "grpb-boundary-b2-n3-s127-h24-d6-f16",
965
+ "preset": "stress",
966
+ "provenance": {
967
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
968
+ },
969
+ "attrs": { "num_heads": 3 },
970
+ "vars": { "dtype": "float16", "count": 96774, "relCount": 48387 },
971
+ "inputs": {
972
+ "queryLayerT": { "shape": [2, 127, 72], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
973
+ "queryBiasT": { "shape": [72], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
974
+ "relPosT": { "shape": [1, 3, 127, 127], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
975
+ "weightT": { "shape": [24, 6], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
976
+ "biasT": { "shape": [6], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
977
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
978
+ },
979
+ "outputs": { "outputT": { "shape": [2, 3, 127, 127], "dtype": "float16" } },
980
+ "bench": {
981
+ "metrics": [
982
+ {
983
+ "type": "bandwidth",
984
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
985
+ },
986
+ {
987
+ "type": "gflops",
988
+ "name": "projection_and_row_scale",
989
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
990
+ }
991
+ ]
992
+ }
993
+ },
994
+ {
995
+ "name": "grpb-boundary-b2-n3-s128-h64-d10-f16",
996
+ "preset": "stress",
997
+ "provenance": {
998
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
999
+ },
1000
+ "attrs": { "num_heads": 3 },
1001
+ "vars": { "dtype": "float16", "count": 98304, "relCount": 49152 },
1002
+ "inputs": {
1003
+ "queryLayerT": { "shape": [2, 128, 192], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1004
+ "queryBiasT": { "shape": [192], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1005
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1006
+ "weightT": { "shape": [64, 10], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1007
+ "biasT": { "shape": [10], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1008
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1009
+ },
1010
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float16" } },
1011
+ "bench": {
1012
+ "metrics": [
1013
+ {
1014
+ "type": "bandwidth",
1015
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1016
+ },
1017
+ {
1018
+ "type": "gflops",
1019
+ "name": "projection_and_row_scale",
1020
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1021
+ }
1022
+ ]
1023
+ }
1024
+ },
1025
+ {
1026
+ "name": "grpb-general-b2-n3-s1-h1-d2-f16",
1027
+ "preset": "stress",
1028
+ "provenance": {
1029
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1030
+ },
1031
+ "attrs": { "num_heads": 3 },
1032
+ "vars": { "dtype": "float16", "count": 6, "relCount": 3 },
1033
+ "inputs": {
1034
+ "queryLayerT": { "shape": [2, 1, 3], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1035
+ "queryBiasT": { "shape": [3], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1036
+ "relPosT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1037
+ "weightT": { "shape": [1, 2], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1038
+ "biasT": { "shape": [2], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1039
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1040
+ },
1041
+ "outputs": { "outputT": { "shape": [2, 3, 1, 1], "dtype": "float16" } },
1042
+ "bench": {
1043
+ "metrics": [
1044
+ {
1045
+ "type": "bandwidth",
1046
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1047
+ },
1048
+ {
1049
+ "type": "gflops",
1050
+ "name": "projection_and_row_scale",
1051
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1052
+ }
1053
+ ]
1054
+ }
1055
+ },
1056
+ {
1057
+ "name": "grpb-general-b2-n3-s5-h3-d32-f16",
1058
+ "preset": "stress",
1059
+ "provenance": {
1060
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1061
+ },
1062
+ "attrs": { "num_heads": 3 },
1063
+ "vars": { "dtype": "float16", "count": 150, "relCount": 75 },
1064
+ "inputs": {
1065
+ "queryLayerT": { "shape": [2, 5, 9], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1066
+ "queryBiasT": { "shape": [9], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1067
+ "relPosT": { "shape": [1, 3, 5, 5], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1068
+ "weightT": { "shape": [3, 32], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1069
+ "biasT": { "shape": [32], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1070
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1071
+ },
1072
+ "outputs": { "outputT": { "shape": [2, 3, 5, 5], "dtype": "float16" } },
1073
+ "bench": {
1074
+ "metrics": [
1075
+ {
1076
+ "type": "bandwidth",
1077
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1078
+ },
1079
+ {
1080
+ "type": "gflops",
1081
+ "name": "projection_and_row_scale",
1082
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1083
+ }
1084
+ ]
1085
+ }
1086
+ },
1087
+ {
1088
+ "name": "grpb-general-b2-n3-s17-h8-d16-f16",
1089
+ "preset": "stress",
1090
+ "provenance": {
1091
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1092
+ },
1093
+ "attrs": { "num_heads": 3 },
1094
+ "vars": { "dtype": "float16", "count": 1734, "relCount": 867 },
1095
+ "inputs": {
1096
+ "queryLayerT": { "shape": [2, 17, 24], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1097
+ "queryBiasT": { "shape": [24], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1098
+ "relPosT": { "shape": [1, 3, 17, 17], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1099
+ "weightT": { "shape": [8, 16], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1100
+ "biasT": { "shape": [16], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1101
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1102
+ },
1103
+ "outputs": { "outputT": { "shape": [2, 3, 17, 17], "dtype": "float16" } },
1104
+ "bench": {
1105
+ "metrics": [
1106
+ {
1107
+ "type": "bandwidth",
1108
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1109
+ },
1110
+ {
1111
+ "type": "gflops",
1112
+ "name": "projection_and_row_scale",
1113
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1114
+ }
1115
+ ]
1116
+ }
1117
+ },
1118
+ {
1119
+ "name": "grpb-general-b2-n3-s31-h31-d6-f16",
1120
+ "preset": "stress",
1121
+ "provenance": {
1122
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1123
+ },
1124
+ "attrs": { "num_heads": 3 },
1125
+ "vars": { "dtype": "float16", "count": 5766, "relCount": 2883 },
1126
+ "inputs": {
1127
+ "queryLayerT": { "shape": [2, 31, 93], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1128
+ "queryBiasT": { "shape": [93], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1129
+ "relPosT": { "shape": [1, 3, 31, 31], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1130
+ "weightT": { "shape": [31, 6], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1131
+ "biasT": { "shape": [6], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1132
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1133
+ },
1134
+ "outputs": { "outputT": { "shape": [2, 3, 31, 31], "dtype": "float16" } },
1135
+ "bench": {
1136
+ "metrics": [
1137
+ {
1138
+ "type": "bandwidth",
1139
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1140
+ },
1141
+ {
1142
+ "type": "gflops",
1143
+ "name": "projection_and_row_scale",
1144
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1145
+ }
1146
+ ]
1147
+ }
1148
+ },
1149
+ {
1150
+ "name": "grpb-general-b2-n3-s64-h32-d32-f16",
1151
+ "preset": "stress",
1152
+ "provenance": {
1153
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1154
+ },
1155
+ "attrs": { "num_heads": 3 },
1156
+ "vars": { "dtype": "float16", "count": 24576, "relCount": 12288 },
1157
+ "inputs": {
1158
+ "queryLayerT": { "shape": [2, 64, 96], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1159
+ "queryBiasT": { "shape": [96], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1160
+ "relPosT": { "shape": [1, 3, 64, 64], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1161
+ "weightT": { "shape": [32, 32], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1162
+ "biasT": { "shape": [32], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1163
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1164
+ },
1165
+ "outputs": { "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16" } },
1166
+ "bench": {
1167
+ "metrics": [
1168
+ {
1169
+ "type": "bandwidth",
1170
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1171
+ },
1172
+ {
1173
+ "type": "gflops",
1174
+ "name": "projection_and_row_scale",
1175
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1176
+ }
1177
+ ]
1178
+ }
1179
+ },
1180
+ {
1181
+ "name": "grpb-general-b2-n3-s127-h64-d2-f16",
1182
+ "preset": "stress",
1183
+ "provenance": {
1184
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1185
+ },
1186
+ "attrs": { "num_heads": 3 },
1187
+ "vars": { "dtype": "float16", "count": 96774, "relCount": 48387 },
1188
+ "inputs": {
1189
+ "queryLayerT": { "shape": [2, 127, 192], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1190
+ "queryBiasT": { "shape": [192], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1191
+ "relPosT": { "shape": [1, 3, 127, 127], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1192
+ "weightT": { "shape": [64, 2], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1193
+ "biasT": { "shape": [2], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1194
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1195
+ },
1196
+ "outputs": { "outputT": { "shape": [2, 3, 127, 127], "dtype": "float16" } },
1197
+ "bench": {
1198
+ "metrics": [
1199
+ {
1200
+ "type": "bandwidth",
1201
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1202
+ },
1203
+ {
1204
+ "type": "gflops",
1205
+ "name": "projection_and_row_scale",
1206
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1207
+ }
1208
+ ]
1209
+ }
1210
+ },
1211
+ {
1212
+ "name": "grpb-general-b2-n3-s128-h63-d8-f16",
1213
+ "preset": "stress",
1214
+ "provenance": {
1215
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1216
+ },
1217
+ "attrs": { "num_heads": 3 },
1218
+ "vars": { "dtype": "float16", "count": 98304, "relCount": 49152 },
1219
+ "inputs": {
1220
+ "queryLayerT": { "shape": [2, 128, 189], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1221
+ "queryBiasT": { "shape": [189], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1222
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1223
+ "weightT": { "shape": [63, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1224
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1225
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1226
+ },
1227
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float16" } },
1228
+ "bench": {
1229
+ "metrics": [
1230
+ {
1231
+ "type": "bandwidth",
1232
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1233
+ },
1234
+ {
1235
+ "type": "gflops",
1236
+ "name": "projection_and_row_scale",
1237
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1238
+ }
1239
+ ]
1240
+ }
1241
+ },
1242
+ {
1243
+ "name": "grpb-general-b2-n3-s257-h127-d6-f16",
1244
+ "preset": "stress",
1245
+ "provenance": {
1246
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1247
+ },
1248
+ "attrs": { "num_heads": 3 },
1249
+ "vars": { "dtype": "float16", "count": 396294, "relCount": 198147 },
1250
+ "inputs": {
1251
+ "queryLayerT": { "shape": [2, 257, 381], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1252
+ "queryBiasT": { "shape": [381], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1253
+ "relPosT": { "shape": [1, 3, 257, 257], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1254
+ "weightT": { "shape": [127, 6], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1255
+ "biasT": { "shape": [6], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1256
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1257
+ },
1258
+ "outputs": { "outputT": { "shape": [2, 3, 257, 257], "dtype": "float16" } },
1259
+ "bench": {
1260
+ "metrics": [
1261
+ {
1262
+ "type": "bandwidth",
1263
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1264
+ },
1265
+ {
1266
+ "type": "gflops",
1267
+ "name": "projection_and_row_scale",
1268
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1269
+ }
1270
+ ]
1271
+ }
1272
+ },
1273
+ {
1274
+ "name": "grpb-general-b2-n3-s256-h128-d32-f16",
1275
+ "preset": "stress",
1276
+ "provenance": {
1277
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1278
+ },
1279
+ "attrs": { "num_heads": 3 },
1280
+ "vars": { "dtype": "float16", "count": 393216, "relCount": 196608 },
1281
+ "inputs": {
1282
+ "queryLayerT": { "shape": [2, 256, 384], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1283
+ "queryBiasT": { "shape": [384], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1284
+ "relPosT": { "shape": [1, 3, 256, 256], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1285
+ "weightT": { "shape": [128, 32], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1286
+ "biasT": { "shape": [32], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1287
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1288
+ },
1289
+ "outputs": { "outputT": { "shape": [2, 3, 256, 256], "dtype": "float16" } },
1290
+ "bench": {
1291
+ "metrics": [
1292
+ {
1293
+ "type": "bandwidth",
1294
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1295
+ },
1296
+ {
1297
+ "type": "gflops",
1298
+ "name": "projection_and_row_scale",
1299
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1300
+ }
1301
+ ]
1302
+ }
1303
+ },
1304
+ {
1305
+ "name": "grpb-general-b1-n4-s512-h256-d16-f16",
1306
+ "preset": "stress",
1307
+ "provenance": {
1308
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1309
+ },
1310
+ "attrs": { "num_heads": 4 },
1311
+ "vars": { "dtype": "float16", "count": 1048576, "relCount": 1048576 },
1312
+ "inputs": {
1313
+ "queryLayerT": { "shape": [1, 512, 1024], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1314
+ "queryBiasT": { "shape": [1024], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1315
+ "relPosT": { "shape": [1, 4, 512, 512], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1316
+ "weightT": { "shape": [256, 16], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1317
+ "biasT": { "shape": [16], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1318
+ "ecoAT": { "shape": [1, 4, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1319
+ },
1320
+ "outputs": { "outputT": { "shape": [1, 4, 512, 512], "dtype": "float16" } },
1321
+ "bench": {
1322
+ "metrics": [
1323
+ {
1324
+ "type": "bandwidth",
1325
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1326
+ },
1327
+ {
1328
+ "type": "gflops",
1329
+ "name": "projection_and_row_scale",
1330
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1331
+ }
1332
+ ]
1333
+ }
1334
+ },
1335
+ {
1336
+ "name": "grpb-general-b1-n2-s63-h512-d8-f16",
1337
+ "preset": "stress",
1338
+ "provenance": {
1339
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1340
+ },
1341
+ "attrs": { "num_heads": 2 },
1342
+ "vars": { "dtype": "float16", "count": 7938, "relCount": 7938 },
1343
+ "inputs": {
1344
+ "queryLayerT": { "shape": [1, 63, 1024], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1345
+ "queryBiasT": { "shape": [1024], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1346
+ "relPosT": { "shape": [1, 2, 63, 63], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1347
+ "weightT": { "shape": [512, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1348
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1349
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1350
+ },
1351
+ "outputs": { "outputT": { "shape": [1, 2, 63, 63], "dtype": "float16" } },
1352
+ "bench": {
1353
+ "metrics": [
1354
+ {
1355
+ "type": "bandwidth",
1356
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1357
+ },
1358
+ {
1359
+ "type": "gflops",
1360
+ "name": "projection_and_row_scale",
1361
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1362
+ }
1363
+ ]
1364
+ }
1365
+ },
1366
+ {
1367
+ "name": "grpb-general-b1-n2-s32-h1024-d32-f16",
1368
+ "preset": "stress",
1369
+ "provenance": {
1370
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1371
+ },
1372
+ "attrs": { "num_heads": 2 },
1373
+ "vars": { "dtype": "float16", "count": 2048, "relCount": 2048 },
1374
+ "inputs": {
1375
+ "queryLayerT": { "shape": [1, 32, 2048], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1376
+ "queryBiasT": { "shape": [2048], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1377
+ "relPosT": { "shape": [1, 2, 32, 32], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1378
+ "weightT": { "shape": [1024, 32], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1379
+ "biasT": { "shape": [32], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1380
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1381
+ },
1382
+ "outputs": { "outputT": { "shape": [1, 2, 32, 32], "dtype": "float16" } },
1383
+ "bench": {
1384
+ "metrics": [
1385
+ {
1386
+ "type": "bandwidth",
1387
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1388
+ },
1389
+ {
1390
+ "type": "gflops",
1391
+ "name": "projection_and_row_scale",
1392
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1393
+ }
1394
+ ]
1395
+ }
1396
+ },
1397
+ {
1398
+ "name": "grpb-general-b1-n2-s2053-h64-d8-f16",
1399
+ "preset": "stress",
1400
+ "provenance": {
1401
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1402
+ },
1403
+ "attrs": { "num_heads": 2 },
1404
+ "vars": { "dtype": "float16", "count": 8429618, "relCount": 8429618 },
1405
+ "inputs": {
1406
+ "queryLayerT": { "shape": [1, 2053, 128], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1407
+ "queryBiasT": { "shape": [128], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1408
+ "relPosT": { "shape": [1, 2, 2053, 2053], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1409
+ "weightT": { "shape": [64, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1410
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1411
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1412
+ },
1413
+ "outputs": { "outputT": { "shape": [1, 2, 2053, 2053], "dtype": "float16" } },
1414
+ "bench": {
1415
+ "metrics": [
1416
+ {
1417
+ "type": "bandwidth",
1418
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1419
+ },
1420
+ {
1421
+ "type": "gflops",
1422
+ "name": "projection_and_row_scale",
1423
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1424
+ }
1425
+ ]
1426
+ }
1427
+ },
1428
+ {
1429
+ "name": "grpb-stream-b2-n3-s258-h31-d6-f16",
1430
+ "preset": "stress",
1431
+ "provenance": {
1432
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1433
+ },
1434
+ "attrs": { "num_heads": 3 },
1435
+ "tunables": { "WORKGROUP_SIZE": 32 },
1436
+ "vars": { "dtype": "float16", "count": 399384, "relCount": 199692 },
1437
+ "inputs": {
1438
+ "queryLayerT": { "shape": [2, 258, 93], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1439
+ "queryBiasT": { "shape": [93], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1440
+ "relPosT": { "shape": [1, 3, 258, 258], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1441
+ "weightT": { "shape": [31, 6], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1442
+ "biasT": { "shape": [6], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1443
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1444
+ },
1445
+ "outputs": { "outputT": { "shape": [2, 3, 258, 258], "dtype": "float16" } },
1446
+ "bench": {
1447
+ "metrics": [
1448
+ {
1449
+ "type": "bandwidth",
1450
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1451
+ },
1452
+ {
1453
+ "type": "gflops",
1454
+ "name": "projection_and_row_scale",
1455
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1456
+ }
1457
+ ]
1458
+ }
1459
+ },
1460
+ {
1461
+ "name": "grpb-wg64-b2-n3-s257-h127-d8-f16",
1462
+ "preset": "stress",
1463
+ "provenance": {
1464
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1465
+ },
1466
+ "attrs": { "num_heads": 3 },
1467
+ "tunables": { "WORKGROUP_SIZE": 64, "LANE_WORDS": 1 },
1468
+ "vars": { "dtype": "float16", "count": 396294, "relCount": 198147 },
1469
+ "inputs": {
1470
+ "queryLayerT": { "shape": [2, 257, 381], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1471
+ "queryBiasT": { "shape": [381], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1472
+ "relPosT": { "shape": [1, 3, 257, 257], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1473
+ "weightT": { "shape": [127, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1474
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1475
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1476
+ },
1477
+ "outputs": { "outputT": { "shape": [2, 3, 257, 257], "dtype": "float16" } },
1478
+ "bench": {
1479
+ "metrics": [
1480
+ {
1481
+ "type": "bandwidth",
1482
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1483
+ },
1484
+ {
1485
+ "type": "gflops",
1486
+ "name": "projection_and_row_scale",
1487
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1488
+ }
1489
+ ]
1490
+ }
1491
+ },
1492
+ {
1493
+ "name": "grpb-wg256-b2-n3-s256-h128-d32-f16",
1494
+ "preset": "stress",
1495
+ "provenance": {
1496
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1497
+ },
1498
+ "attrs": { "num_heads": 3 },
1499
+ "tunables": { "WORKGROUP_SIZE": 256, "LANE_WORDS": 1 },
1500
+ "vars": { "dtype": "float16", "count": 393216, "relCount": 196608 },
1501
+ "inputs": {
1502
+ "queryLayerT": { "shape": [2, 256, 384], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1503
+ "queryBiasT": { "shape": [384], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1504
+ "relPosT": { "shape": [1, 3, 256, 256], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1505
+ "weightT": { "shape": [128, 32], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1506
+ "biasT": { "shape": [32], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1507
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1508
+ },
1509
+ "outputs": { "outputT": { "shape": [2, 3, 256, 256], "dtype": "float16" } },
1510
+ "bench": {
1511
+ "metrics": [
1512
+ {
1513
+ "type": "bandwidth",
1514
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1515
+ },
1516
+ {
1517
+ "type": "gflops",
1518
+ "name": "projection_and_row_scale",
1519
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1520
+ }
1521
+ ]
1522
+ }
1523
+ },
1524
+ {
1525
+ "name": "grpb-wg512-b2-n3-s128-h127-d32-f16",
1526
+ "preset": "stress",
1527
+ "provenance": {
1528
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1529
+ },
1530
+ "attrs": { "num_heads": 3 },
1531
+ "tunables": { "WORKGROUP_SIZE": 512, "LANE_WORDS": 8 },
1532
+ "vars": { "dtype": "float16", "count": 98304, "relCount": 49152 },
1533
+ "inputs": {
1534
+ "queryLayerT": { "shape": [2, 128, 381], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1535
+ "queryBiasT": { "shape": [381], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1536
+ "relPosT": { "shape": [1, 3, 128, 128], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1537
+ "weightT": { "shape": [127, 32], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1538
+ "biasT": { "shape": [32], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1539
+ "ecoAT": { "shape": [1, 3, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1540
+ },
1541
+ "outputs": { "outputT": { "shape": [2, 3, 128, 128], "dtype": "float16" } },
1542
+ "bench": {
1543
+ "metrics": [
1544
+ {
1545
+ "type": "bandwidth",
1546
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1547
+ },
1548
+ {
1549
+ "type": "gflops",
1550
+ "name": "projection_and_row_scale",
1551
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1552
+ }
1553
+ ]
1554
+ }
1555
+ },
1556
+ {
1557
+ "name": "grpb-vec-stream-b1-n2-s1056-h64-d8-f32",
1558
+ "preset": "stress",
1559
+ "provenance": {
1560
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1561
+ },
1562
+ "attrs": { "num_heads": 2 },
1563
+ "tunables": { "WORKGROUP_SIZE": 32 },
1564
+ "vars": { "dtype": "float32", "count": 2230272, "relCount": 2230272 },
1565
+ "inputs": {
1566
+ "queryLayerT": { "shape": [1, 1056, 128], "dtype": "float32", "dist": "normal", "seed": 701, "scale": 0.25 },
1567
+ "queryBiasT": { "shape": [128], "dtype": "float32", "dist": "normal", "seed": 702, "scale": 0.2 },
1568
+ "relPosT": { "shape": [1, 2, 1056, 1056], "dtype": "float32", "dist": "normal", "seed": 703, "scale": 1 },
1569
+ "weightT": { "shape": [64, 8], "dtype": "float32", "dist": "normal", "seed": 704, "scale": 0.25 },
1570
+ "biasT": { "shape": [8], "dtype": "float32", "dist": "normal", "seed": 705, "scale": 0.3 },
1571
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float32", "dist": "normal", "seed": 706, "scale": 1 }
1572
+ },
1573
+ "outputs": { "outputT": { "shape": [1, 2, 1056, 1056], "dtype": "float32" } },
1574
+ "bench": {
1575
+ "metrics": [
1576
+ {
1577
+ "type": "bandwidth",
1578
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1579
+ },
1580
+ {
1581
+ "type": "gflops",
1582
+ "name": "projection_and_row_scale",
1583
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1584
+ }
1585
+ ]
1586
+ }
1587
+ },
1588
+ {
1589
+ "name": "grpb-vec-stream-b1-n2-s1056-h64-d8-f16",
1590
+ "preset": "stress",
1591
+ "provenance": {
1592
+ "notes": "Head-projection generalization guard: odd dimensions, projection halves, row streaming and batch/head indexing."
1593
+ },
1594
+ "attrs": { "num_heads": 2 },
1595
+ "tunables": { "WORKGROUP_SIZE": 32 },
1596
+ "vars": { "dtype": "float16", "count": 2230272, "relCount": 2230272 },
1597
+ "inputs": {
1598
+ "queryLayerT": { "shape": [1, 1056, 128], "dtype": "float16", "dist": "normal", "seed": 701, "scale": 0.25 },
1599
+ "queryBiasT": { "shape": [128], "dtype": "float16", "dist": "normal", "seed": 702, "scale": 0.2 },
1600
+ "relPosT": { "shape": [1, 2, 1056, 1056], "dtype": "float16", "dist": "normal", "seed": 703, "scale": 1 },
1601
+ "weightT": { "shape": [64, 8], "dtype": "float16", "dist": "normal", "seed": 704, "scale": 0.25 },
1602
+ "biasT": { "shape": [8], "dtype": "float16", "dist": "normal", "seed": 705, "scale": 0.3 },
1603
+ "ecoAT": { "shape": [1, 2, 1, 1], "dtype": "float16", "dist": "normal", "seed": 706, "scale": 1 }
1604
+ },
1605
+ "outputs": { "outputT": { "shape": [1, 2, 1056, 1056], "dtype": "float16" } },
1606
+ "bench": {
1607
+ "metrics": [
1608
+ {
1609
+ "type": "bandwidth",
1610
+ "value": "(args.count + args.relCount + numel(shapes.queryLayerT) + numel(shapes.queryBiasT) + numel(shapes.weightT) + numel(shapes.biasT) + numel(shapes.ecoAT)) * dtypeBytes(args.dtype)"
1611
+ },
1612
+ {
1613
+ "type": "gflops",
1614
+ "name": "projection_and_row_scale",
1615
+ "value": "2 * numel(shapes.queryLayerT) * dim(shapes.weightT, 1) + numel(shapes.outputT)"
1616
+ }
1617
+ ]
1618
+ }
1619
+ }
1620
+ ]
1621
+ }
build/webgpu/gated-relative-position-bias.wgsl.jinja ADDED
@@ -0,0 +1,195 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {% set parallelProjection = parallelProjection | default(false) %}
2
+ {% set useSubgroups = useSubgroups | default(false) %}
3
+ {% if useSubgroups %}
4
+ enable subgroups;
5
+ {% endif %}
6
+ {{ env.wgsl.resourceDeclarations }}
7
+
8
+ // One workgroup projects a query/head row into two float32 gate sums, then
9
+ // streams the relative-position row. Column lanes stage the query for contiguous
10
+ // weight reads. Head lanes keep column accumulators in registers when the narrow
11
+ // projection would leave most execution lanes idle; both layouts narrow only at
12
+ // the output store.
13
+ const HEAD_SIZE: u32 = {{ headSize }}u;
14
+ {% if not parallelProjection %}
15
+ const PROJ_DIM: u32 = {{ projDim }}u;
16
+ const HALF_PROJ: u32 = {{ halfProj }}u;
17
+ {% endif %}
18
+ const NUM_HEADS: u32 = {{ numHeads }}u;
19
+ const ROW_WORDS: u32 = {{ rowWords }}u;
20
+ const WG: u32 = {{ workgroupSize }}u;
21
+ {% if parallelProjection %}
22
+ var<workgroup> parts: array<vec2<f32>, WG>;
23
+ {% if useSubgroups %}
24
+ var<workgroup> row_gate: f32;
25
+ {% else %}
26
+ {% macro wgsl_tree_reduce_f32(name, mode, buffer="partial", wg="WG", trailingBarrier=true, valueType="f32") %}
27
+ fn {{ name }}(value: {{ valueType }}, tid: u32) -> {{ valueType }} {
28
+ {{ buffer }}[tid] = value;
29
+ workgroupBarrier();
30
+ // Ceil-halving keeps every lane when the workgroup size is not a power of
31
+ // two. For even n this matches the power-of-two tree order; for odd n, lanes
32
+ // [0, n-half) fold the upper tail while the middle lane carries forward.
33
+ var n: u32 = {{ wg }};
34
+ loop {
35
+ let half = (n + 1u) / 2u;
36
+ if (tid < n - half) {
37
+ {{ buffer }}[tid] = {{ buffer }}[tid] + {{ buffer }}[tid + half];
38
+ }
39
+ workgroupBarrier();
40
+ n = half;
41
+ if (n == 1u) {
42
+ break;
43
+ }
44
+ }
45
+ // The default trailing barrier makes this helper safe for back-to-back calls: every lane reads
46
+ // slot 0 here, so the next call's first store must not run until all lanes have read it.
47
+ // `trailingBarrier=false` is safe only when the buffer is never written again before kernel exit.
48
+ let reduced = {{ buffer }}[0];
49
+ return reduced;
50
+ }
51
+ {% endmacro %}
52
+ {{ wgsl_tree_reduce_f32("reduce_gates", "sum", "parts", "WG", false, "vec2<f32>") }}
53
+ {% endif %}
54
+ {% else %}
55
+ var<workgroup> query_row: array<f32, HEAD_SIZE>;
56
+ var<workgroup> gate_u_parts: array<f32, WG>;
57
+ var<workgroup> gate_r_parts: array<f32, WG>;
58
+ var<workgroup> row_gate: f32;
59
+ {% endif %}
60
+ fn sigmoid_safe(x: f32) -> f32 {
61
+ if (x >= 0.0) {
62
+ let z = exp(-x);
63
+ return 1.0 / (1.0 + z);
64
+ }
65
+ let z = exp(x);
66
+ return z / (1.0 + z);
67
+ }
68
+ @compute @workgroup_size(WG)
69
+ {% if parallelProjection %}
70
+ fn main(@builtin(workgroup_id) wg: vec3<u32>, @builtin(local_invocation_index) lane: u32{% if useSubgroups %},
71
+ @builtin(subgroup_id) sgId: u32, @builtin(num_subgroups) numSg: u32{% endif %}) {
72
+ {% else %}
73
+ fn main(@builtin(workgroup_id) wg: vec3<u32>, @builtin(local_invocation_id) lid: vec3<u32>) {
74
+ {% endif %}
75
+ let row = wg.x + wg.y * {{ DISPATCH_FOLD_WIDTH }}u;
76
+ {% if not parallelProjection %}
77
+ let lane = lid.x;
78
+ {% endif %}
79
+ let position = row % params.seqLen;
80
+ let head = (row / params.seqLen) % NUM_HEADS;
81
+ let batch = row / (params.seqLen * NUM_HEADS);
82
+ let queryBase = (batch * params.seqLen + position) * (NUM_HEADS * HEAD_SIZE) + head * HEAD_SIZE;
83
+ {% if parallelProjection %}
84
+ {% for d in range(projDim) %}
85
+ var acc{{ d }} = 0.0;
86
+ {% endfor %}
87
+ for (var h = lane; h < HEAD_SIZE; h += WG) {
88
+ let q = f32(query_layer[queryBase + h]) + f32(query_bias[head * HEAD_SIZE + h]);
89
+ {% for d in range(projDim) %}
90
+ acc{{ d }} = fma(q, f32(weight[h * {{ projDim }}u + {{ d }}u]), acc{{ d }});
91
+ {% endfor %}
92
+ }
93
+ var partial = vec2<f32>(
94
+ {% for d in range(halfProj) %}{{ "+ " if not loop.first else "" }}acc{{ d }}{% endfor %},
95
+ {% for d in range(halfProj, projDim) %}{{ "+ " if not loop.first else "" }}acc{{ d }}{% endfor %});
96
+ partial = select(partial, partial + vec2<f32>(
97
+ {% for d in range(halfProj) %}{{ "+ " if not loop.first else "" }}f32(bias[{{ d }}u]){% endfor %},
98
+ {% for d in range(halfProj, projDim) %}{{ "+ " if not loop.first else "" }}f32(bias[{{ d }}u]){% endfor %}), lane == 0u);
99
+ {% if useSubgroups %}
100
+ let folded = subgroupAdd(partial);
101
+ if (subgroupElect()) { parts[sgId] = folded; }
102
+ workgroupBarrier();
103
+ if (lane == 0u) {
104
+ var total = vec2<f32>(0.0);
105
+ for (var s = 0u; s < numSg; s += 1u) { total += parts[s]; }
106
+ row_gate = fma(sigmoid_safe(total.x), fma(sigmoid_safe(total.y), f32(eco_a[head]), -1.0), 2.0);
107
+ }
108
+ workgroupBarrier();
109
+ let gate = row_gate;
110
+ {% else %}
111
+ let total = reduce_gates(partial, lane);
112
+ let gate = fma(sigmoid_safe(total.x), fma(sigmoid_safe(total.y), f32(eco_a[head]), -1.0), 2.0);
113
+ {% endif %}
114
+ {% else %}
115
+ // Stage the head's slice of the biased query row.
116
+ for (var h = lane; h < HEAD_SIZE; h = h + WG) {
117
+ query_row[h] = f32(query_layer[queryBase + h]) + f32(query_bias[head * HEAD_SIZE + h]);
118
+ }
119
+ workgroupBarrier();
120
+
121
+ // Each lane owns a set of projection columns; the two halves accumulate apart so
122
+ // the row fold below carries both gates at once.
123
+ var uPart = 0.0;
124
+ var rPart = 0.0;
125
+ for (var d = lane; d < PROJ_DIM; d = d + WG) {
126
+ var acc = f32(bias[d]);
127
+ for (var h = 0u; h < HEAD_SIZE; h = h + 1u) {
128
+ acc = fma(query_row[h], f32(weight[h * PROJ_DIM + d]), acc);
129
+ }
130
+ if (d < HALF_PROJ) {
131
+ uPart = uPart + acc;
132
+ } else {
133
+ rPart = rPart + acc;
134
+ }
135
+ }
136
+ gate_u_parts[lane] = uPart;
137
+ gate_r_parts[lane] = rPart;
138
+ workgroupBarrier();
139
+ {% macro wgsl_tree_fold_stmt(a, op, idx, svar) %}
140
+ {% if op == "max" or op == "min" %}
141
+ {{ a }}[{{ idx }}] = {{ op }}({{ a }}[{{ idx }}], {{ a }}[{{ idx }} + {{ svar }}]);{% else %}
142
+ {{ a }}[{{ idx }}] = {{ a }}[{{ idx }}] {{ "*" if op == "prod" else "+" }} {{ a }}[{{ idx }} + {{ svar }}];{% endif %}{% endmacro %}
143
+ {% macro wgsl_tree_fold(arrays, op="add", idx="lid", wg="WORKGROUP_SIZE", svar="stride", typed=false, form="tail", breakInline=false, bodyInline=false, barrierFirst=false, reuse=false) %}
144
+ var {{ svar }}{{ ": u32 " if typed else " " }}= {{ wg }} / 2u;
145
+ loop {
146
+ if ({{ svar }} == 0u) {
147
+ break;
148
+ }
149
+ if ({{ idx }} < {{ svar }}) {
150
+ {% for a in arrays %}
151
+ {{ wgsl_tree_fold_stmt(a, op, idx, svar) }}
152
+ {% endfor %}
153
+ }
154
+ workgroupBarrier();
155
+ {{ svar }} = {{ svar }} / 2u;
156
+ }{% endmacro %}
157
+ {{ wgsl_tree_fold(["gate_u_parts", "gate_r_parts"], idx="lane", wg="WG", form="head", barrierFirst=true) }}
158
+ if (lane == 0u) {
159
+ let gateU = sigmoid_safe(gate_u_parts[0]);
160
+ let gateR = sigmoid_safe(gate_r_parts[0]);
161
+ row_gate = fma(gateU, fma(gateR, f32(eco_a[head]), -1.0), 2.0);
162
+ }
163
+ workgroupBarrier();
164
+ let gate = row_gate;
165
+
166
+ {% endif %}
167
+ let relBase = (head * params.seqLen + position) * ROW_WORDS;
168
+ let outBase = row * ROW_WORDS;
169
+ {% if stageRow %}
170
+ {% for i in range(wordsPerLane) %}
171
+ {
172
+ let c = lane + {{ i }}u * WG;
173
+ {% if not parallelProjection or rowWords % workgroupSize != 0 %}
174
+ if (c < ROW_WORDS) {
175
+ {% endif %}
176
+ {% if useVec4 %}
177
+ output[outBase + c] = {{ vectorScalar }}(vec4<f32>(rel_pos[relBase + c]) * gate);
178
+ {% else %}
179
+ output[outBase + c] = {{ scalar }}(f32(rel_pos[relBase + c]) * gate);
180
+ {% endif %}
181
+ {% if not parallelProjection or rowWords % workgroupSize != 0 %}
182
+ }
183
+ {% endif %}
184
+ }
185
+ {% endfor %}
186
+ {% else %}
187
+ for (var c = lane; c < ROW_WORDS; {% if parallelProjection %}c += WG{% else %}c = c + WG{% endif %}) {
188
+ {% if useVec4 %}
189
+ output[outBase + c] = {{ vectorScalar }}(vec4<f32>(rel_pos[relBase + c]) * gate);
190
+ {% else %}
191
+ output[outBase + c] = {{ scalar }}(f32(rel_pos[relBase + c]) * gate);
192
+ {% endif %}
193
+ }
194
+ {% endif %}
195
+ }
build/webgpu/manifest.json ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "domain": "com.microsoft",
3
+ "name": "GatedRelativePositionBias",
4
+ "sinceVersion": 1,
5
+ "inputs": {
6
+ "queryLayerT": { "onnx": "query_layer", "dtype": "T", "rank": 3 },
7
+ "queryBiasT": { "onnx": "query_bias", "dtype": "T", "rank": 1 },
8
+ "relPosT": { "onnx": "rel_pos", "dtype": "T", "rank": 4 },
9
+ "weightT": { "onnx": "weight", "dtype": "T", "rank": 2 },
10
+ "biasT": { "onnx": "bias", "dtype": "T", "rank": 1 },
11
+ "ecoAT": { "onnx": "eco_a", "dtype": "T", "rank": 4 }
12
+ },
13
+ "outputs": {
14
+ "outputT": {
15
+ "onnx": "output",
16
+ "dtype": "T",
17
+ "rank": 4,
18
+ "shape": "[dim(shapes.queryLayerT, 0), attrs.num_heads, dim(shapes.queryLayerT, 1), dim(shapes.queryLayerT, 1)]"
19
+ }
20
+ },
21
+ "attributes": { "num_heads": {} },
22
+ "attributeConstraints": { "num_heads": { "required": true } },
23
+ "typeConstraints": { "T": ["float32", "float16"] },
24
+ "tunables": { "WORKGROUP_SIZE": { "default": 128 }, "LANE_WORDS": { "default": 2 } },
25
+ "derive": {
26
+ "deviceWorkgroupCap": "min(device.limits.maxComputeInvocationsPerWorkgroup, device.limits.maxComputeWorkgroupSizeX)",
27
+ "foldedDispatchCapacity": "min(device.limits.maxComputeWorkgroupsPerDimension, 65535) * min(device.limits.maxComputeWorkgroupsPerDimension, 65535)",
28
+ "variableSubgroup16To32": "device.features.has(\"subgroups\") and has(device.adapterInfo, \"subgroupMinSize\") and has(device.adapterInfo, \"subgroupMaxSize\") and device.adapterInfo.subgroupMinSize == 16 and device.adapterInfo.subgroupMaxSize == 32",
29
+ "batchSize": "dim(shapes.queryLayerT, 0)",
30
+ "seqLen": "dim(shapes.queryLayerT, 1)",
31
+ "queryHidden": "dim(shapes.queryLayerT, 2)",
32
+ "numHeads": "attrs.num_heads",
33
+ "headSize": "queryHidden / max(1, numHeads)",
34
+ "projDim": "dim(shapes.weightT, 1)",
35
+ "halfProj": "projDim / 2",
36
+ "rowCount": "batchSize * numHeads * seqLen",
37
+ "dtypeOk": "tensorDtypes.queryLayerT == tensorDtypes.queryBiasT and tensorDtypes.queryLayerT == tensorDtypes.relPosT and tensorDtypes.queryLayerT == tensorDtypes.weightT and tensorDtypes.queryLayerT == tensorDtypes.biasT and tensorDtypes.queryLayerT == tensorDtypes.ecoAT and tensorDtypes.queryLayerT == tensorDtypes.outputT and f16Ok(dtypes.T)",
38
+ "headsOk": "numHeads > 0 and queryHidden > 0 and queryHidden % numHeads == 0 and headSize > 0",
39
+ "projOk": "ranks.weightT == 2 and dim(shapes.weightT, 0) == headSize and projDim > 0 and projDim % 2 == 0 and projDim <= 32 and ranks.biasT == 1 and dim(shapes.biasT, 0) == projDim",
40
+ "relPosOk": "ranks.relPosT == 4 and dim(shapes.relPosT, 0) == 1 and dim(shapes.relPosT, 1) == numHeads and dim(shapes.relPosT, 2) == seqLen and dim(shapes.relPosT, 3) == seqLen",
41
+ "ecoOk": "ranks.ecoAT == 4 and numel(shapes.ecoAT) == numHeads",
42
+ "outputOk": "ranks.outputT == 4 and dim(shapes.outputT, 0) == batchSize and dim(shapes.outputT, 1) == numHeads and dim(shapes.outputT, 2) == seqLen and dim(shapes.outputT, 3) == seqLen",
43
+ "biasOk": "ranks.queryBiasT == 1 and dim(shapes.queryBiasT, 0) == queryHidden",
44
+ "contractOk": "ranks.queryLayerT == 3 and seqLen > 0 and dtypeOk and headsOk and projOk and relPosOk and ecoOk and outputOk and biasOk",
45
+ "workgroupOk": "tunables.WORKGROUP_SIZE > 0 and tunables.WORKGROUP_SIZE <= deviceWorkgroupCap and pow2ceil(tunables.WORKGROUP_SIZE) == tunables.WORKGROUP_SIZE and tunables.LANE_WORDS > 0 and headSize + tunables.WORKGROUP_SIZE * 2 + 1 <= device.limits.maxComputeWorkgroupStorageSize / 4",
46
+ "dispatchOk": "rowCount <= foldedDispatchCapacity",
47
+ "vec4Ok": "seqLen % 4 == 0",
48
+ "vecWords": "seqLen / 4",
49
+ "vecWorkgroup": "max(32, min(tunables.WORKGROUP_SIZE, pow2ceil(max(1, ceilDiv(vecWords, max(1, tunables.LANE_WORDS))))))",
50
+ "scalarWorkgroup": "max(32, min(tunables.WORKGROUP_SIZE, pow2ceil(max(1, ceilDiv(seqLen, max(1, tunables.LANE_WORDS))))))",
51
+ "vecWordsPerLane": "ceilDiv(vecWords, max(1, vecWorkgroup))",
52
+ "scalarWordsPerLane": "ceilDiv(seqLen, max(1, scalarWorkgroup))",
53
+ "vectorElement": "\"vec4<\" ~ dtypes.T ~ \">\"",
54
+ "projectionLaneWidth": "min(vecWorkgroup, scalarWorkgroup, device.adapterInfo.subgroupMinSize) if (has(device.adapterInfo, \"subgroupMinSize\") and device.adapterInfo.subgroupMinSize > 0) else min(vecWorkgroup, scalarWorkgroup, 32)",
55
+ "headParallelPreferred": "min(headSize, projectionLaneWidth) >= 4 * min(projDim, projectionLaneWidth) or ((variableSubgroup16To32 or (not device.features.has(\"subgroups\") and not has(device.adapterInfo, \"subgroupMinSize\"))) and vec4Ok and headSize >= 8 * projDim and seqLen >= 2 * tunables.WORKGROUP_SIZE)",
56
+ "useVec4": "vec4Ok",
57
+ "rowWords": "vecWords if vec4Ok else seqLen",
58
+ "wordsPerLane": "vecWordsPerLane if vec4Ok else scalarWordsPerLane",
59
+ "stageRow": "wordsPerLane <= 8",
60
+ "workgroupSize": "vecWorkgroup if vec4Ok else scalarWorkgroup",
61
+ "scalar": "dtypes.T",
62
+ "vectorScalar": "vectorElement",
63
+ "rowElement": "vectorElement if vec4Ok else scalar"
64
+ },
65
+ "when": ["contractOk", "workgroupOk", "dispatchOk"],
66
+ "bindings": {
67
+ "query_layer": { "arg": "queryLayerT", "elementType": "$scalar" },
68
+ "query_bias": { "arg": "queryBiasT", "elementType": "$scalar" },
69
+ "weight": { "arg": "weightT", "elementType": "$scalar" },
70
+ "bias": { "arg": "biasT", "elementType": "$scalar" },
71
+ "eco_a": { "arg": "ecoAT", "elementType": "$scalar" },
72
+ "rel_pos": { "arg": "relPosT", "elementType": "$rowElement" },
73
+ "output": { "arg": "outputT", "elementType": "$rowElement" },
74
+ "params": { "struct": [{ "name": "seqLen", "type": "u32", "value": "seqLen" }] }
75
+ },
76
+ "variants": [
77
+ {
78
+ "id": "row",
79
+ "priority": 10,
80
+ "passes": [
81
+ {
82
+ "id": "main",
83
+ "name": "GatedRelativePositionBias.Row",
84
+ "shader": "gated-relative-position-bias.wgsl.jinja",
85
+ "bindings": ["query_layer", "query_bias", "weight", "bias", "eco_a", "rel_pos", "output", "params"],
86
+ "dispatch": { "x": "min(rowCount, DISPATCH_FOLD_WIDTH)", "y": "ceilDiv(rowCount, DISPATCH_FOLD_WIDTH)" }
87
+ }
88
+ ]
89
+ },
90
+ {
91
+ "id": "row_head_tree",
92
+ "priority": 30,
93
+ "when": ["true"],
94
+ "demoteWhen": ["not headParallelPreferred"],
95
+ "requires": { "features": [] },
96
+ "derive": { "parallelProjection": true, "useSubgroups": false },
97
+ "passes": [
98
+ {
99
+ "id": "main",
100
+ "name": "GatedRelativePositionBias.RowHeadTree",
101
+ "shader": "gated-relative-position-bias.wgsl.jinja",
102
+ "bindings": ["query_layer", "query_bias", "weight", "bias", "eco_a", "rel_pos", "output", "params"],
103
+ "dispatch": { "x": "min(rowCount, DISPATCH_FOLD_WIDTH)", "y": "ceilDiv(rowCount, DISPATCH_FOLD_WIDTH)" }
104
+ }
105
+ ]
106
+ },
107
+ {
108
+ "id": "row_head_subgroups",
109
+ "priority": 40,
110
+ "when": ["device.wgslLanguageFeatures.has(\"subgroup_id\")"],
111
+ "demoteWhen": ["not headParallelPreferred"],
112
+ "requires": { "features": ["subgroups"] },
113
+ "derive": { "parallelProjection": true, "useSubgroups": true },
114
+ "passes": [
115
+ {
116
+ "id": "main",
117
+ "name": "GatedRelativePositionBias.RowHeadSubgroups",
118
+ "shader": "gated-relative-position-bias.wgsl.jinja",
119
+ "bindings": ["query_layer", "query_bias", "weight", "bias", "eco_a", "rel_pos", "output", "params"],
120
+ "dispatch": { "x": "min(rowCount, DISPATCH_FOLD_WIDTH)", "y": "ceilDiv(rowCount, DISPATCH_FOLD_WIDTH)" }
121
+ }
122
+ ]
123
+ }
124
+ ]
125
+ }
build/webgpu/metadata.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "com.microsoft.GatedRelativePositionBias",
3
+ "id": "_com_microsoft_gatedrelativepositionbias_webgpu_ac02f73",
4
+ "version": 1,
5
+ "license": "Apache-2.0",
6
+ "backend": { "type": "webgpu" },
7
+ "digest": {
8
+ "algorithm": "sha256",
9
+ "files": {
10
+ "bench.json": "hmOUnRZGYrFPY2lGdNGbL7Wm2HTw3AlM0T79OkEwd7o=",
11
+ "gated-relative-position-bias.wgsl.jinja": "wSm5dB+odVwYIT3J2CSe2JSdrHdZRGogi4ie9DFe4Fs=",
12
+ "manifest.json": "o9xF1iE0tRsDjMoY5Q99zW6vCn57vJI5W/f31HmhuLA=",
13
+ "test.json": "sTb8nEnp/Vwfqei65rVWDVUYEe0AWCv78pxVjyzkIYM="
14
+ }
15
+ },
16
+ "provenance": { "kernel": { "sha": "6fdf6301e2bbcc2f03bf1eaf493b7ad55ef33afc", "dirty": false } },
17
+ "webgpu": {
18
+ "manifestSpec": "2.1",
19
+ "variants": {
20
+ "row": ["gated-relative-position-bias.wgsl.jinja"],
21
+ "row_head_tree": ["gated-relative-position-bias.wgsl.jinja"],
22
+ "row_head_subgroups": ["gated-relative-position-bias.wgsl.jinja"]
23
+ }
24
+ }
25
+ }
build/webgpu/test.json ADDED
@@ -0,0 +1,2478 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cases": [
3
+ {
4
+ "name": "tiny_b1_h2_s4_head8_d4",
5
+ "provenance": {
6
+ "notes": "Smallest shape that still distinguishes the two gate halves, the per-head eco_a and the column index."
7
+ },
8
+ "attrs": { "num_heads": 2 },
9
+ "inputs": {
10
+ "queryLayerT": {
11
+ "dtype": "float32",
12
+ "shape": [1, 4, 16],
13
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
14
+ },
15
+ "queryBiasT": {
16
+ "dtype": "float32",
17
+ "shape": [16],
18
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
19
+ },
20
+ "relPosT": {
21
+ "dtype": "float32",
22
+ "shape": [1, 2, 4, 4],
23
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
24
+ },
25
+ "weightT": {
26
+ "dtype": "float32",
27
+ "shape": [8, 4],
28
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
29
+ },
30
+ "biasT": {
31
+ "dtype": "float32",
32
+ "shape": [4],
33
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
34
+ },
35
+ "ecoAT": {
36
+ "dtype": "float32",
37
+ "shape": [1, 2, 1, 1],
38
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
39
+ }
40
+ },
41
+ "outputs": {
42
+ "outputT": { "dtype": "float32", "shape": [1, 2, 4, 4], "tolerance": 0.000002, "relTolerance": 0.00002 }
43
+ }
44
+ },
45
+ {
46
+ "name": "batch2_heads3_s8_head16_d8",
47
+ "provenance": {
48
+ "notes": "Batch 2 with three heads: the batch stride is zero at batch 1, so a wrong batch term is invisible without this."
49
+ },
50
+ "attrs": { "num_heads": 3 },
51
+ "inputs": {
52
+ "queryLayerT": {
53
+ "dtype": "float32",
54
+ "shape": [2, 8, 48],
55
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
56
+ },
57
+ "queryBiasT": {
58
+ "dtype": "float32",
59
+ "shape": [48],
60
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
61
+ },
62
+ "relPosT": {
63
+ "dtype": "float32",
64
+ "shape": [1, 3, 8, 8],
65
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
66
+ },
67
+ "weightT": {
68
+ "dtype": "float32",
69
+ "shape": [16, 8],
70
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
71
+ },
72
+ "biasT": {
73
+ "dtype": "float32",
74
+ "shape": [8],
75
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
76
+ },
77
+ "ecoAT": {
78
+ "dtype": "float32",
79
+ "shape": [1, 3, 1, 1],
80
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
81
+ }
82
+ },
83
+ "outputs": {
84
+ "outputT": { "dtype": "float32", "shape": [2, 3, 8, 8], "tolerance": 0.000002, "relTolerance": 0.00002 }
85
+ }
86
+ },
87
+ {
88
+ "name": "batch2_heads3_s8_head16_d8_f16",
89
+ "attrs": { "num_heads": 3 },
90
+ "inputs": {
91
+ "queryLayerT": {
92
+ "dtype": "float16",
93
+ "shape": [2, 8, 48],
94
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
95
+ },
96
+ "queryBiasT": {
97
+ "dtype": "float16",
98
+ "shape": [48],
99
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
100
+ },
101
+ "relPosT": {
102
+ "dtype": "float16",
103
+ "shape": [1, 3, 8, 8],
104
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
105
+ },
106
+ "weightT": {
107
+ "dtype": "float16",
108
+ "shape": [16, 8],
109
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
110
+ },
111
+ "biasT": {
112
+ "dtype": "float16",
113
+ "shape": [8],
114
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
115
+ },
116
+ "ecoAT": {
117
+ "dtype": "float16",
118
+ "shape": [1, 3, 1, 1],
119
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
120
+ }
121
+ },
122
+ "outputs": {
123
+ "outputT": { "dtype": "float16", "shape": [2, 3, 8, 8], "tolerance": 0.0005, "relTolerance": 0.002 }
124
+ }
125
+ },
126
+ {
127
+ "name": "odd_seq_len_six_scalar_route",
128
+ "provenance": { "notes": "A sequence length that is not a multiple of four, so the scalar route runs." },
129
+ "attrs": { "num_heads": 2 },
130
+ "inputs": {
131
+ "queryLayerT": {
132
+ "dtype": "float32",
133
+ "shape": [1, 6, 16],
134
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
135
+ },
136
+ "queryBiasT": {
137
+ "dtype": "float32",
138
+ "shape": [16],
139
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
140
+ },
141
+ "relPosT": {
142
+ "dtype": "float32",
143
+ "shape": [1, 2, 6, 6],
144
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
145
+ },
146
+ "weightT": {
147
+ "dtype": "float32",
148
+ "shape": [8, 4],
149
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
150
+ },
151
+ "biasT": {
152
+ "dtype": "float32",
153
+ "shape": [4],
154
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
155
+ },
156
+ "ecoAT": {
157
+ "dtype": "float32",
158
+ "shape": [1, 2, 1, 1],
159
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
160
+ }
161
+ },
162
+ "outputs": {
163
+ "outputT": { "dtype": "float32", "shape": [1, 2, 6, 6], "tolerance": 0.000002, "relTolerance": 0.00002 }
164
+ }
165
+ },
166
+ {
167
+ "name": "seq_len_one",
168
+ "provenance": {
169
+ "notes": "A single query position: the row is one element wide and the gate is the whole computation."
170
+ },
171
+ "attrs": { "num_heads": 3 },
172
+ "inputs": {
173
+ "queryLayerT": {
174
+ "dtype": "float32",
175
+ "shape": [1, 1, 24],
176
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
177
+ },
178
+ "queryBiasT": {
179
+ "dtype": "float32",
180
+ "shape": [24],
181
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
182
+ },
183
+ "relPosT": {
184
+ "dtype": "float32",
185
+ "shape": [1, 3, 1, 1],
186
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
187
+ },
188
+ "weightT": {
189
+ "dtype": "float32",
190
+ "shape": [8, 4],
191
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
192
+ },
193
+ "biasT": {
194
+ "dtype": "float32",
195
+ "shape": [4],
196
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
197
+ },
198
+ "ecoAT": {
199
+ "dtype": "float32",
200
+ "shape": [1, 3, 1, 1],
201
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
202
+ }
203
+ },
204
+ "outputs": {
205
+ "outputT": { "dtype": "float32", "shape": [1, 3, 1, 1], "tolerance": 0.000002, "relTolerance": 0.00002 }
206
+ }
207
+ },
208
+ {
209
+ "name": "proj_dim_two_minimum",
210
+ "provenance": { "notes": "D = 2, the smallest even projection width, so each gate is a single column." },
211
+ "attrs": { "num_heads": 2 },
212
+ "inputs": {
213
+ "queryLayerT": {
214
+ "dtype": "float32",
215
+ "shape": [1, 8, 32],
216
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
217
+ },
218
+ "queryBiasT": {
219
+ "dtype": "float32",
220
+ "shape": [32],
221
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
222
+ },
223
+ "relPosT": {
224
+ "dtype": "float32",
225
+ "shape": [1, 2, 8, 8],
226
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
227
+ },
228
+ "weightT": {
229
+ "dtype": "float32",
230
+ "shape": [16, 2],
231
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
232
+ },
233
+ "biasT": {
234
+ "dtype": "float32",
235
+ "shape": [2],
236
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
237
+ },
238
+ "ecoAT": {
239
+ "dtype": "float32",
240
+ "shape": [1, 2, 1, 1],
241
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
242
+ }
243
+ },
244
+ "outputs": {
245
+ "outputT": { "dtype": "float32", "shape": [1, 2, 8, 8], "tolerance": 0.000002, "relTolerance": 0.00002 }
246
+ }
247
+ },
248
+ {
249
+ "name": "proj_dim_thirty_two_maximum",
250
+ "provenance": {
251
+ "notes": "Projection width D=32, the largest used anywhere in this file, checks GatedRelativePositionBias with num_heads=2 at its largest per-head projection size."
252
+ },
253
+ "attrs": { "num_heads": 2 },
254
+ "inputs": {
255
+ "queryLayerT": {
256
+ "dtype": "float32",
257
+ "shape": [1, 8, 32],
258
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
259
+ },
260
+ "queryBiasT": {
261
+ "dtype": "float32",
262
+ "shape": [32],
263
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
264
+ },
265
+ "relPosT": {
266
+ "dtype": "float32",
267
+ "shape": [1, 2, 8, 8],
268
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
269
+ },
270
+ "weightT": {
271
+ "dtype": "float32",
272
+ "shape": [16, 32],
273
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
274
+ },
275
+ "biasT": {
276
+ "dtype": "float32",
277
+ "shape": [32],
278
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
279
+ },
280
+ "ecoAT": {
281
+ "dtype": "float32",
282
+ "shape": [1, 2, 1, 1],
283
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
284
+ }
285
+ },
286
+ "outputs": {
287
+ "outputT": { "dtype": "float32", "shape": [1, 2, 8, 8], "tolerance": 0.000002, "relTolerance": 0.00002 }
288
+ }
289
+ },
290
+ {
291
+ "name": "head_size_64_seq_16",
292
+ "provenance": { "notes": "A model-shaped head size of 64 with four heads." },
293
+ "attrs": { "num_heads": 4 },
294
+ "inputs": {
295
+ "queryLayerT": {
296
+ "dtype": "float32",
297
+ "shape": [1, 16, 256],
298
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
299
+ },
300
+ "queryBiasT": {
301
+ "dtype": "float32",
302
+ "shape": [256],
303
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
304
+ },
305
+ "relPosT": {
306
+ "dtype": "float32",
307
+ "shape": [1, 4, 16, 16],
308
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
309
+ },
310
+ "weightT": {
311
+ "dtype": "float32",
312
+ "shape": [64, 8],
313
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
314
+ },
315
+ "biasT": {
316
+ "dtype": "float32",
317
+ "shape": [8],
318
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
319
+ },
320
+ "ecoAT": {
321
+ "dtype": "float32",
322
+ "shape": [1, 4, 1, 1],
323
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
324
+ }
325
+ },
326
+ "outputs": {
327
+ "outputT": { "dtype": "float32", "shape": [1, 4, 16, 16], "tolerance": 0.000002, "relTolerance": 0.00002 }
328
+ }
329
+ },
330
+ {
331
+ "name": "head_size_64_seq_16_f16",
332
+ "attrs": { "num_heads": 4 },
333
+ "inputs": {
334
+ "queryLayerT": {
335
+ "dtype": "float16",
336
+ "shape": [1, 16, 256],
337
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
338
+ },
339
+ "queryBiasT": {
340
+ "dtype": "float16",
341
+ "shape": [256],
342
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
343
+ },
344
+ "relPosT": {
345
+ "dtype": "float16",
346
+ "shape": [1, 4, 16, 16],
347
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
348
+ },
349
+ "weightT": {
350
+ "dtype": "float16",
351
+ "shape": [64, 8],
352
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
353
+ },
354
+ "biasT": {
355
+ "dtype": "float16",
356
+ "shape": [8],
357
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
358
+ },
359
+ "ecoAT": {
360
+ "dtype": "float16",
361
+ "shape": [1, 4, 1, 1],
362
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
363
+ }
364
+ },
365
+ "outputs": {
366
+ "outputT": { "dtype": "float16", "shape": [1, 4, 16, 16], "tolerance": 0.0005, "relTolerance": 0.002 }
367
+ }
368
+ },
369
+ {
370
+ "name": "seq_128_wide_row",
371
+ "provenance": { "notes": "A row wide enough that each invocation streams several elements of it." },
372
+ "attrs": { "num_heads": 2 },
373
+ "inputs": {
374
+ "queryLayerT": {
375
+ "dtype": "float32",
376
+ "shape": [1, 128, 64],
377
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
378
+ },
379
+ "queryBiasT": {
380
+ "dtype": "float32",
381
+ "shape": [64],
382
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
383
+ },
384
+ "relPosT": {
385
+ "dtype": "float32",
386
+ "shape": [1, 2, 128, 128],
387
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
388
+ },
389
+ "weightT": {
390
+ "dtype": "float32",
391
+ "shape": [32, 8],
392
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
393
+ },
394
+ "biasT": {
395
+ "dtype": "float32",
396
+ "shape": [8],
397
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
398
+ },
399
+ "ecoAT": {
400
+ "dtype": "float32",
401
+ "shape": [1, 2, 1, 1],
402
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
403
+ }
404
+ },
405
+ "outputs": {
406
+ "outputT": { "dtype": "float32", "shape": [1, 2, 128, 128], "tolerance": 0.000002, "relTolerance": 0.00002 }
407
+ }
408
+ },
409
+ {
410
+ "name": "negative_eco_a",
411
+ "provenance": { "notes": "Negative eco_a values, which flip the sign of the gate_r term before the offset." },
412
+ "attrs": { "num_heads": 3 },
413
+ "inputs": {
414
+ "queryLayerT": {
415
+ "dtype": "float32",
416
+ "shape": [1, 8, 48],
417
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
418
+ },
419
+ "queryBiasT": {
420
+ "dtype": "float32",
421
+ "shape": [48],
422
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
423
+ },
424
+ "relPosT": {
425
+ "dtype": "float32",
426
+ "shape": [1, 3, 8, 8],
427
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
428
+ },
429
+ "weightT": {
430
+ "dtype": "float32",
431
+ "shape": [16, 8],
432
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.25 }
433
+ },
434
+ "biasT": {
435
+ "dtype": "float32",
436
+ "shape": [8],
437
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
438
+ },
439
+ "ecoAT": {
440
+ "dtype": "float32",
441
+ "shape": [1, 3, 1, 1],
442
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": -1.5 }
443
+ }
444
+ },
445
+ "outputs": {
446
+ "outputT": { "dtype": "float32", "shape": [1, 3, 8, 8], "tolerance": 0.000002, "relTolerance": 0.00002 }
447
+ }
448
+ },
449
+ {
450
+ "name": "saturating_gates",
451
+ "provenance": {
452
+ "notes": "Large projections drive both logistics into their flat tails, where the gate stops depending on the query."
453
+ },
454
+ "attrs": { "num_heads": 2 },
455
+ "inputs": {
456
+ "queryLayerT": {
457
+ "dtype": "float32",
458
+ "shape": [1, 8, 32],
459
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 4.0 }
460
+ },
461
+ "queryBiasT": {
462
+ "dtype": "float32",
463
+ "shape": [32],
464
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
465
+ },
466
+ "relPosT": {
467
+ "dtype": "float32",
468
+ "shape": [1, 2, 8, 8],
469
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
470
+ },
471
+ "weightT": {
472
+ "dtype": "float32",
473
+ "shape": [16, 8],
474
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 4.0 }
475
+ },
476
+ "biasT": {
477
+ "dtype": "float32",
478
+ "shape": [8],
479
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
480
+ },
481
+ "ecoAT": {
482
+ "dtype": "float32",
483
+ "shape": [1, 2, 1, 1],
484
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
485
+ }
486
+ },
487
+ "outputs": {
488
+ "outputT": { "dtype": "float32", "shape": [1, 2, 8, 8], "tolerance": 0.000002, "relTolerance": 0.00002 }
489
+ }
490
+ },
491
+ {
492
+ "name": "head_size_128_seq_8",
493
+ "provenance": {
494
+ "notes": "A head size larger than the workgroup, so the staged query row is loaded in several strides."
495
+ },
496
+ "attrs": { "num_heads": 2 },
497
+ "inputs": {
498
+ "queryLayerT": {
499
+ "dtype": "float32",
500
+ "shape": [1, 8, 256],
501
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.12 }
502
+ },
503
+ "queryBiasT": {
504
+ "dtype": "float32",
505
+ "shape": [256],
506
+ "data": { "kind": "fillFloat32", "sinStep": 0.37, "cosStep": 0.11, "scale": 0.2 }
507
+ },
508
+ "relPosT": {
509
+ "dtype": "float32",
510
+ "shape": [1, 2, 8, 8],
511
+ "data": { "kind": "fillFloat32", "sinStep": 0.061, "cosStep": 0.257, "scale": 1.5 }
512
+ },
513
+ "weightT": {
514
+ "dtype": "float32",
515
+ "shape": [128, 8],
516
+ "data": { "kind": "fillFloat32", "sinStep": 0.19, "cosStep": 0.33, "scale": 0.12 }
517
+ },
518
+ "biasT": {
519
+ "dtype": "float32",
520
+ "shape": [8],
521
+ "data": { "kind": "fillFloat32", "sinStep": 0.41, "cosStep": 0.07, "scale": 0.3 }
522
+ },
523
+ "ecoAT": {
524
+ "dtype": "float32",
525
+ "shape": [1, 2, 1, 1],
526
+ "data": { "kind": "fillFloat32", "sinStep": 0.53, "cosStep": 0.17, "scale": 1.0 }
527
+ }
528
+ },
529
+ "outputs": {
530
+ "outputT": { "dtype": "float32", "shape": [1, 2, 8, 8], "tolerance": 0.000002, "relTolerance": 0.00002 }
531
+ }
532
+ },
533
+ {
534
+ "name": "head_projection_boundary-b2-n3-s128-h31-d8-f32_fixture",
535
+ "provenance": {
536
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
537
+ },
538
+ "attrs": { "num_heads": 3 },
539
+ "inputs": {
540
+ "queryLayerT": {
541
+ "dtype": "float32",
542
+ "shape": [2, 64, 93],
543
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
544
+ },
545
+ "queryBiasT": {
546
+ "dtype": "float32",
547
+ "shape": [93],
548
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
549
+ },
550
+ "relPosT": {
551
+ "dtype": "float32",
552
+ "shape": [1, 3, 64, 64],
553
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
554
+ },
555
+ "weightT": {
556
+ "dtype": "float32",
557
+ "shape": [31, 8],
558
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
559
+ },
560
+ "biasT": {
561
+ "dtype": "float32",
562
+ "shape": [8],
563
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
564
+ },
565
+ "ecoAT": {
566
+ "dtype": "float32",
567
+ "shape": [1, 3, 1, 1],
568
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
569
+ }
570
+ },
571
+ "outputs": {
572
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
573
+ }
574
+ },
575
+ {
576
+ "name": "head_projection_boundary-b2-n3-s128-h32-d8-f32_fixture",
577
+ "provenance": {
578
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
579
+ },
580
+ "attrs": { "num_heads": 3 },
581
+ "inputs": {
582
+ "queryLayerT": {
583
+ "dtype": "float32",
584
+ "shape": [2, 64, 96],
585
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
586
+ },
587
+ "queryBiasT": {
588
+ "dtype": "float32",
589
+ "shape": [96],
590
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
591
+ },
592
+ "relPosT": {
593
+ "dtype": "float32",
594
+ "shape": [1, 3, 64, 64],
595
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
596
+ },
597
+ "weightT": {
598
+ "dtype": "float32",
599
+ "shape": [32, 8],
600
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
601
+ },
602
+ "biasT": {
603
+ "dtype": "float32",
604
+ "shape": [8],
605
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
606
+ },
607
+ "ecoAT": {
608
+ "dtype": "float32",
609
+ "shape": [1, 3, 1, 1],
610
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
611
+ }
612
+ },
613
+ "outputs": {
614
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
615
+ }
616
+ },
617
+ {
618
+ "name": "head_projection_boundary-b2-n3-s127-h23-d6-f32_fixture",
619
+ "provenance": {
620
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
621
+ },
622
+ "attrs": { "num_heads": 3 },
623
+ "inputs": {
624
+ "queryLayerT": {
625
+ "dtype": "float32",
626
+ "shape": [2, 64, 69],
627
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
628
+ },
629
+ "queryBiasT": {
630
+ "dtype": "float32",
631
+ "shape": [69],
632
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
633
+ },
634
+ "relPosT": {
635
+ "dtype": "float32",
636
+ "shape": [1, 3, 64, 64],
637
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
638
+ },
639
+ "weightT": {
640
+ "dtype": "float32",
641
+ "shape": [23, 6],
642
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
643
+ },
644
+ "biasT": {
645
+ "dtype": "float32",
646
+ "shape": [6],
647
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
648
+ },
649
+ "ecoAT": {
650
+ "dtype": "float32",
651
+ "shape": [1, 3, 1, 1],
652
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
653
+ }
654
+ },
655
+ "outputs": {
656
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
657
+ }
658
+ },
659
+ {
660
+ "name": "head_projection_boundary-b2-n3-s127-h24-d6-f32_fixture",
661
+ "provenance": {
662
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
663
+ },
664
+ "attrs": { "num_heads": 3 },
665
+ "inputs": {
666
+ "queryLayerT": {
667
+ "dtype": "float32",
668
+ "shape": [2, 64, 72],
669
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
670
+ },
671
+ "queryBiasT": {
672
+ "dtype": "float32",
673
+ "shape": [72],
674
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
675
+ },
676
+ "relPosT": {
677
+ "dtype": "float32",
678
+ "shape": [1, 3, 64, 64],
679
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
680
+ },
681
+ "weightT": {
682
+ "dtype": "float32",
683
+ "shape": [24, 6],
684
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
685
+ },
686
+ "biasT": {
687
+ "dtype": "float32",
688
+ "shape": [6],
689
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
690
+ },
691
+ "ecoAT": {
692
+ "dtype": "float32",
693
+ "shape": [1, 3, 1, 1],
694
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
695
+ }
696
+ },
697
+ "outputs": {
698
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
699
+ }
700
+ },
701
+ {
702
+ "name": "head_projection_boundary-b2-n3-s128-h64-d10-f32_fixture",
703
+ "provenance": {
704
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
705
+ },
706
+ "attrs": { "num_heads": 3 },
707
+ "inputs": {
708
+ "queryLayerT": {
709
+ "dtype": "float32",
710
+ "shape": [2, 64, 192],
711
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
712
+ },
713
+ "queryBiasT": {
714
+ "dtype": "float32",
715
+ "shape": [192],
716
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
717
+ },
718
+ "relPosT": {
719
+ "dtype": "float32",
720
+ "shape": [1, 3, 64, 64],
721
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
722
+ },
723
+ "weightT": {
724
+ "dtype": "float32",
725
+ "shape": [64, 10],
726
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
727
+ },
728
+ "biasT": {
729
+ "dtype": "float32",
730
+ "shape": [10],
731
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
732
+ },
733
+ "ecoAT": {
734
+ "dtype": "float32",
735
+ "shape": [1, 3, 1, 1],
736
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
737
+ }
738
+ },
739
+ "outputs": {
740
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
741
+ }
742
+ },
743
+ {
744
+ "name": "head_projection_general-b2-n3-s1-h1-d2-f32_fixture",
745
+ "provenance": {
746
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
747
+ },
748
+ "attrs": { "num_heads": 3 },
749
+ "inputs": {
750
+ "queryLayerT": {
751
+ "dtype": "float32",
752
+ "shape": [2, 1, 3],
753
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
754
+ },
755
+ "queryBiasT": {
756
+ "dtype": "float32",
757
+ "shape": [3],
758
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
759
+ },
760
+ "relPosT": {
761
+ "dtype": "float32",
762
+ "shape": [1, 3, 1, 1],
763
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
764
+ },
765
+ "weightT": {
766
+ "dtype": "float32",
767
+ "shape": [1, 2],
768
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
769
+ },
770
+ "biasT": {
771
+ "dtype": "float32",
772
+ "shape": [2],
773
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
774
+ },
775
+ "ecoAT": {
776
+ "dtype": "float32",
777
+ "shape": [1, 3, 1, 1],
778
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
779
+ }
780
+ },
781
+ "outputs": {
782
+ "outputT": { "shape": [2, 3, 1, 1], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
783
+ }
784
+ },
785
+ {
786
+ "name": "head_projection_general-b2-n3-s5-h3-d32-f32_fixture",
787
+ "provenance": {
788
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
789
+ },
790
+ "attrs": { "num_heads": 3 },
791
+ "inputs": {
792
+ "queryLayerT": {
793
+ "dtype": "float32",
794
+ "shape": [2, 5, 9],
795
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
796
+ },
797
+ "queryBiasT": {
798
+ "dtype": "float32",
799
+ "shape": [9],
800
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
801
+ },
802
+ "relPosT": {
803
+ "dtype": "float32",
804
+ "shape": [1, 3, 5, 5],
805
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
806
+ },
807
+ "weightT": {
808
+ "dtype": "float32",
809
+ "shape": [3, 32],
810
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
811
+ },
812
+ "biasT": {
813
+ "dtype": "float32",
814
+ "shape": [32],
815
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
816
+ },
817
+ "ecoAT": {
818
+ "dtype": "float32",
819
+ "shape": [1, 3, 1, 1],
820
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
821
+ }
822
+ },
823
+ "outputs": {
824
+ "outputT": { "shape": [2, 3, 5, 5], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
825
+ }
826
+ },
827
+ {
828
+ "name": "head_projection_general-b2-n3-s17-h8-d16-f32_fixture",
829
+ "provenance": {
830
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
831
+ },
832
+ "attrs": { "num_heads": 3 },
833
+ "inputs": {
834
+ "queryLayerT": {
835
+ "dtype": "float32",
836
+ "shape": [2, 17, 24],
837
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
838
+ },
839
+ "queryBiasT": {
840
+ "dtype": "float32",
841
+ "shape": [24],
842
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
843
+ },
844
+ "relPosT": {
845
+ "dtype": "float32",
846
+ "shape": [1, 3, 17, 17],
847
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
848
+ },
849
+ "weightT": {
850
+ "dtype": "float32",
851
+ "shape": [8, 16],
852
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
853
+ },
854
+ "biasT": {
855
+ "dtype": "float32",
856
+ "shape": [16],
857
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
858
+ },
859
+ "ecoAT": {
860
+ "dtype": "float32",
861
+ "shape": [1, 3, 1, 1],
862
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
863
+ }
864
+ },
865
+ "outputs": {
866
+ "outputT": { "shape": [2, 3, 17, 17], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
867
+ }
868
+ },
869
+ {
870
+ "name": "head_projection_general-b2-n3-s31-h31-d6-f32_fixture",
871
+ "provenance": {
872
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
873
+ },
874
+ "attrs": { "num_heads": 3 },
875
+ "inputs": {
876
+ "queryLayerT": {
877
+ "dtype": "float32",
878
+ "shape": [2, 31, 93],
879
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
880
+ },
881
+ "queryBiasT": {
882
+ "dtype": "float32",
883
+ "shape": [93],
884
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
885
+ },
886
+ "relPosT": {
887
+ "dtype": "float32",
888
+ "shape": [1, 3, 31, 31],
889
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
890
+ },
891
+ "weightT": {
892
+ "dtype": "float32",
893
+ "shape": [31, 6],
894
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
895
+ },
896
+ "biasT": {
897
+ "dtype": "float32",
898
+ "shape": [6],
899
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
900
+ },
901
+ "ecoAT": {
902
+ "dtype": "float32",
903
+ "shape": [1, 3, 1, 1],
904
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
905
+ }
906
+ },
907
+ "outputs": {
908
+ "outputT": { "shape": [2, 3, 31, 31], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
909
+ }
910
+ },
911
+ {
912
+ "name": "head_projection_general-b2-n3-s64-h32-d32-f32_fixture",
913
+ "provenance": {
914
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
915
+ },
916
+ "attrs": { "num_heads": 3 },
917
+ "inputs": {
918
+ "queryLayerT": {
919
+ "dtype": "float32",
920
+ "shape": [2, 64, 96],
921
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
922
+ },
923
+ "queryBiasT": {
924
+ "dtype": "float32",
925
+ "shape": [96],
926
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
927
+ },
928
+ "relPosT": {
929
+ "dtype": "float32",
930
+ "shape": [1, 3, 64, 64],
931
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
932
+ },
933
+ "weightT": {
934
+ "dtype": "float32",
935
+ "shape": [32, 32],
936
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
937
+ },
938
+ "biasT": {
939
+ "dtype": "float32",
940
+ "shape": [32],
941
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
942
+ },
943
+ "ecoAT": {
944
+ "dtype": "float32",
945
+ "shape": [1, 3, 1, 1],
946
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
947
+ }
948
+ },
949
+ "outputs": {
950
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
951
+ }
952
+ },
953
+ {
954
+ "name": "head_projection_general-b2-n3-s127-h64-d2-f32_fixture",
955
+ "provenance": {
956
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
957
+ },
958
+ "attrs": { "num_heads": 3 },
959
+ "inputs": {
960
+ "queryLayerT": {
961
+ "dtype": "float32",
962
+ "shape": [2, 64, 192],
963
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
964
+ },
965
+ "queryBiasT": {
966
+ "dtype": "float32",
967
+ "shape": [192],
968
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
969
+ },
970
+ "relPosT": {
971
+ "dtype": "float32",
972
+ "shape": [1, 3, 64, 64],
973
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
974
+ },
975
+ "weightT": {
976
+ "dtype": "float32",
977
+ "shape": [64, 2],
978
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
979
+ },
980
+ "biasT": {
981
+ "dtype": "float32",
982
+ "shape": [2],
983
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
984
+ },
985
+ "ecoAT": {
986
+ "dtype": "float32",
987
+ "shape": [1, 3, 1, 1],
988
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
989
+ }
990
+ },
991
+ "outputs": {
992
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
993
+ }
994
+ },
995
+ {
996
+ "name": "head_projection_general-b2-n3-s128-h63-d8-f32_fixture",
997
+ "provenance": {
998
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
999
+ },
1000
+ "attrs": { "num_heads": 3 },
1001
+ "inputs": {
1002
+ "queryLayerT": {
1003
+ "dtype": "float32",
1004
+ "shape": [2, 64, 189],
1005
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1006
+ },
1007
+ "queryBiasT": {
1008
+ "dtype": "float32",
1009
+ "shape": [189],
1010
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1011
+ },
1012
+ "relPosT": {
1013
+ "dtype": "float32",
1014
+ "shape": [1, 3, 64, 64],
1015
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1016
+ },
1017
+ "weightT": {
1018
+ "dtype": "float32",
1019
+ "shape": [63, 8],
1020
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1021
+ },
1022
+ "biasT": {
1023
+ "dtype": "float32",
1024
+ "shape": [8],
1025
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1026
+ },
1027
+ "ecoAT": {
1028
+ "dtype": "float32",
1029
+ "shape": [1, 3, 1, 1],
1030
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1031
+ }
1032
+ },
1033
+ "outputs": {
1034
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1035
+ }
1036
+ },
1037
+ {
1038
+ "name": "head_projection_general-b2-n3-s257-h127-d6-f32_fixture",
1039
+ "provenance": {
1040
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1041
+ },
1042
+ "attrs": { "num_heads": 3 },
1043
+ "inputs": {
1044
+ "queryLayerT": {
1045
+ "dtype": "float32",
1046
+ "shape": [2, 64, 381],
1047
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1048
+ },
1049
+ "queryBiasT": {
1050
+ "dtype": "float32",
1051
+ "shape": [381],
1052
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1053
+ },
1054
+ "relPosT": {
1055
+ "dtype": "float32",
1056
+ "shape": [1, 3, 64, 64],
1057
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1058
+ },
1059
+ "weightT": {
1060
+ "dtype": "float32",
1061
+ "shape": [127, 6],
1062
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1063
+ },
1064
+ "biasT": {
1065
+ "dtype": "float32",
1066
+ "shape": [6],
1067
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1068
+ },
1069
+ "ecoAT": {
1070
+ "dtype": "float32",
1071
+ "shape": [1, 3, 1, 1],
1072
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1073
+ }
1074
+ },
1075
+ "outputs": {
1076
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1077
+ }
1078
+ },
1079
+ {
1080
+ "name": "head_projection_general-b2-n3-s256-h128-d32-f32_fixture",
1081
+ "provenance": {
1082
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1083
+ },
1084
+ "attrs": { "num_heads": 3 },
1085
+ "inputs": {
1086
+ "queryLayerT": {
1087
+ "dtype": "float32",
1088
+ "shape": [2, 64, 384],
1089
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1090
+ },
1091
+ "queryBiasT": {
1092
+ "dtype": "float32",
1093
+ "shape": [384],
1094
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1095
+ },
1096
+ "relPosT": {
1097
+ "dtype": "float32",
1098
+ "shape": [1, 3, 64, 64],
1099
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1100
+ },
1101
+ "weightT": {
1102
+ "dtype": "float32",
1103
+ "shape": [128, 32],
1104
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1105
+ },
1106
+ "biasT": {
1107
+ "dtype": "float32",
1108
+ "shape": [32],
1109
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1110
+ },
1111
+ "ecoAT": {
1112
+ "dtype": "float32",
1113
+ "shape": [1, 3, 1, 1],
1114
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1115
+ }
1116
+ },
1117
+ "outputs": {
1118
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1119
+ }
1120
+ },
1121
+ {
1122
+ "name": "head_projection_general-b1-n4-s512-h256-d16-f32_fixture",
1123
+ "provenance": {
1124
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1125
+ },
1126
+ "attrs": { "num_heads": 4 },
1127
+ "inputs": {
1128
+ "queryLayerT": {
1129
+ "dtype": "float32",
1130
+ "shape": [1, 64, 1024],
1131
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1132
+ },
1133
+ "queryBiasT": {
1134
+ "dtype": "float32",
1135
+ "shape": [1024],
1136
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1137
+ },
1138
+ "relPosT": {
1139
+ "dtype": "float32",
1140
+ "shape": [1, 4, 64, 64],
1141
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1142
+ },
1143
+ "weightT": {
1144
+ "dtype": "float32",
1145
+ "shape": [256, 16],
1146
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1147
+ },
1148
+ "biasT": {
1149
+ "dtype": "float32",
1150
+ "shape": [16],
1151
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1152
+ },
1153
+ "ecoAT": {
1154
+ "dtype": "float32",
1155
+ "shape": [1, 4, 1, 1],
1156
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1157
+ }
1158
+ },
1159
+ "outputs": {
1160
+ "outputT": { "shape": [1, 4, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1161
+ }
1162
+ },
1163
+ {
1164
+ "name": "head_projection_general-b1-n2-s63-h512-d8-f32_fixture",
1165
+ "provenance": {
1166
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1167
+ },
1168
+ "attrs": { "num_heads": 2 },
1169
+ "inputs": {
1170
+ "queryLayerT": {
1171
+ "dtype": "float32",
1172
+ "shape": [1, 63, 1024],
1173
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1174
+ },
1175
+ "queryBiasT": {
1176
+ "dtype": "float32",
1177
+ "shape": [1024],
1178
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1179
+ },
1180
+ "relPosT": {
1181
+ "dtype": "float32",
1182
+ "shape": [1, 2, 63, 63],
1183
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1184
+ },
1185
+ "weightT": {
1186
+ "dtype": "float32",
1187
+ "shape": [512, 8],
1188
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1189
+ },
1190
+ "biasT": {
1191
+ "dtype": "float32",
1192
+ "shape": [8],
1193
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1194
+ },
1195
+ "ecoAT": {
1196
+ "dtype": "float32",
1197
+ "shape": [1, 2, 1, 1],
1198
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1199
+ }
1200
+ },
1201
+ "outputs": {
1202
+ "outputT": { "shape": [1, 2, 63, 63], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1203
+ }
1204
+ },
1205
+ {
1206
+ "name": "head_projection_general-b1-n2-s32-h1024-d32-f32_fixture",
1207
+ "provenance": {
1208
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1209
+ },
1210
+ "attrs": { "num_heads": 2 },
1211
+ "inputs": {
1212
+ "queryLayerT": {
1213
+ "dtype": "float32",
1214
+ "shape": [1, 32, 2048],
1215
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1216
+ },
1217
+ "queryBiasT": {
1218
+ "dtype": "float32",
1219
+ "shape": [2048],
1220
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1221
+ },
1222
+ "relPosT": {
1223
+ "dtype": "float32",
1224
+ "shape": [1, 2, 32, 32],
1225
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1226
+ },
1227
+ "weightT": {
1228
+ "dtype": "float32",
1229
+ "shape": [1024, 32],
1230
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1231
+ },
1232
+ "biasT": {
1233
+ "dtype": "float32",
1234
+ "shape": [32],
1235
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1236
+ },
1237
+ "ecoAT": {
1238
+ "dtype": "float32",
1239
+ "shape": [1, 2, 1, 1],
1240
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1241
+ }
1242
+ },
1243
+ "outputs": {
1244
+ "outputT": { "shape": [1, 2, 32, 32], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1245
+ }
1246
+ },
1247
+ {
1248
+ "name": "head_projection_general-b1-n2-s2053-h64-d8-f32_fixture",
1249
+ "provenance": {
1250
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1251
+ },
1252
+ "attrs": { "num_heads": 2 },
1253
+ "tunables": { "WORKGROUP_SIZE": 32 },
1254
+ "inputs": {
1255
+ "queryLayerT": {
1256
+ "dtype": "float32",
1257
+ "shape": [1, 258, 128],
1258
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1259
+ },
1260
+ "queryBiasT": {
1261
+ "dtype": "float32",
1262
+ "shape": [128],
1263
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1264
+ },
1265
+ "relPosT": {
1266
+ "dtype": "float32",
1267
+ "shape": [1, 2, 258, 258],
1268
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1269
+ },
1270
+ "weightT": {
1271
+ "dtype": "float32",
1272
+ "shape": [64, 8],
1273
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1274
+ },
1275
+ "biasT": {
1276
+ "dtype": "float32",
1277
+ "shape": [8],
1278
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1279
+ },
1280
+ "ecoAT": {
1281
+ "dtype": "float32",
1282
+ "shape": [1, 2, 1, 1],
1283
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1284
+ }
1285
+ },
1286
+ "outputs": {
1287
+ "outputT": { "shape": [1, 2, 258, 258], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1288
+ }
1289
+ },
1290
+ {
1291
+ "name": "head_projection_stream-b2-n3-s258-h31-d6-f32_fixture",
1292
+ "provenance": {
1293
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1294
+ },
1295
+ "attrs": { "num_heads": 3 },
1296
+ "tunables": { "WORKGROUP_SIZE": 32 },
1297
+ "inputs": {
1298
+ "queryLayerT": {
1299
+ "dtype": "float32",
1300
+ "shape": [2, 258, 93],
1301
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1302
+ },
1303
+ "queryBiasT": {
1304
+ "dtype": "float32",
1305
+ "shape": [93],
1306
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1307
+ },
1308
+ "relPosT": {
1309
+ "dtype": "float32",
1310
+ "shape": [1, 3, 258, 258],
1311
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1312
+ },
1313
+ "weightT": {
1314
+ "dtype": "float32",
1315
+ "shape": [31, 6],
1316
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1317
+ },
1318
+ "biasT": {
1319
+ "dtype": "float32",
1320
+ "shape": [6],
1321
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1322
+ },
1323
+ "ecoAT": {
1324
+ "dtype": "float32",
1325
+ "shape": [1, 3, 1, 1],
1326
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1327
+ }
1328
+ },
1329
+ "outputs": {
1330
+ "outputT": { "shape": [2, 3, 258, 258], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1331
+ }
1332
+ },
1333
+ {
1334
+ "name": "head_projection_wg64-b2-n3-s257-h127-d8-f32_fixture",
1335
+ "provenance": {
1336
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1337
+ },
1338
+ "attrs": { "num_heads": 3 },
1339
+ "tunables": { "WORKGROUP_SIZE": 64, "LANE_WORDS": 1 },
1340
+ "inputs": {
1341
+ "queryLayerT": {
1342
+ "dtype": "float32",
1343
+ "shape": [2, 64, 381],
1344
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1345
+ },
1346
+ "queryBiasT": {
1347
+ "dtype": "float32",
1348
+ "shape": [381],
1349
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1350
+ },
1351
+ "relPosT": {
1352
+ "dtype": "float32",
1353
+ "shape": [1, 3, 64, 64],
1354
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1355
+ },
1356
+ "weightT": {
1357
+ "dtype": "float32",
1358
+ "shape": [127, 8],
1359
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1360
+ },
1361
+ "biasT": {
1362
+ "dtype": "float32",
1363
+ "shape": [8],
1364
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1365
+ },
1366
+ "ecoAT": {
1367
+ "dtype": "float32",
1368
+ "shape": [1, 3, 1, 1],
1369
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1370
+ }
1371
+ },
1372
+ "outputs": {
1373
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1374
+ }
1375
+ },
1376
+ {
1377
+ "name": "head_projection_wg256-b2-n3-s256-h128-d32-f32_fixture",
1378
+ "provenance": {
1379
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1380
+ },
1381
+ "attrs": { "num_heads": 3 },
1382
+ "tunables": { "WORKGROUP_SIZE": 256, "LANE_WORDS": 1 },
1383
+ "inputs": {
1384
+ "queryLayerT": {
1385
+ "dtype": "float32",
1386
+ "shape": [2, 64, 384],
1387
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1388
+ },
1389
+ "queryBiasT": {
1390
+ "dtype": "float32",
1391
+ "shape": [384],
1392
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1393
+ },
1394
+ "relPosT": {
1395
+ "dtype": "float32",
1396
+ "shape": [1, 3, 64, 64],
1397
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1398
+ },
1399
+ "weightT": {
1400
+ "dtype": "float32",
1401
+ "shape": [128, 32],
1402
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1403
+ },
1404
+ "biasT": {
1405
+ "dtype": "float32",
1406
+ "shape": [32],
1407
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1408
+ },
1409
+ "ecoAT": {
1410
+ "dtype": "float32",
1411
+ "shape": [1, 3, 1, 1],
1412
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1413
+ }
1414
+ },
1415
+ "outputs": {
1416
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1417
+ }
1418
+ },
1419
+ {
1420
+ "name": "head_projection_wg512-b2-n3-s128-h127-d32-f32_fixture",
1421
+ "provenance": {
1422
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1423
+ },
1424
+ "attrs": { "num_heads": 3 },
1425
+ "tunables": { "WORKGROUP_SIZE": 256, "LANE_WORDS": 8 },
1426
+ "inputs": {
1427
+ "queryLayerT": {
1428
+ "dtype": "float32",
1429
+ "shape": [2, 64, 381],
1430
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1431
+ },
1432
+ "queryBiasT": {
1433
+ "dtype": "float32",
1434
+ "shape": [381],
1435
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1436
+ },
1437
+ "relPosT": {
1438
+ "dtype": "float32",
1439
+ "shape": [1, 3, 64, 64],
1440
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1441
+ },
1442
+ "weightT": {
1443
+ "dtype": "float32",
1444
+ "shape": [127, 32],
1445
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1446
+ },
1447
+ "biasT": {
1448
+ "dtype": "float32",
1449
+ "shape": [32],
1450
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1451
+ },
1452
+ "ecoAT": {
1453
+ "dtype": "float32",
1454
+ "shape": [1, 3, 1, 1],
1455
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1456
+ }
1457
+ },
1458
+ "outputs": {
1459
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
1460
+ }
1461
+ },
1462
+ {
1463
+ "name": "head_projection_boundary-b2-n3-s128-h31-d8-f16_fixture",
1464
+ "provenance": {
1465
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1466
+ },
1467
+ "attrs": { "num_heads": 3 },
1468
+ "inputs": {
1469
+ "queryLayerT": {
1470
+ "dtype": "float16",
1471
+ "shape": [2, 64, 93],
1472
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1473
+ },
1474
+ "queryBiasT": {
1475
+ "dtype": "float16",
1476
+ "shape": [93],
1477
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1478
+ },
1479
+ "relPosT": {
1480
+ "dtype": "float16",
1481
+ "shape": [1, 3, 64, 64],
1482
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1483
+ },
1484
+ "weightT": {
1485
+ "dtype": "float16",
1486
+ "shape": [31, 8],
1487
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1488
+ },
1489
+ "biasT": {
1490
+ "dtype": "float16",
1491
+ "shape": [8],
1492
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1493
+ },
1494
+ "ecoAT": {
1495
+ "dtype": "float16",
1496
+ "shape": [1, 3, 1, 1],
1497
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1498
+ }
1499
+ },
1500
+ "outputs": {
1501
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1502
+ }
1503
+ },
1504
+ {
1505
+ "name": "head_projection_boundary-b2-n3-s128-h32-d8-f16_fixture",
1506
+ "provenance": {
1507
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1508
+ },
1509
+ "attrs": { "num_heads": 3 },
1510
+ "inputs": {
1511
+ "queryLayerT": {
1512
+ "dtype": "float16",
1513
+ "shape": [2, 64, 96],
1514
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1515
+ },
1516
+ "queryBiasT": {
1517
+ "dtype": "float16",
1518
+ "shape": [96],
1519
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1520
+ },
1521
+ "relPosT": {
1522
+ "dtype": "float16",
1523
+ "shape": [1, 3, 64, 64],
1524
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1525
+ },
1526
+ "weightT": {
1527
+ "dtype": "float16",
1528
+ "shape": [32, 8],
1529
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1530
+ },
1531
+ "biasT": {
1532
+ "dtype": "float16",
1533
+ "shape": [8],
1534
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1535
+ },
1536
+ "ecoAT": {
1537
+ "dtype": "float16",
1538
+ "shape": [1, 3, 1, 1],
1539
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1540
+ }
1541
+ },
1542
+ "outputs": {
1543
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1544
+ }
1545
+ },
1546
+ {
1547
+ "name": "head_projection_boundary-b2-n3-s127-h23-d6-f16_fixture",
1548
+ "provenance": {
1549
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1550
+ },
1551
+ "attrs": { "num_heads": 3 },
1552
+ "inputs": {
1553
+ "queryLayerT": {
1554
+ "dtype": "float16",
1555
+ "shape": [2, 64, 69],
1556
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1557
+ },
1558
+ "queryBiasT": {
1559
+ "dtype": "float16",
1560
+ "shape": [69],
1561
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1562
+ },
1563
+ "relPosT": {
1564
+ "dtype": "float16",
1565
+ "shape": [1, 3, 64, 64],
1566
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1567
+ },
1568
+ "weightT": {
1569
+ "dtype": "float16",
1570
+ "shape": [23, 6],
1571
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1572
+ },
1573
+ "biasT": {
1574
+ "dtype": "float16",
1575
+ "shape": [6],
1576
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1577
+ },
1578
+ "ecoAT": {
1579
+ "dtype": "float16",
1580
+ "shape": [1, 3, 1, 1],
1581
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1582
+ }
1583
+ },
1584
+ "outputs": {
1585
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1586
+ }
1587
+ },
1588
+ {
1589
+ "name": "head_projection_boundary-b2-n3-s127-h24-d6-f16_fixture",
1590
+ "provenance": {
1591
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1592
+ },
1593
+ "attrs": { "num_heads": 3 },
1594
+ "inputs": {
1595
+ "queryLayerT": {
1596
+ "dtype": "float16",
1597
+ "shape": [2, 64, 72],
1598
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1599
+ },
1600
+ "queryBiasT": {
1601
+ "dtype": "float16",
1602
+ "shape": [72],
1603
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1604
+ },
1605
+ "relPosT": {
1606
+ "dtype": "float16",
1607
+ "shape": [1, 3, 64, 64],
1608
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1609
+ },
1610
+ "weightT": {
1611
+ "dtype": "float16",
1612
+ "shape": [24, 6],
1613
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1614
+ },
1615
+ "biasT": {
1616
+ "dtype": "float16",
1617
+ "shape": [6],
1618
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1619
+ },
1620
+ "ecoAT": {
1621
+ "dtype": "float16",
1622
+ "shape": [1, 3, 1, 1],
1623
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1624
+ }
1625
+ },
1626
+ "outputs": {
1627
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1628
+ }
1629
+ },
1630
+ {
1631
+ "name": "head_projection_boundary-b2-n3-s128-h64-d10-f16_fixture",
1632
+ "provenance": {
1633
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1634
+ },
1635
+ "attrs": { "num_heads": 3 },
1636
+ "inputs": {
1637
+ "queryLayerT": {
1638
+ "dtype": "float16",
1639
+ "shape": [2, 64, 192],
1640
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1641
+ },
1642
+ "queryBiasT": {
1643
+ "dtype": "float16",
1644
+ "shape": [192],
1645
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1646
+ },
1647
+ "relPosT": {
1648
+ "dtype": "float16",
1649
+ "shape": [1, 3, 64, 64],
1650
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1651
+ },
1652
+ "weightT": {
1653
+ "dtype": "float16",
1654
+ "shape": [64, 10],
1655
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1656
+ },
1657
+ "biasT": {
1658
+ "dtype": "float16",
1659
+ "shape": [10],
1660
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1661
+ },
1662
+ "ecoAT": {
1663
+ "dtype": "float16",
1664
+ "shape": [1, 3, 1, 1],
1665
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1666
+ }
1667
+ },
1668
+ "outputs": {
1669
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1670
+ }
1671
+ },
1672
+ {
1673
+ "name": "head_projection_general-b2-n3-s1-h1-d2-f16_fixture",
1674
+ "provenance": {
1675
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1676
+ },
1677
+ "attrs": { "num_heads": 3 },
1678
+ "inputs": {
1679
+ "queryLayerT": {
1680
+ "dtype": "float16",
1681
+ "shape": [2, 1, 3],
1682
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1683
+ },
1684
+ "queryBiasT": {
1685
+ "dtype": "float16",
1686
+ "shape": [3],
1687
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1688
+ },
1689
+ "relPosT": {
1690
+ "dtype": "float16",
1691
+ "shape": [1, 3, 1, 1],
1692
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1693
+ },
1694
+ "weightT": {
1695
+ "dtype": "float16",
1696
+ "shape": [1, 2],
1697
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1698
+ },
1699
+ "biasT": {
1700
+ "dtype": "float16",
1701
+ "shape": [2],
1702
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1703
+ },
1704
+ "ecoAT": {
1705
+ "dtype": "float16",
1706
+ "shape": [1, 3, 1, 1],
1707
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1708
+ }
1709
+ },
1710
+ "outputs": {
1711
+ "outputT": { "shape": [2, 3, 1, 1], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1712
+ }
1713
+ },
1714
+ {
1715
+ "name": "head_projection_general-b2-n3-s5-h3-d32-f16_fixture",
1716
+ "provenance": {
1717
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1718
+ },
1719
+ "attrs": { "num_heads": 3 },
1720
+ "inputs": {
1721
+ "queryLayerT": {
1722
+ "dtype": "float16",
1723
+ "shape": [2, 5, 9],
1724
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1725
+ },
1726
+ "queryBiasT": {
1727
+ "dtype": "float16",
1728
+ "shape": [9],
1729
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1730
+ },
1731
+ "relPosT": {
1732
+ "dtype": "float16",
1733
+ "shape": [1, 3, 5, 5],
1734
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1735
+ },
1736
+ "weightT": {
1737
+ "dtype": "float16",
1738
+ "shape": [3, 32],
1739
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1740
+ },
1741
+ "biasT": {
1742
+ "dtype": "float16",
1743
+ "shape": [32],
1744
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1745
+ },
1746
+ "ecoAT": {
1747
+ "dtype": "float16",
1748
+ "shape": [1, 3, 1, 1],
1749
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1750
+ }
1751
+ },
1752
+ "outputs": {
1753
+ "outputT": { "shape": [2, 3, 5, 5], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1754
+ }
1755
+ },
1756
+ {
1757
+ "name": "head_projection_general-b2-n3-s17-h8-d16-f16_fixture",
1758
+ "provenance": {
1759
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1760
+ },
1761
+ "attrs": { "num_heads": 3 },
1762
+ "inputs": {
1763
+ "queryLayerT": {
1764
+ "dtype": "float16",
1765
+ "shape": [2, 17, 24],
1766
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1767
+ },
1768
+ "queryBiasT": {
1769
+ "dtype": "float16",
1770
+ "shape": [24],
1771
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1772
+ },
1773
+ "relPosT": {
1774
+ "dtype": "float16",
1775
+ "shape": [1, 3, 17, 17],
1776
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1777
+ },
1778
+ "weightT": {
1779
+ "dtype": "float16",
1780
+ "shape": [8, 16],
1781
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1782
+ },
1783
+ "biasT": {
1784
+ "dtype": "float16",
1785
+ "shape": [16],
1786
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1787
+ },
1788
+ "ecoAT": {
1789
+ "dtype": "float16",
1790
+ "shape": [1, 3, 1, 1],
1791
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1792
+ }
1793
+ },
1794
+ "outputs": {
1795
+ "outputT": { "shape": [2, 3, 17, 17], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1796
+ }
1797
+ },
1798
+ {
1799
+ "name": "head_projection_general-b2-n3-s31-h31-d6-f16_fixture",
1800
+ "provenance": {
1801
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1802
+ },
1803
+ "attrs": { "num_heads": 3 },
1804
+ "inputs": {
1805
+ "queryLayerT": {
1806
+ "dtype": "float16",
1807
+ "shape": [2, 31, 93],
1808
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1809
+ },
1810
+ "queryBiasT": {
1811
+ "dtype": "float16",
1812
+ "shape": [93],
1813
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1814
+ },
1815
+ "relPosT": {
1816
+ "dtype": "float16",
1817
+ "shape": [1, 3, 31, 31],
1818
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1819
+ },
1820
+ "weightT": {
1821
+ "dtype": "float16",
1822
+ "shape": [31, 6],
1823
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1824
+ },
1825
+ "biasT": {
1826
+ "dtype": "float16",
1827
+ "shape": [6],
1828
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1829
+ },
1830
+ "ecoAT": {
1831
+ "dtype": "float16",
1832
+ "shape": [1, 3, 1, 1],
1833
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1834
+ }
1835
+ },
1836
+ "outputs": {
1837
+ "outputT": { "shape": [2, 3, 31, 31], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1838
+ }
1839
+ },
1840
+ {
1841
+ "name": "head_projection_general-b2-n3-s64-h32-d32-f16_fixture",
1842
+ "provenance": {
1843
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
1844
+ },
1845
+ "attrs": { "num_heads": 3 },
1846
+ "inputs": {
1847
+ "queryLayerT": {
1848
+ "dtype": "float16",
1849
+ "shape": [2, 64, 96],
1850
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1851
+ },
1852
+ "queryBiasT": {
1853
+ "dtype": "float16",
1854
+ "shape": [96],
1855
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1856
+ },
1857
+ "relPosT": {
1858
+ "dtype": "float16",
1859
+ "shape": [1, 3, 64, 64],
1860
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1861
+ },
1862
+ "weightT": {
1863
+ "dtype": "float16",
1864
+ "shape": [32, 32],
1865
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1866
+ },
1867
+ "biasT": {
1868
+ "dtype": "float16",
1869
+ "shape": [32],
1870
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1871
+ },
1872
+ "ecoAT": {
1873
+ "dtype": "float16",
1874
+ "shape": [1, 3, 1, 1],
1875
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1876
+ }
1877
+ },
1878
+ "outputs": {
1879
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1880
+ }
1881
+ },
1882
+ {
1883
+ "name": "head_projection_general-b2-n3-s127-h64-d2-f16_fixture",
1884
+ "provenance": {
1885
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1886
+ },
1887
+ "attrs": { "num_heads": 3 },
1888
+ "inputs": {
1889
+ "queryLayerT": {
1890
+ "dtype": "float16",
1891
+ "shape": [2, 64, 192],
1892
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1893
+ },
1894
+ "queryBiasT": {
1895
+ "dtype": "float16",
1896
+ "shape": [192],
1897
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1898
+ },
1899
+ "relPosT": {
1900
+ "dtype": "float16",
1901
+ "shape": [1, 3, 64, 64],
1902
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1903
+ },
1904
+ "weightT": {
1905
+ "dtype": "float16",
1906
+ "shape": [64, 2],
1907
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1908
+ },
1909
+ "biasT": {
1910
+ "dtype": "float16",
1911
+ "shape": [2],
1912
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1913
+ },
1914
+ "ecoAT": {
1915
+ "dtype": "float16",
1916
+ "shape": [1, 3, 1, 1],
1917
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1918
+ }
1919
+ },
1920
+ "outputs": {
1921
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1922
+ }
1923
+ },
1924
+ {
1925
+ "name": "head_projection_general-b2-n3-s128-h63-d8-f16_fixture",
1926
+ "provenance": {
1927
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1928
+ },
1929
+ "attrs": { "num_heads": 3 },
1930
+ "inputs": {
1931
+ "queryLayerT": {
1932
+ "dtype": "float16",
1933
+ "shape": [2, 64, 189],
1934
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1935
+ },
1936
+ "queryBiasT": {
1937
+ "dtype": "float16",
1938
+ "shape": [189],
1939
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1940
+ },
1941
+ "relPosT": {
1942
+ "dtype": "float16",
1943
+ "shape": [1, 3, 64, 64],
1944
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1945
+ },
1946
+ "weightT": {
1947
+ "dtype": "float16",
1948
+ "shape": [63, 8],
1949
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1950
+ },
1951
+ "biasT": {
1952
+ "dtype": "float16",
1953
+ "shape": [8],
1954
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1955
+ },
1956
+ "ecoAT": {
1957
+ "dtype": "float16",
1958
+ "shape": [1, 3, 1, 1],
1959
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
1960
+ }
1961
+ },
1962
+ "outputs": {
1963
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
1964
+ }
1965
+ },
1966
+ {
1967
+ "name": "head_projection_general-b2-n3-s257-h127-d6-f16_fixture",
1968
+ "provenance": {
1969
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
1970
+ },
1971
+ "attrs": { "num_heads": 3 },
1972
+ "inputs": {
1973
+ "queryLayerT": {
1974
+ "dtype": "float16",
1975
+ "shape": [2, 64, 381],
1976
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
1977
+ },
1978
+ "queryBiasT": {
1979
+ "dtype": "float16",
1980
+ "shape": [381],
1981
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
1982
+ },
1983
+ "relPosT": {
1984
+ "dtype": "float16",
1985
+ "shape": [1, 3, 64, 64],
1986
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
1987
+ },
1988
+ "weightT": {
1989
+ "dtype": "float16",
1990
+ "shape": [127, 6],
1991
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
1992
+ },
1993
+ "biasT": {
1994
+ "dtype": "float16",
1995
+ "shape": [6],
1996
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
1997
+ },
1998
+ "ecoAT": {
1999
+ "dtype": "float16",
2000
+ "shape": [1, 3, 1, 1],
2001
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2002
+ }
2003
+ },
2004
+ "outputs": {
2005
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2006
+ }
2007
+ },
2008
+ {
2009
+ "name": "head_projection_general-b2-n3-s256-h128-d32-f16_fixture",
2010
+ "provenance": {
2011
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
2012
+ },
2013
+ "attrs": { "num_heads": 3 },
2014
+ "inputs": {
2015
+ "queryLayerT": {
2016
+ "dtype": "float16",
2017
+ "shape": [2, 64, 384],
2018
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2019
+ },
2020
+ "queryBiasT": {
2021
+ "dtype": "float16",
2022
+ "shape": [384],
2023
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2024
+ },
2025
+ "relPosT": {
2026
+ "dtype": "float16",
2027
+ "shape": [1, 3, 64, 64],
2028
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2029
+ },
2030
+ "weightT": {
2031
+ "dtype": "float16",
2032
+ "shape": [128, 32],
2033
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2034
+ },
2035
+ "biasT": {
2036
+ "dtype": "float16",
2037
+ "shape": [32],
2038
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2039
+ },
2040
+ "ecoAT": {
2041
+ "dtype": "float16",
2042
+ "shape": [1, 3, 1, 1],
2043
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2044
+ }
2045
+ },
2046
+ "outputs": {
2047
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2048
+ }
2049
+ },
2050
+ {
2051
+ "name": "head_projection_general-b1-n4-s512-h256-d16-f16_fixture",
2052
+ "provenance": {
2053
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
2054
+ },
2055
+ "attrs": { "num_heads": 4 },
2056
+ "inputs": {
2057
+ "queryLayerT": {
2058
+ "dtype": "float16",
2059
+ "shape": [1, 64, 1024],
2060
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2061
+ },
2062
+ "queryBiasT": {
2063
+ "dtype": "float16",
2064
+ "shape": [1024],
2065
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2066
+ },
2067
+ "relPosT": {
2068
+ "dtype": "float16",
2069
+ "shape": [1, 4, 64, 64],
2070
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2071
+ },
2072
+ "weightT": {
2073
+ "dtype": "float16",
2074
+ "shape": [256, 16],
2075
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2076
+ },
2077
+ "biasT": {
2078
+ "dtype": "float16",
2079
+ "shape": [16],
2080
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2081
+ },
2082
+ "ecoAT": {
2083
+ "dtype": "float16",
2084
+ "shape": [1, 4, 1, 1],
2085
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2086
+ }
2087
+ },
2088
+ "outputs": {
2089
+ "outputT": { "shape": [1, 4, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2090
+ }
2091
+ },
2092
+ {
2093
+ "name": "head_projection_general-b1-n2-s63-h512-d8-f16_fixture",
2094
+ "provenance": {
2095
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
2096
+ },
2097
+ "attrs": { "num_heads": 2 },
2098
+ "inputs": {
2099
+ "queryLayerT": {
2100
+ "dtype": "float16",
2101
+ "shape": [1, 63, 1024],
2102
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2103
+ },
2104
+ "queryBiasT": {
2105
+ "dtype": "float16",
2106
+ "shape": [1024],
2107
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2108
+ },
2109
+ "relPosT": {
2110
+ "dtype": "float16",
2111
+ "shape": [1, 2, 63, 63],
2112
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2113
+ },
2114
+ "weightT": {
2115
+ "dtype": "float16",
2116
+ "shape": [512, 8],
2117
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2118
+ },
2119
+ "biasT": {
2120
+ "dtype": "float16",
2121
+ "shape": [8],
2122
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2123
+ },
2124
+ "ecoAT": {
2125
+ "dtype": "float16",
2126
+ "shape": [1, 2, 1, 1],
2127
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2128
+ }
2129
+ },
2130
+ "outputs": {
2131
+ "outputT": { "shape": [1, 2, 63, 63], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2132
+ }
2133
+ },
2134
+ {
2135
+ "name": "head_projection_general-b1-n2-s32-h1024-d32-f16_fixture",
2136
+ "provenance": {
2137
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
2138
+ },
2139
+ "attrs": { "num_heads": 2 },
2140
+ "inputs": {
2141
+ "queryLayerT": {
2142
+ "dtype": "float16",
2143
+ "shape": [1, 32, 2048],
2144
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2145
+ },
2146
+ "queryBiasT": {
2147
+ "dtype": "float16",
2148
+ "shape": [2048],
2149
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2150
+ },
2151
+ "relPosT": {
2152
+ "dtype": "float16",
2153
+ "shape": [1, 2, 32, 32],
2154
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2155
+ },
2156
+ "weightT": {
2157
+ "dtype": "float16",
2158
+ "shape": [1024, 32],
2159
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2160
+ },
2161
+ "biasT": {
2162
+ "dtype": "float16",
2163
+ "shape": [32],
2164
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2165
+ },
2166
+ "ecoAT": {
2167
+ "dtype": "float16",
2168
+ "shape": [1, 2, 1, 1],
2169
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2170
+ }
2171
+ },
2172
+ "outputs": {
2173
+ "outputT": { "shape": [1, 2, 32, 32], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2174
+ }
2175
+ },
2176
+ {
2177
+ "name": "head_projection_general-b1-n2-s2053-h64-d8-f16_fixture",
2178
+ "provenance": {
2179
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
2180
+ },
2181
+ "attrs": { "num_heads": 2 },
2182
+ "tunables": { "WORKGROUP_SIZE": 32 },
2183
+ "inputs": {
2184
+ "queryLayerT": {
2185
+ "dtype": "float16",
2186
+ "shape": [1, 258, 128],
2187
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2188
+ },
2189
+ "queryBiasT": {
2190
+ "dtype": "float16",
2191
+ "shape": [128],
2192
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2193
+ },
2194
+ "relPosT": {
2195
+ "dtype": "float16",
2196
+ "shape": [1, 2, 258, 258],
2197
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2198
+ },
2199
+ "weightT": {
2200
+ "dtype": "float16",
2201
+ "shape": [64, 8],
2202
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2203
+ },
2204
+ "biasT": {
2205
+ "dtype": "float16",
2206
+ "shape": [8],
2207
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2208
+ },
2209
+ "ecoAT": {
2210
+ "dtype": "float16",
2211
+ "shape": [1, 2, 1, 1],
2212
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2213
+ }
2214
+ },
2215
+ "outputs": {
2216
+ "outputT": { "shape": [1, 2, 258, 258], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2217
+ }
2218
+ },
2219
+ {
2220
+ "name": "head_projection_stream-b2-n3-s258-h31-d6-f16_fixture",
2221
+ "provenance": {
2222
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
2223
+ },
2224
+ "attrs": { "num_heads": 3 },
2225
+ "tunables": { "WORKGROUP_SIZE": 32 },
2226
+ "inputs": {
2227
+ "queryLayerT": {
2228
+ "dtype": "float16",
2229
+ "shape": [2, 258, 93],
2230
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2231
+ },
2232
+ "queryBiasT": {
2233
+ "dtype": "float16",
2234
+ "shape": [93],
2235
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2236
+ },
2237
+ "relPosT": {
2238
+ "dtype": "float16",
2239
+ "shape": [1, 3, 258, 258],
2240
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2241
+ },
2242
+ "weightT": {
2243
+ "dtype": "float16",
2244
+ "shape": [31, 6],
2245
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2246
+ },
2247
+ "biasT": {
2248
+ "dtype": "float16",
2249
+ "shape": [6],
2250
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2251
+ },
2252
+ "ecoAT": {
2253
+ "dtype": "float16",
2254
+ "shape": [1, 3, 1, 1],
2255
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2256
+ }
2257
+ },
2258
+ "outputs": {
2259
+ "outputT": { "shape": [2, 3, 258, 258], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2260
+ }
2261
+ },
2262
+ {
2263
+ "name": "head_projection_wg64-b2-n3-s257-h127-d8-f16_fixture",
2264
+ "provenance": {
2265
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
2266
+ },
2267
+ "attrs": { "num_heads": 3 },
2268
+ "tunables": { "WORKGROUP_SIZE": 64, "LANE_WORDS": 1 },
2269
+ "inputs": {
2270
+ "queryLayerT": {
2271
+ "dtype": "float16",
2272
+ "shape": [2, 64, 381],
2273
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2274
+ },
2275
+ "queryBiasT": {
2276
+ "dtype": "float16",
2277
+ "shape": [381],
2278
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2279
+ },
2280
+ "relPosT": {
2281
+ "dtype": "float16",
2282
+ "shape": [1, 3, 64, 64],
2283
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2284
+ },
2285
+ "weightT": {
2286
+ "dtype": "float16",
2287
+ "shape": [127, 8],
2288
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2289
+ },
2290
+ "biasT": {
2291
+ "dtype": "float16",
2292
+ "shape": [8],
2293
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2294
+ },
2295
+ "ecoAT": {
2296
+ "dtype": "float16",
2297
+ "shape": [1, 3, 1, 1],
2298
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2299
+ }
2300
+ },
2301
+ "outputs": {
2302
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2303
+ }
2304
+ },
2305
+ {
2306
+ "name": "head_projection_wg256-b2-n3-s256-h128-d32-f16_fixture",
2307
+ "provenance": {
2308
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
2309
+ },
2310
+ "attrs": { "num_heads": 3 },
2311
+ "tunables": { "WORKGROUP_SIZE": 256, "LANE_WORDS": 1 },
2312
+ "inputs": {
2313
+ "queryLayerT": {
2314
+ "dtype": "float16",
2315
+ "shape": [2, 64, 384],
2316
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2317
+ },
2318
+ "queryBiasT": {
2319
+ "dtype": "float16",
2320
+ "shape": [384],
2321
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2322
+ },
2323
+ "relPosT": {
2324
+ "dtype": "float16",
2325
+ "shape": [1, 3, 64, 64],
2326
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2327
+ },
2328
+ "weightT": {
2329
+ "dtype": "float16",
2330
+ "shape": [128, 32],
2331
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2332
+ },
2333
+ "biasT": {
2334
+ "dtype": "float16",
2335
+ "shape": [32],
2336
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2337
+ },
2338
+ "ecoAT": {
2339
+ "dtype": "float16",
2340
+ "shape": [1, 3, 1, 1],
2341
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2342
+ }
2343
+ },
2344
+ "outputs": {
2345
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2346
+ }
2347
+ },
2348
+ {
2349
+ "name": "head_projection_wg512-b2-n3-s128-h127-d32-f16_fixture",
2350
+ "provenance": {
2351
+ "notes": "Compact projection-layout case with the same head and projection dimensions, two gate halves, and dtype as the paired large case."
2352
+ },
2353
+ "attrs": { "num_heads": 3 },
2354
+ "tunables": { "WORKGROUP_SIZE": 256, "LANE_WORDS": 8 },
2355
+ "inputs": {
2356
+ "queryLayerT": {
2357
+ "dtype": "float16",
2358
+ "shape": [2, 64, 381],
2359
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2360
+ },
2361
+ "queryBiasT": {
2362
+ "dtype": "float16",
2363
+ "shape": [381],
2364
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2365
+ },
2366
+ "relPosT": {
2367
+ "dtype": "float16",
2368
+ "shape": [1, 3, 64, 64],
2369
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2370
+ },
2371
+ "weightT": {
2372
+ "dtype": "float16",
2373
+ "shape": [127, 32],
2374
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2375
+ },
2376
+ "biasT": {
2377
+ "dtype": "float16",
2378
+ "shape": [32],
2379
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2380
+ },
2381
+ "ecoAT": {
2382
+ "dtype": "float16",
2383
+ "shape": [1, 3, 1, 1],
2384
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2385
+ }
2386
+ },
2387
+ "outputs": {
2388
+ "outputT": { "shape": [2, 3, 64, 64], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2389
+ }
2390
+ },
2391
+ {
2392
+ "name": "head_projection_vec-stream-b1-n2-s1056-h64-d8-f32_fixture",
2393
+ "provenance": {
2394
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
2395
+ },
2396
+ "attrs": { "num_heads": 2 },
2397
+ "tunables": { "WORKGROUP_SIZE": 32 },
2398
+ "inputs": {
2399
+ "queryLayerT": {
2400
+ "dtype": "float32",
2401
+ "shape": [1, 1056, 128],
2402
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2403
+ },
2404
+ "queryBiasT": {
2405
+ "dtype": "float32",
2406
+ "shape": [128],
2407
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2408
+ },
2409
+ "relPosT": {
2410
+ "dtype": "float32",
2411
+ "shape": [1, 2, 1056, 1056],
2412
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2413
+ },
2414
+ "weightT": {
2415
+ "dtype": "float32",
2416
+ "shape": [64, 8],
2417
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2418
+ },
2419
+ "biasT": {
2420
+ "dtype": "float32",
2421
+ "shape": [8],
2422
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2423
+ },
2424
+ "ecoAT": {
2425
+ "dtype": "float32",
2426
+ "shape": [1, 2, 1, 1],
2427
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2428
+ }
2429
+ },
2430
+ "outputs": {
2431
+ "outputT": { "shape": [1, 2, 1056, 1056], "dtype": "float32", "tolerance": 0.000002, "relTolerance": 0.00002 }
2432
+ }
2433
+ },
2434
+ {
2435
+ "name": "head_projection_vec-stream-b1-n2-s1056-h64-d8-f16_fixture",
2436
+ "provenance": {
2437
+ "notes": "Projection-layout case with the same shapes, head and projection dimensions, two gate halves, and dtype as the paired benchmark case."
2438
+ },
2439
+ "attrs": { "num_heads": 2 },
2440
+ "tunables": { "WORKGROUP_SIZE": 32 },
2441
+ "inputs": {
2442
+ "queryLayerT": {
2443
+ "dtype": "float16",
2444
+ "shape": [1, 1056, 128],
2445
+ "data": { "kind": "fillFloat32", "sinStep": 0.13, "cosStep": 0.29, "scale": 0.25 }
2446
+ },
2447
+ "queryBiasT": {
2448
+ "dtype": "float16",
2449
+ "shape": [128],
2450
+ "data": { "kind": "fillFloat32", "sinStep": 0.177, "cosStep": 0.26099999999999995, "scale": 0.2 }
2451
+ },
2452
+ "relPosT": {
2453
+ "dtype": "float16",
2454
+ "shape": [1, 2, 1056, 1056],
2455
+ "data": { "kind": "fillFloat32", "sinStep": 0.224, "cosStep": 0.23199999999999998, "scale": 1.0 }
2456
+ },
2457
+ "weightT": {
2458
+ "dtype": "float16",
2459
+ "shape": [64, 8],
2460
+ "data": { "kind": "fillFloat32", "sinStep": 0.271, "cosStep": 0.20299999999999996, "scale": 0.25 }
2461
+ },
2462
+ "biasT": {
2463
+ "dtype": "float16",
2464
+ "shape": [8],
2465
+ "data": { "kind": "fillFloat32", "sinStep": 0.318, "cosStep": 0.174, "scale": 0.3 }
2466
+ },
2467
+ "ecoAT": {
2468
+ "dtype": "float16",
2469
+ "shape": [1, 2, 1, 1],
2470
+ "data": { "kind": "fillFloat32", "sinStep": 0.365, "cosStep": 0.14499999999999996, "scale": 1.0 }
2471
+ }
2472
+ },
2473
+ "outputs": {
2474
+ "outputT": { "shape": [1, 2, 1056, 1056], "dtype": "float16", "tolerance": 0.0005, "relTolerance": 0.002 }
2475
+ }
2476
+ }
2477
+ ]
2478
+ }