Remove selected validation and provenance files
Browse filesDelete the two standalone deployment validators, validation reports and provenance files. Retain component README and benchmark results, and clean obsolete documentation links and commands.
- PRECISION_ANALYSIS.md +1 -2
- README.md +2 -4
- gdn/README.md +5 -9
- gdn/provenance.json +0 -26
- gdn/validate.py +0 -126
- gdn/validation_results.json +0 -112
- kda/README.md +5 -9
- kda/provenance.json +0 -26
- kda/validate.py +0 -126
- kda/validation_results.json +0 -112
PRECISION_ANALYSIS.md
CHANGED
|
@@ -268,8 +268,7 @@ than a long chain of each implementation's own outputs.
|
|
| 268 |
- Unmodified TileLang baselines: [GDN](gdn/baseline.py), [KDA](kda/baseline.py).
|
| 269 |
- Released CuTe sources: [GDN](build/torch-cuda/_gdn.py), [KDA](build/torch-cuda/_kda.py).
|
| 270 |
- Matched measurements: [GDN](gdn/precision_comparison.json), [KDA](kda/precision_comparison.json).
|
| 271 |
-
-
|
| 272 |
-
- Source hashes and provenance: [GDN](gdn/provenance.json), [KDA](kda/provenance.json).
|
| 273 |
|
| 274 |
Using the pinned [dependencies](requirements.txt), run on an idle B300:
|
| 275 |
|
|
|
|
| 268 |
- Unmodified TileLang baselines: [GDN](gdn/baseline.py), [KDA](kda/baseline.py).
|
| 269 |
- Released CuTe sources: [GDN](build/torch-cuda/_gdn.py), [KDA](build/torch-cuda/_kda.py).
|
| 270 |
- Matched measurements: [GDN](gdn/precision_comparison.json), [KDA](kda/precision_comparison.json).
|
| 271 |
+
- Source hashes and release details: [GDN](gdn/README.md#files-and-provenance), [KDA](kda/README.md#files-and-provenance).
|
|
|
|
| 272 |
|
| 273 |
Using the pinned [dependencies](requirements.txt), run on an idle B300:
|
| 274 |
|
README.md
CHANGED
|
@@ -52,8 +52,8 @@ The `main` and `v1` branches both contain the complete release. `v1` is the
|
|
| 52 |
version loaded above. The runtime sources are `build/torch-cuda/_gdn.py` and
|
| 53 |
`build/torch-cuda/_kda.py`; metadata, the public import wrapper and the runtime
|
| 54 |
license are alongside them. Each component directory contains its README,
|
| 55 |
-
original workload, benchmark, baseline and reference definition, plus
|
| 56 |
-
and
|
| 57 |
|
| 58 |
## Precision
|
| 59 |
|
|
@@ -76,10 +76,8 @@ cd leapquant-kernels
|
|
| 76 |
python -m pip install -r requirements.txt
|
| 77 |
cd gdn
|
| 78 |
python benchmark.py --impl ../build/torch-cuda/_gdn.py
|
| 79 |
-
python validate.py
|
| 80 |
cd ../kda
|
| 81 |
python benchmark.py --impl ../build/torch-cuda/_kda.py
|
| 82 |
-
python validate.py
|
| 83 |
```
|
| 84 |
|
| 85 |
Run `python benchmark.py` in either component directory for the original
|
|
|
|
| 52 |
version loaded above. The runtime sources are `build/torch-cuda/_gdn.py` and
|
| 53 |
`build/torch-cuda/_kda.py`; metadata, the public import wrapper and the runtime
|
| 54 |
license are alongside them. Each component directory contains its README,
|
| 55 |
+
original workload, benchmark, baseline and reference definition, plus benchmark
|
| 56 |
+
and precision results. Dependency pins and Apache-2.0 licensing are shared at the root.
|
| 57 |
|
| 58 |
## Precision
|
| 59 |
|
|
|
|
| 76 |
python -m pip install -r requirements.txt
|
| 77 |
cd gdn
|
| 78 |
python benchmark.py --impl ../build/torch-cuda/_gdn.py
|
|
|
|
| 79 |
cd ../kda
|
| 80 |
python benchmark.py --impl ../build/torch-cuda/_kda.py
|
|
|
|
| 81 |
```
|
| 82 |
|
| 83 |
Run `python benchmark.py` in either component directory for the original
|
gdn/README.md
CHANGED
|
@@ -84,9 +84,7 @@ be scanning the due flags.
|
|
| 84 |
Construct the pool and perform warmup before CUDA graph capture. Refill the
|
| 85 |
window buffers and counters before each new window. Calls use the current CUDA
|
| 86 |
stream on the pool's device. Keep the pool, index vector and `q0` buffers alive
|
| 87 |
-
while a graph using their addresses is alive.
|
| 88 |
-
mixed due heads, slot 0, reset/no-op behavior, a non-default stream and graph
|
| 89 |
-
replay with changed input contents.
|
| 90 |
|
| 91 |
## Accuracy and timing
|
| 92 |
|
|
@@ -127,7 +125,6 @@ python -m pip install torch==2.12.1 --index-url https://download.pytorch.org/whl
|
|
| 127 |
python -m pip install -r ../requirements.txt
|
| 128 |
python benchmark.py --impl ../build/torch-cuda/_gdn.py # six original workloads
|
| 129 |
python benchmark.py # original TileLang baseline
|
| 130 |
-
python validate.py # package loader and deployment checks
|
| 131 |
```
|
| 132 |
|
| 133 |
TileLang and `nvcc` on `PATH` are needed only to benchmark the original baseline.
|
|
@@ -136,9 +133,8 @@ steady-state timings or perform first-use compilation inside graph capture.
|
|
| 136 |
|
| 137 |
|
| 138 |
The unchanged kernel passed all six original workloads (three input families
|
| 139 |
-
each) and deployment checks on B300 before consolidation
|
| 140 |
-
|
| 141 |
-
optimization-run measurement, not a new baseline comparison.
|
| 142 |
|
| 143 |
## Matched precision comparison
|
| 144 |
|
|
@@ -158,14 +154,14 @@ Reproduce from the repository root with `python compare_precision.py gdn`.
|
|
| 158 |
|
| 159 |
The combined runtime is in `../build/torch-cuda/`; this component's unchanged
|
| 160 |
kernel is `_gdn.py`. The files in this directory provide its original
|
| 161 |
-
benchmark, baseline, definition, workloads,
|
| 162 |
Shared dependencies and licenses are at the repository root.
|
| 163 |
|
| 164 |
The runtime kernel is byte-for-byte the final accepted submission
|
| 165 |
`8f705942197446f4` (SHA-256 `a734b59e75a8c8e870d68280c59c4fe05e9e3d2334c4d7e62d7101c67c9cd9b6`), from a 48-hour KDA experiment
|
| 166 |
with GPT 6.1 and Claude Opus 5.5. Request sources are pinned to public commit
|
| 167 |
`48f02e6ba0e98a1841ab6f0a2d99a90b2de1a326`. The release adds a public import wrapper, metadata
|
| 168 |
-
and
|
| 169 |
see `../LICENSE.txt` and `../NOTICE`.
|
| 170 |
|
| 171 |
## Available functions
|
|
|
|
| 84 |
Construct the pool and perform warmup before CUDA graph capture. Refill the
|
| 85 |
window buffers and counters before each new window. Calls use the current CUDA
|
| 86 |
stream on the pool's device. Keep the pool, index vector and `q0` buffers alive
|
| 87 |
+
while a graph using their addresses is alive.
|
|
|
|
|
|
|
| 88 |
|
| 89 |
## Accuracy and timing
|
| 90 |
|
|
|
|
| 125 |
python -m pip install -r ../requirements.txt
|
| 126 |
python benchmark.py --impl ../build/torch-cuda/_gdn.py # six original workloads
|
| 127 |
python benchmark.py # original TileLang baseline
|
|
|
|
| 128 |
```
|
| 129 |
|
| 130 |
TileLang and `nvcc` on `PATH` are needed only to benchmark the original baseline.
|
|
|
|
| 133 |
|
| 134 |
|
| 135 |
The unchanged kernel passed all six original workloads (three input families
|
| 136 |
+
each) and deployment checks on B300 before consolidation. The speedup table
|
| 137 |
+
retains the paired optimization-run measurement, not a new baseline comparison.
|
|
|
|
| 138 |
|
| 139 |
## Matched precision comparison
|
| 140 |
|
|
|
|
| 154 |
|
| 155 |
The combined runtime is in `../build/torch-cuda/`; this component's unchanged
|
| 156 |
kernel is `_gdn.py`. The files in this directory provide its original
|
| 157 |
+
benchmark, baseline, definition, workloads, benchmark and precision results.
|
| 158 |
Shared dependencies and licenses are at the repository root.
|
| 159 |
|
| 160 |
The runtime kernel is byte-for-byte the final accepted submission
|
| 161 |
`8f705942197446f4` (SHA-256 `a734b59e75a8c8e870d68280c59c4fe05e9e3d2334c4d7e62d7101c67c9cd9b6`), from a 48-hour KDA experiment
|
| 162 |
with GPT 6.1 and Claude Opus 5.5. Request sources are pinned to public commit
|
| 163 |
`48f02e6ba0e98a1841ab6f0a2d99a90b2de1a326`. The release adds a public import wrapper, metadata
|
| 164 |
+
and precision comparison tooling; it does not change the kernel algorithm. Apache-2.0;
|
| 165 |
see `../LICENSE.txt` and `../NOTICE`.
|
| 166 |
|
| 167 |
## Available functions
|
gdn/provenance.json
DELETED
|
@@ -1,26 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"repo_id": "humanfia-lab/leapquant",
|
| 3 |
-
"version": 1,
|
| 4 |
-
"source_url": "https://github.com/NVlabs/kda/pull/10",
|
| 5 |
-
"source_commit": "48f02e6ba0e98a1841ab6f0a2d99a90b2de1a326",
|
| 6 |
-
"upstream_sha256": {
|
| 7 |
-
"baseline.py": "b53f30426a9647e523fc6463a5a8d1ba1a9c0f9a09d83599b10abf631346712f",
|
| 8 |
-
"benchmark.py": "58c5404386dd121ef7e2dca73c3488c7e23b1e0d16f69e23dc1d33b563d3d62f",
|
| 9 |
-
"definition.json": "313e8b7d2d7305fb09aaa6771eb6482ee2182567339a4d7c33388e9db68296d7",
|
| 10 |
-
"workloads.jsonl": "4fb72f317e2eabd9752907e6bd0d4cebc4e9a23c6c48c5a4a95750e13e805145"
|
| 11 |
-
},
|
| 12 |
-
"kernel_sha256": "a734b59e75a8c8e870d68280c59c4fe05e9e3d2334c4d7e62d7101c67c9cd9b6",
|
| 13 |
-
"kernel_git_commit": "06975ceabe46d5122fe76b742f71d1eeb841a9bd",
|
| 14 |
-
"official_submission_id": "8f705942197446f4",
|
| 15 |
-
"official_artifact_digest": "8d7d94ad883d3c95960eea61a2b63cb85b9633f900ed26394cb1bd82ccc2ef11",
|
| 16 |
-
"official_benchmarks_sha": "3ac105d12ef108513555f8eac2a39203c0a18c5af3a8b6f77cb12751274ede75",
|
| 17 |
-
"official_evaluation_utc": "2026-10-24T10:57:39Z",
|
| 18 |
-
"experiment_hours": 48,
|
| 19 |
-
"models": [
|
| 20 |
-
"codex/gpt-6.1-sol:max",
|
| 21 |
-
"claude/claude-opus-5-5:max"
|
| 22 |
-
],
|
| 23 |
-
"packaging": "Unmodified accepted kernel; exposed as a module of the combined HF package.",
|
| 24 |
-
"component": "gdn",
|
| 25 |
-
"kernel_path": "build/torch-cuda/_gdn.py"
|
| 26 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
gdn/validate.py
DELETED
|
@@ -1,126 +0,0 @@
|
|
| 1 |
-
# SPDX-License-Identifier: Apache-2.0
|
| 2 |
-
"""Validate the local HF package and caller-owned deployment contract on CUDA."""
|
| 3 |
-
from __future__ import annotations
|
| 4 |
-
|
| 5 |
-
import base64
|
| 6 |
-
import hashlib
|
| 7 |
-
import json
|
| 8 |
-
from pathlib import Path
|
| 9 |
-
|
| 10 |
-
import torch
|
| 11 |
-
from kernels import get_local_kernel
|
| 12 |
-
|
| 13 |
-
import benchmark as bench
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
def main() -> None:
|
| 17 |
-
here = Path(__file__).resolve().parent
|
| 18 |
-
runtime = here.parent / "build/torch-cuda"
|
| 19 |
-
metadata = json.loads((runtime / "metadata.json").read_text())
|
| 20 |
-
for filename, expected in metadata["digest"]["files"].items():
|
| 21 |
-
actual = base64.b64encode(hashlib.sha256((runtime / filename).read_bytes()).digest()).decode()
|
| 22 |
-
assert actual == expected, f"digest mismatch: {filename}"
|
| 23 |
-
op = get_local_kernel(here.parent, trust_remote_code=True).gdn
|
| 24 |
-
assert all(hasattr(op, name) for name in ("Pool", "flush_inplace", "run"))
|
| 25 |
-
namespace = {}
|
| 26 |
-
exec(json.loads((here / "definition.json").read_text())["reference"], namespace)
|
| 27 |
-
reference = namespace["run"]
|
| 28 |
-
device = torch.device("cuda")
|
| 29 |
-
ns = 8
|
| 30 |
-
pool = op.Pool(ns, device=device)
|
| 31 |
-
is_kda = hasattr(pool, "gbuf")
|
| 32 |
-
idx = torch.tensor([0, 7, 2, 5, 1, 3], dtype=torch.int32, device=device)
|
| 33 |
-
counters = torch.where(torch.arange(ns * 32, device=device).reshape(ns, 32) % 3 == 0, 16, 7).int()
|
| 34 |
-
selected = torch.zeros(ns, dtype=torch.bool, device=device)
|
| 35 |
-
selected[idx.long()] = True
|
| 36 |
-
selected[0] = False
|
| 37 |
-
due = selected[:, None] & (counters == 16)
|
| 38 |
-
fields = ["codes", "s_k", "s_v", "uq", "kbuf", "ubuf", "w", "hcnt"] + (["gbuf", "pcum"] if is_kda else ["p"])
|
| 39 |
-
q0 = torch.empty(128, 4, device=device)
|
| 40 |
-
|
| 41 |
-
def restore(inputs):
|
| 42 |
-
pool.codes.copy_(inputs[0]); pool.s_k.copy_(inputs[1]); pool.s_v.copy_(inputs[2])
|
| 43 |
-
pool.uq[..., :128].copy_(inputs[3]); pool.uq[..., 128:].copy_(inputs[4])
|
| 44 |
-
pool.kbuf.copy_(inputs[5]); pool.ubuf.copy_(inputs[6])
|
| 45 |
-
if is_kda:
|
| 46 |
-
pool.gbuf.copy_(inputs[7]); pool.w.copy_(inputs[8]); pool.pcum.copy_(inputs[9])
|
| 47 |
-
else:
|
| 48 |
-
pool.w.copy_(inputs[7]); pool.p[..., 0].copy_(inputs[8])
|
| 49 |
-
pool.hcnt.copy_(counters)
|
| 50 |
-
q0.copy_(inputs[-1])
|
| 51 |
-
|
| 52 |
-
def snapshot():
|
| 53 |
-
return {name: getattr(pool, name).clone() for name in fields}
|
| 54 |
-
|
| 55 |
-
def verify(inputs, before):
|
| 56 |
-
ref = reference(*[tensor.clone() for tensor in inputs])
|
| 57 |
-
actual = (pool.codes, pool.s_k, pool.s_v, pool.uq[..., :128], pool.uq[..., 128:])
|
| 58 |
-
# Each selected (slot, head) is a separate checked head, never averaged across slots.
|
| 59 |
-
out = tuple(tensor[due].unsqueeze(0) for tensor in actual)
|
| 60 |
-
ref = tuple(tensor[due].unsqueeze(0) for tensor in ref)
|
| 61 |
-
exact = bench.exact_state(inputs)[due].unsqueeze(0)
|
| 62 |
-
passed, ratio = bench.judge(out, ref, exact)
|
| 63 |
-
assert passed, f"deployment accuracy failed: {ratio}"
|
| 64 |
-
for name in fields:
|
| 65 |
-
assert torch.equal(getattr(pool, name)[~due], before[name][~due]), f"non-due/slot-0 write: {name}"
|
| 66 |
-
for name in ("kbuf", "ubuf"):
|
| 67 |
-
assert torch.equal(getattr(pool, name), before[name]), f"update buffer changed: {name}"
|
| 68 |
-
# The serving contract leaves hcnt at L ("flushed, empty"); the decode
|
| 69 |
-
# step consumes that sentinel. Clearing it in flush would race scanners.
|
| 70 |
-
assert torch.equal(pool.hcnt, before["hcnt"]), "flush changed fill counters"
|
| 71 |
-
assert bool((pool.w[due] == 0).all())
|
| 72 |
-
if is_kda:
|
| 73 |
-
assert bool((pool.gbuf[due] == 1).all())
|
| 74 |
-
assert bool((pool.pcum[due] == 0).all())
|
| 75 |
-
else:
|
| 76 |
-
assert bool((pool.p[due] == 1).all())
|
| 77 |
-
return float(ratio)
|
| 78 |
-
|
| 79 |
-
stream = torch.cuda.Stream()
|
| 80 |
-
inputs = bench.structured_inputs(ns, device, seed=91)
|
| 81 |
-
restore(inputs)
|
| 82 |
-
torch.cuda.synchronize()
|
| 83 |
-
before = snapshot()
|
| 84 |
-
torch.cuda.synchronize()
|
| 85 |
-
with torch.cuda.stream(stream):
|
| 86 |
-
op.flush_inplace(pool, idx, q0)
|
| 87 |
-
stream.synchronize()
|
| 88 |
-
ratios = [verify(inputs, before)]
|
| 89 |
-
before = snapshot()
|
| 90 |
-
op.flush_inplace(pool, torch.empty(0, dtype=torch.int32, device=device), q0)
|
| 91 |
-
torch.cuda.synchronize()
|
| 92 |
-
assert all(torch.equal(getattr(pool, name), before[name]) for name in fields), "empty idx is not a no-op"
|
| 93 |
-
|
| 94 |
-
restore(inputs)
|
| 95 |
-
torch.cuda.synchronize()
|
| 96 |
-
graph = torch.cuda.CUDAGraph()
|
| 97 |
-
with torch.cuda.graph(graph, stream=stream):
|
| 98 |
-
op.flush_inplace(pool, idx, q0)
|
| 99 |
-
for seed in (92, 93):
|
| 100 |
-
inputs = bench.structured_inputs(ns, device, seed=seed)
|
| 101 |
-
# structured_inputs always uses the same q0; vary it explicitly here.
|
| 102 |
-
generator = torch.Generator(device=device).manual_seed(seed)
|
| 103 |
-
inputs = (*inputs[:-1], torch.randn(128, 4, device=device, generator=generator))
|
| 104 |
-
restore(inputs)
|
| 105 |
-
before = snapshot()
|
| 106 |
-
torch.cuda.synchronize()
|
| 107 |
-
graph.replay()
|
| 108 |
-
torch.cuda.synchronize()
|
| 109 |
-
ratios.append(verify(inputs, before))
|
| 110 |
-
# Simulate the caller advancing the counters so no head is due.
|
| 111 |
-
pool.hcnt[due] = 0
|
| 112 |
-
after = snapshot()
|
| 113 |
-
torch.cuda.synchronize()
|
| 114 |
-
graph.replay()
|
| 115 |
-
torch.cuda.synchronize()
|
| 116 |
-
assert all(torch.equal(getattr(pool, name), after[name]) for name in fields), "replay with no due heads is not a no-op"
|
| 117 |
-
print(json.dumps({
|
| 118 |
-
"status": "pass", "gpu": torch.cuda.get_device_name(),
|
| 119 |
-
"compute_capability": list(torch.cuda.get_device_capability()),
|
| 120 |
-
"checks": ["runtime digests", "HF get_local_kernel", "mixed due heads", "unselected slots", "reserved slot 0", "window resets", "fill counters unchanged", "empty idx", "non-default stream", "graph capture", "graph replay with changed inputs and q0", "no-op replay with no due heads"],
|
| 121 |
-
"deployment_worst_head_error_ratios": ratios,
|
| 122 |
-
}, indent=2))
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
if __name__ == "__main__":
|
| 126 |
-
main()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
gdn/validation_results.json
DELETED
|
@@ -1,112 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"validated_at_utc": "2026-10-25T20:03:16.823824+00:00",
|
| 3 |
-
"kernel_sha256": "a734b59e75a8c8e870d68280c59c4fe05e9e3d2334c4d7e62d7101c67c9cd9b6",
|
| 4 |
-
"deployment": {
|
| 5 |
-
"status": "pass",
|
| 6 |
-
"gpu": "NVIDIA B300 SXM6 AC",
|
| 7 |
-
"compute_capability": [
|
| 8 |
-
10,
|
| 9 |
-
3
|
| 10 |
-
],
|
| 11 |
-
"checks": [
|
| 12 |
-
"runtime digests",
|
| 13 |
-
"HF get_local_kernel",
|
| 14 |
-
"mixed due heads",
|
| 15 |
-
"unselected slots",
|
| 16 |
-
"reserved slot 0",
|
| 17 |
-
"window resets",
|
| 18 |
-
"fill counters unchanged",
|
| 19 |
-
"empty idx",
|
| 20 |
-
"non-default stream",
|
| 21 |
-
"graph capture",
|
| 22 |
-
"graph replay with changed inputs and q0",
|
| 23 |
-
"no-op replay with no due heads"
|
| 24 |
-
],
|
| 25 |
-
"deployment_worst_head_error_ratios": [
|
| 26 |
-
1.0022517222625789,
|
| 27 |
-
1.0016240993273937,
|
| 28 |
-
1.0018801265800452
|
| 29 |
-
]
|
| 30 |
-
},
|
| 31 |
-
"original_benchmark": {
|
| 32 |
-
"workloads_passed": 6,
|
| 33 |
-
"input_families_per_workload": 3,
|
| 34 |
-
"thresholds_modified": false,
|
| 35 |
-
"workloads": [
|
| 36 |
-
{
|
| 37 |
-
"batch_size": 16,
|
| 38 |
-
"programs": 512,
|
| 39 |
-
"error_ratios": [
|
| 40 |
-
1.0004,
|
| 41 |
-
1.0003,
|
| 42 |
-
1.0004
|
| 43 |
-
],
|
| 44 |
-
"latency_ms": 0.0121,
|
| 45 |
-
"correctness_passed": true
|
| 46 |
-
},
|
| 47 |
-
{
|
| 48 |
-
"batch_size": 32,
|
| 49 |
-
"programs": 1024,
|
| 50 |
-
"error_ratios": [
|
| 51 |
-
1.0004,
|
| 52 |
-
1.0002,
|
| 53 |
-
1.0002
|
| 54 |
-
],
|
| 55 |
-
"latency_ms": 0.0183,
|
| 56 |
-
"correctness_passed": true
|
| 57 |
-
},
|
| 58 |
-
{
|
| 59 |
-
"batch_size": 64,
|
| 60 |
-
"programs": 2048,
|
| 61 |
-
"error_ratios": [
|
| 62 |
-
1.0002,
|
| 63 |
-
1.0002,
|
| 64 |
-
1.0002
|
| 65 |
-
],
|
| 66 |
-
"latency_ms": 0.029,
|
| 67 |
-
"correctness_passed": true
|
| 68 |
-
},
|
| 69 |
-
{
|
| 70 |
-
"batch_size": 128,
|
| 71 |
-
"programs": 4096,
|
| 72 |
-
"error_ratios": [
|
| 73 |
-
1.0002,
|
| 74 |
-
1.0004,
|
| 75 |
-
1.0001
|
| 76 |
-
],
|
| 77 |
-
"latency_ms": 0.0499,
|
| 78 |
-
"correctness_passed": true
|
| 79 |
-
},
|
| 80 |
-
{
|
| 81 |
-
"batch_size": 256,
|
| 82 |
-
"programs": 8192,
|
| 83 |
-
"error_ratios": [
|
| 84 |
-
1.0001,
|
| 85 |
-
1.0004,
|
| 86 |
-
1.0001
|
| 87 |
-
],
|
| 88 |
-
"latency_ms": 0.091,
|
| 89 |
-
"correctness_passed": true
|
| 90 |
-
},
|
| 91 |
-
{
|
| 92 |
-
"batch_size": 512,
|
| 93 |
-
"programs": 16384,
|
| 94 |
-
"error_ratios": [
|
| 95 |
-
1.0001,
|
| 96 |
-
1.0003,
|
| 97 |
-
1.0001
|
| 98 |
-
],
|
| 99 |
-
"latency_ms": 0.1706,
|
| 100 |
-
"correctness_passed": true
|
| 101 |
-
}
|
| 102 |
-
]
|
| 103 |
-
},
|
| 104 |
-
"environment": {
|
| 105 |
-
"python": "3.12.13",
|
| 106 |
-
"torch": "2.12.1+cu130",
|
| 107 |
-
"cutlass_dsl": "4.6.0",
|
| 108 |
-
"cuda_bindings": "13.3.1",
|
| 109 |
-
"kernels": "0.17.2",
|
| 110 |
-
"huggingface_hub": "2.2.0"
|
| 111 |
-
}
|
| 112 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
kda/README.md
CHANGED
|
@@ -86,9 +86,7 @@ be scanning the due flags.
|
|
| 86 |
Construct the pool and perform warmup before CUDA graph capture. Refill the
|
| 87 |
window buffers and counters before each new window. Calls use the current CUDA
|
| 88 |
stream on the pool's device. Keep the pool, index vector and `q0` buffers alive
|
| 89 |
-
while a graph using their addresses is alive.
|
| 90 |
-
mixed due heads, slot 0, reset/no-op behavior, a non-default stream and graph
|
| 91 |
-
replay with changed input contents.
|
| 92 |
|
| 93 |
## Accuracy and timing
|
| 94 |
|
|
@@ -129,7 +127,6 @@ python -m pip install torch==2.12.1 --index-url https://download.pytorch.org/whl
|
|
| 129 |
python -m pip install -r ../requirements.txt
|
| 130 |
python benchmark.py --impl ../build/torch-cuda/_kda.py # six original workloads
|
| 131 |
python benchmark.py # original TileLang baseline
|
| 132 |
-
python validate.py # package loader and deployment checks
|
| 133 |
```
|
| 134 |
|
| 135 |
TileLang and `nvcc` on `PATH` are needed only to benchmark the original baseline.
|
|
@@ -138,9 +135,8 @@ steady-state timings or perform first-use compilation inside graph capture.
|
|
| 138 |
|
| 139 |
|
| 140 |
The unchanged kernel passed all six original workloads (three input families
|
| 141 |
-
each) and deployment checks on B300 before consolidation
|
| 142 |
-
|
| 143 |
-
optimization-run measurement, not a new baseline comparison.
|
| 144 |
|
| 145 |
## Matched precision comparison
|
| 146 |
|
|
@@ -160,14 +156,14 @@ Reproduce from the repository root with `python compare_precision.py kda`.
|
|
| 160 |
|
| 161 |
The combined runtime is in `../build/torch-cuda/`; this component's unchanged
|
| 162 |
kernel is `_kda.py`. The files in this directory provide its original
|
| 163 |
-
benchmark, baseline, definition, workloads,
|
| 164 |
Shared dependencies and licenses are at the repository root.
|
| 165 |
|
| 166 |
The runtime kernel is byte-for-byte the final accepted submission
|
| 167 |
`e797d585430d45e1` (SHA-256 `2f635896b70d13ddfd76822d711f75c8c3abb4606f0ccd25dd04b583b95b7ec6`), from a 48-hour KDA experiment
|
| 168 |
with GPT 6.1 and Claude Opus 5.5. Request sources are pinned to public commit
|
| 169 |
`48f02e6ba0e98a1841ab6f0a2d99a90b2de1a326`. The release adds a public import wrapper, metadata
|
| 170 |
-
and
|
| 171 |
see `../LICENSE.txt` and `../NOTICE`.
|
| 172 |
|
| 173 |
## Available functions
|
|
|
|
| 86 |
Construct the pool and perform warmup before CUDA graph capture. Refill the
|
| 87 |
window buffers and counters before each new window. Calls use the current CUDA
|
| 88 |
stream on the pool's device. Keep the pool, index vector and `q0` buffers alive
|
| 89 |
+
while a graph using their addresses is alive.
|
|
|
|
|
|
|
| 90 |
|
| 91 |
## Accuracy and timing
|
| 92 |
|
|
|
|
| 127 |
python -m pip install -r ../requirements.txt
|
| 128 |
python benchmark.py --impl ../build/torch-cuda/_kda.py # six original workloads
|
| 129 |
python benchmark.py # original TileLang baseline
|
|
|
|
| 130 |
```
|
| 131 |
|
| 132 |
TileLang and `nvcc` on `PATH` are needed only to benchmark the original baseline.
|
|
|
|
| 135 |
|
| 136 |
|
| 137 |
The unchanged kernel passed all six original workloads (three input families
|
| 138 |
+
each) and deployment checks on B300 before consolidation. The speedup table
|
| 139 |
+
retains the paired optimization-run measurement, not a new baseline comparison.
|
|
|
|
| 140 |
|
| 141 |
## Matched precision comparison
|
| 142 |
|
|
|
|
| 156 |
|
| 157 |
The combined runtime is in `../build/torch-cuda/`; this component's unchanged
|
| 158 |
kernel is `_kda.py`. The files in this directory provide its original
|
| 159 |
+
benchmark, baseline, definition, workloads, benchmark and precision results.
|
| 160 |
Shared dependencies and licenses are at the repository root.
|
| 161 |
|
| 162 |
The runtime kernel is byte-for-byte the final accepted submission
|
| 163 |
`e797d585430d45e1` (SHA-256 `2f635896b70d13ddfd76822d711f75c8c3abb4606f0ccd25dd04b583b95b7ec6`), from a 48-hour KDA experiment
|
| 164 |
with GPT 6.1 and Claude Opus 5.5. Request sources are pinned to public commit
|
| 165 |
`48f02e6ba0e98a1841ab6f0a2d99a90b2de1a326`. The release adds a public import wrapper, metadata
|
| 166 |
+
and precision comparison tooling; it does not change the kernel algorithm. Apache-2.0;
|
| 167 |
see `../LICENSE.txt` and `../NOTICE`.
|
| 168 |
|
| 169 |
## Available functions
|
kda/provenance.json
DELETED
|
@@ -1,26 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"repo_id": "humanfia-lab/leapquant",
|
| 3 |
-
"version": 1,
|
| 4 |
-
"source_url": "https://github.com/NVlabs/kda/pull/10",
|
| 5 |
-
"source_commit": "48f02e6ba0e98a1841ab6f0a2d99a90b2de1a326",
|
| 6 |
-
"upstream_sha256": {
|
| 7 |
-
"baseline.py": "97b4e538674bd5a9310e41c3e7dfa19a9c76314f511acd5f5c29fdce95bfffb5",
|
| 8 |
-
"benchmark.py": "69e24658f3091c9fdd23873498dcbc6b243ce3816a7480435bd59cd0ef82751d",
|
| 9 |
-
"definition.json": "f25aac78d32ccc50401e2fb1a56d2c0a3ee27548b5d91f9e4d7e5a7c9d2dc323",
|
| 10 |
-
"workloads.jsonl": "ee5c437eeca601e5a83f5b3a4c07a6c11822e4e9438487a893ad6668a7075dfe"
|
| 11 |
-
},
|
| 12 |
-
"kernel_sha256": "2f635896b70d13ddfd76822d711f75c8c3abb4606f0ccd25dd04b583b95b7ec6",
|
| 13 |
-
"kernel_git_commit": "61f43860e7a487c9d9cbaccc043f270f2eb8693c",
|
| 14 |
-
"official_submission_id": "e797d585430d45e1",
|
| 15 |
-
"official_artifact_digest": "ac983001fb75c5ae2f66b557dcacf40332493853f741d812245839bb7110849d",
|
| 16 |
-
"official_benchmarks_sha": "3ac105d12ef108513555f8eac2a39203c0a18c5af3a8b6f77cb12751274ede75",
|
| 17 |
-
"official_evaluation_utc": "2026-10-24T12:38:34Z",
|
| 18 |
-
"experiment_hours": 48,
|
| 19 |
-
"models": [
|
| 20 |
-
"codex/gpt-6.1-sol:max",
|
| 21 |
-
"claude/claude-opus-5-5:max"
|
| 22 |
-
],
|
| 23 |
-
"packaging": "Unmodified accepted kernel; exposed as a module of the combined HF package.",
|
| 24 |
-
"component": "kda",
|
| 25 |
-
"kernel_path": "build/torch-cuda/_kda.py"
|
| 26 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
kda/validate.py
DELETED
|
@@ -1,126 +0,0 @@
|
|
| 1 |
-
# SPDX-License-Identifier: Apache-2.0
|
| 2 |
-
"""Validate the local HF package and caller-owned deployment contract on CUDA."""
|
| 3 |
-
from __future__ import annotations
|
| 4 |
-
|
| 5 |
-
import base64
|
| 6 |
-
import hashlib
|
| 7 |
-
import json
|
| 8 |
-
from pathlib import Path
|
| 9 |
-
|
| 10 |
-
import torch
|
| 11 |
-
from kernels import get_local_kernel
|
| 12 |
-
|
| 13 |
-
import benchmark as bench
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
def main() -> None:
|
| 17 |
-
here = Path(__file__).resolve().parent
|
| 18 |
-
runtime = here.parent / "build/torch-cuda"
|
| 19 |
-
metadata = json.loads((runtime / "metadata.json").read_text())
|
| 20 |
-
for filename, expected in metadata["digest"]["files"].items():
|
| 21 |
-
actual = base64.b64encode(hashlib.sha256((runtime / filename).read_bytes()).digest()).decode()
|
| 22 |
-
assert actual == expected, f"digest mismatch: {filename}"
|
| 23 |
-
op = get_local_kernel(here.parent, trust_remote_code=True).kda
|
| 24 |
-
assert all(hasattr(op, name) for name in ("Pool", "flush_inplace", "run"))
|
| 25 |
-
namespace = {}
|
| 26 |
-
exec(json.loads((here / "definition.json").read_text())["reference"], namespace)
|
| 27 |
-
reference = namespace["run"]
|
| 28 |
-
device = torch.device("cuda")
|
| 29 |
-
ns = 8
|
| 30 |
-
pool = op.Pool(ns, device=device)
|
| 31 |
-
is_kda = hasattr(pool, "gbuf")
|
| 32 |
-
idx = torch.tensor([0, 7, 2, 5, 1, 3], dtype=torch.int32, device=device)
|
| 33 |
-
counters = torch.where(torch.arange(ns * 32, device=device).reshape(ns, 32) % 3 == 0, 16, 7).int()
|
| 34 |
-
selected = torch.zeros(ns, dtype=torch.bool, device=device)
|
| 35 |
-
selected[idx.long()] = True
|
| 36 |
-
selected[0] = False
|
| 37 |
-
due = selected[:, None] & (counters == 16)
|
| 38 |
-
fields = ["codes", "s_k", "s_v", "uq", "kbuf", "ubuf", "w", "hcnt"] + (["gbuf", "pcum"] if is_kda else ["p"])
|
| 39 |
-
q0 = torch.empty(128, 4, device=device)
|
| 40 |
-
|
| 41 |
-
def restore(inputs):
|
| 42 |
-
pool.codes.copy_(inputs[0]); pool.s_k.copy_(inputs[1]); pool.s_v.copy_(inputs[2])
|
| 43 |
-
pool.uq[..., :128].copy_(inputs[3]); pool.uq[..., 128:].copy_(inputs[4])
|
| 44 |
-
pool.kbuf.copy_(inputs[5]); pool.ubuf.copy_(inputs[6])
|
| 45 |
-
if is_kda:
|
| 46 |
-
pool.gbuf.copy_(inputs[7]); pool.w.copy_(inputs[8]); pool.pcum.copy_(inputs[9])
|
| 47 |
-
else:
|
| 48 |
-
pool.w.copy_(inputs[7]); pool.p[..., 0].copy_(inputs[8])
|
| 49 |
-
pool.hcnt.copy_(counters)
|
| 50 |
-
q0.copy_(inputs[-1])
|
| 51 |
-
|
| 52 |
-
def snapshot():
|
| 53 |
-
return {name: getattr(pool, name).clone() for name in fields}
|
| 54 |
-
|
| 55 |
-
def verify(inputs, before):
|
| 56 |
-
ref = reference(*[tensor.clone() for tensor in inputs])
|
| 57 |
-
actual = (pool.codes, pool.s_k, pool.s_v, pool.uq[..., :128], pool.uq[..., 128:])
|
| 58 |
-
# Each selected (slot, head) is a separate checked head, never averaged across slots.
|
| 59 |
-
out = tuple(tensor[due].unsqueeze(0) for tensor in actual)
|
| 60 |
-
ref = tuple(tensor[due].unsqueeze(0) for tensor in ref)
|
| 61 |
-
exact = bench.exact_state(inputs)[due].unsqueeze(0)
|
| 62 |
-
passed, ratio = bench.judge(out, ref, exact)
|
| 63 |
-
assert passed, f"deployment accuracy failed: {ratio}"
|
| 64 |
-
for name in fields:
|
| 65 |
-
assert torch.equal(getattr(pool, name)[~due], before[name][~due]), f"non-due/slot-0 write: {name}"
|
| 66 |
-
for name in ("kbuf", "ubuf"):
|
| 67 |
-
assert torch.equal(getattr(pool, name), before[name]), f"update buffer changed: {name}"
|
| 68 |
-
# The serving contract leaves hcnt at L ("flushed, empty"); the decode
|
| 69 |
-
# step consumes that sentinel. Clearing it in flush would race scanners.
|
| 70 |
-
assert torch.equal(pool.hcnt, before["hcnt"]), "flush changed fill counters"
|
| 71 |
-
assert bool((pool.w[due] == 0).all())
|
| 72 |
-
if is_kda:
|
| 73 |
-
assert bool((pool.gbuf[due] == 1).all())
|
| 74 |
-
assert bool((pool.pcum[due] == 0).all())
|
| 75 |
-
else:
|
| 76 |
-
assert bool((pool.p[due] == 1).all())
|
| 77 |
-
return float(ratio)
|
| 78 |
-
|
| 79 |
-
stream = torch.cuda.Stream()
|
| 80 |
-
inputs = bench.structured_inputs(ns, device, seed=91)
|
| 81 |
-
restore(inputs)
|
| 82 |
-
torch.cuda.synchronize()
|
| 83 |
-
before = snapshot()
|
| 84 |
-
torch.cuda.synchronize()
|
| 85 |
-
with torch.cuda.stream(stream):
|
| 86 |
-
op.flush_inplace(pool, idx, q0)
|
| 87 |
-
stream.synchronize()
|
| 88 |
-
ratios = [verify(inputs, before)]
|
| 89 |
-
before = snapshot()
|
| 90 |
-
op.flush_inplace(pool, torch.empty(0, dtype=torch.int32, device=device), q0)
|
| 91 |
-
torch.cuda.synchronize()
|
| 92 |
-
assert all(torch.equal(getattr(pool, name), before[name]) for name in fields), "empty idx is not a no-op"
|
| 93 |
-
|
| 94 |
-
restore(inputs)
|
| 95 |
-
torch.cuda.synchronize()
|
| 96 |
-
graph = torch.cuda.CUDAGraph()
|
| 97 |
-
with torch.cuda.graph(graph, stream=stream):
|
| 98 |
-
op.flush_inplace(pool, idx, q0)
|
| 99 |
-
for seed in (92, 93):
|
| 100 |
-
inputs = bench.structured_inputs(ns, device, seed=seed)
|
| 101 |
-
# structured_inputs always uses the same q0; vary it explicitly here.
|
| 102 |
-
generator = torch.Generator(device=device).manual_seed(seed)
|
| 103 |
-
inputs = (*inputs[:-1], torch.randn(128, 4, device=device, generator=generator))
|
| 104 |
-
restore(inputs)
|
| 105 |
-
before = snapshot()
|
| 106 |
-
torch.cuda.synchronize()
|
| 107 |
-
graph.replay()
|
| 108 |
-
torch.cuda.synchronize()
|
| 109 |
-
ratios.append(verify(inputs, before))
|
| 110 |
-
# Simulate the caller advancing the counters so no head is due.
|
| 111 |
-
pool.hcnt[due] = 0
|
| 112 |
-
after = snapshot()
|
| 113 |
-
torch.cuda.synchronize()
|
| 114 |
-
graph.replay()
|
| 115 |
-
torch.cuda.synchronize()
|
| 116 |
-
assert all(torch.equal(getattr(pool, name), after[name]) for name in fields), "replay with no due heads is not a no-op"
|
| 117 |
-
print(json.dumps({
|
| 118 |
-
"status": "pass", "gpu": torch.cuda.get_device_name(),
|
| 119 |
-
"compute_capability": list(torch.cuda.get_device_capability()),
|
| 120 |
-
"checks": ["runtime digests", "HF get_local_kernel", "mixed due heads", "unselected slots", "reserved slot 0", "window resets", "fill counters unchanged", "empty idx", "non-default stream", "graph capture", "graph replay with changed inputs and q0", "no-op replay with no due heads"],
|
| 121 |
-
"deployment_worst_head_error_ratios": ratios,
|
| 122 |
-
}, indent=2))
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
if __name__ == "__main__":
|
| 126 |
-
main()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
kda/validation_results.json
DELETED
|
@@ -1,112 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"validated_at_utc": "2026-10-25T20:03:16.828102+00:00",
|
| 3 |
-
"kernel_sha256": "2f635896b70d13ddfd76822d711f75c8c3abb4606f0ccd25dd04b583b95b7ec6",
|
| 4 |
-
"deployment": {
|
| 5 |
-
"status": "pass",
|
| 6 |
-
"gpu": "NVIDIA B300 SXM6 AC",
|
| 7 |
-
"compute_capability": [
|
| 8 |
-
10,
|
| 9 |
-
3
|
| 10 |
-
],
|
| 11 |
-
"checks": [
|
| 12 |
-
"runtime digests",
|
| 13 |
-
"HF get_local_kernel",
|
| 14 |
-
"mixed due heads",
|
| 15 |
-
"unselected slots",
|
| 16 |
-
"reserved slot 0",
|
| 17 |
-
"window resets",
|
| 18 |
-
"fill counters unchanged",
|
| 19 |
-
"empty idx",
|
| 20 |
-
"non-default stream",
|
| 21 |
-
"graph capture",
|
| 22 |
-
"graph replay with changed inputs and q0",
|
| 23 |
-
"no-op replay with no due heads"
|
| 24 |
-
],
|
| 25 |
-
"deployment_worst_head_error_ratios": [
|
| 26 |
-
1.00270965549408,
|
| 27 |
-
1.0036139557099675,
|
| 28 |
-
1.0016564812747861
|
| 29 |
-
]
|
| 30 |
-
},
|
| 31 |
-
"original_benchmark": {
|
| 32 |
-
"workloads_passed": 6,
|
| 33 |
-
"input_families_per_workload": 3,
|
| 34 |
-
"thresholds_modified": false,
|
| 35 |
-
"workloads": [
|
| 36 |
-
{
|
| 37 |
-
"batch_size": 16,
|
| 38 |
-
"programs": 512,
|
| 39 |
-
"error_ratios": [
|
| 40 |
-
1.0004,
|
| 41 |
-
1.0006,
|
| 42 |
-
1.0011
|
| 43 |
-
],
|
| 44 |
-
"latency_ms": 0.0118,
|
| 45 |
-
"correctness_passed": true
|
| 46 |
-
},
|
| 47 |
-
{
|
| 48 |
-
"batch_size": 32,
|
| 49 |
-
"programs": 1024,
|
| 50 |
-
"error_ratios": [
|
| 51 |
-
1.0004,
|
| 52 |
-
1.0004,
|
| 53 |
-
1.0005
|
| 54 |
-
],
|
| 55 |
-
"latency_ms": 0.0181,
|
| 56 |
-
"correctness_passed": true
|
| 57 |
-
},
|
| 58 |
-
{
|
| 59 |
-
"batch_size": 64,
|
| 60 |
-
"programs": 2048,
|
| 61 |
-
"error_ratios": [
|
| 62 |
-
1.0004,
|
| 63 |
-
1.0003,
|
| 64 |
-
1.0007
|
| 65 |
-
],
|
| 66 |
-
"latency_ms": 0.0303,
|
| 67 |
-
"correctness_passed": true
|
| 68 |
-
},
|
| 69 |
-
{
|
| 70 |
-
"batch_size": 128,
|
| 71 |
-
"programs": 4096,
|
| 72 |
-
"error_ratios": [
|
| 73 |
-
1.0004,
|
| 74 |
-
1.0002,
|
| 75 |
-
1.0004
|
| 76 |
-
],
|
| 77 |
-
"latency_ms": 0.0546,
|
| 78 |
-
"correctness_passed": true
|
| 79 |
-
},
|
| 80 |
-
{
|
| 81 |
-
"batch_size": 256,
|
| 82 |
-
"programs": 8192,
|
| 83 |
-
"error_ratios": [
|
| 84 |
-
1.0002,
|
| 85 |
-
1.0001,
|
| 86 |
-
1.0002
|
| 87 |
-
],
|
| 88 |
-
"latency_ms": 0.1024,
|
| 89 |
-
"correctness_passed": true
|
| 90 |
-
},
|
| 91 |
-
{
|
| 92 |
-
"batch_size": 512,
|
| 93 |
-
"programs": 16384,
|
| 94 |
-
"error_ratios": [
|
| 95 |
-
1.0001,
|
| 96 |
-
1.0001,
|
| 97 |
-
1.0002
|
| 98 |
-
],
|
| 99 |
-
"latency_ms": 0.1934,
|
| 100 |
-
"correctness_passed": true
|
| 101 |
-
}
|
| 102 |
-
]
|
| 103 |
-
},
|
| 104 |
-
"environment": {
|
| 105 |
-
"python": "3.12.13",
|
| 106 |
-
"torch": "2.12.1+cu130",
|
| 107 |
-
"cutlass_dsl": "4.6.0",
|
| 108 |
-
"cuda_bindings": "13.3.1",
|
| 109 |
-
"kernels": "0.17.2",
|
| 110 |
-
"huggingface_hub": "2.2.0"
|
| 111 |
-
}
|
| 112 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|