pages/claim-4-kernel-performance/page.md

# Claim 4: Kernel performance


---
<!-- trackio-cell
{"type": "markdown", "id": "cell_c26acd888a0c", "created_at": "2026-07-17T07:05:33+00:00", "title": "Partly verified and one supplied attribution refuted. Released A100 INT8 Qwen2.…"}
-->
Partly verified and one supplied attribution refuted. Released A100 INT8 Qwen2.5-7B data gives 1.41x and 1.42x at M=8192/16384. The paper attributes the 1.35-1.37x FP8 range to RTX 4090, not H100; released H100 FP8 Qwen2.5-7B values at those M are 1.08x and 1.19x.


---
<!-- trackio-cell
{"type": "code", "id": "cell_2b0a6c889758", "created_at": "2026-07-17T07:06:48+00:00", "title": "Run: orx (exit 1)", "command": ["orx", "exp", "run", "eeb01f38-ca0c-49a6-baaf-5b68d91f7296", "--backend", "hf", "--flavor", "a100-large", "--timeout", "1h"], "exit_code": 1, "duration_s": 6.658}
-->
````bash
$ orx exp run eeb01f38-ca0c-49a6-baaf-5b68d91f7296 --backend hf --flavor a100-large --timeout 1h
````

exit 1 · 6.7s


````output
Hugging Face job submit failed (403): {"error":"You don't have the required permissions to complete this action on namespace vimarsh, missing permissions: job.write"}

````


---
<!-- trackio-cell
{"type": "code", "id": "cell_af7f0b303f64", "created_at": "2026-07-17T07:07:26+00:00", "title": "Run: orx (exit 0)", "command": ["orx", "exp", "run", "eeb01f38-ca0c-49a6-baaf-5b68d91f7296", "--backend", "local"], "exit_code": 0, "duration_s": 8.786}
-->
````bash
$ orx exp run eeb01f38-ca0c-49a6-baaf-5b68d91f7296 --backend local
````

exit 0 · 8.8s


````output
✓ Local run started.
  dir  /nas_hcfc/vimarsh/hf-icml-agent-repo-2026/.orx/openresearch/local-runs/d50c2df5-6b18-4ba4-a73f-e6e17a12bac2
  run  d50c2df5-6b18-4ba4-a73f-e6e17a12bac2
  Follow it with `orx exp wait eeb01f38-ca0c-49a6-baaf-5b68d91f7296` or `orx logs d50c2df5-6b18-4ba4-a73f-e6e17a12bac2`.

````


---
<!-- trackio-cell
{"type": "markdown", "id": "cell_c165074d620a", "created_at": "2026-07-17T07:09:39+00:00", "title": "HF A100 submission was attempted but rejected before scheduling with HTTP 403:…"}
-->
HF A100 submission was attempted but rejected before scheduling with HTTP 403: token lacks job.write on namespace vimarsh. The first supervised local retry failed during PyTorch CUDA initialization because two devices were discovered before the run exposed one device; descendant retry isolates the free physical GPU 0 before Python import.


---
<!-- trackio-cell
{"type": "code", "id": "cell_eecf46361d8a", "created_at": "2026-07-17T07:10:52+00:00", "title": "Run: orx (exit 0)", "command": ["orx", "exp", "run", "3502d8e3-0d22-4328-8eeb-c21e0d6620a3", "--backend", "local"], "exit_code": 0, "duration_s": 11.546}
-->
````bash
$ orx exp run 3502d8e3-0d22-4328-8eeb-c21e0d6620a3 --backend local
````

exit 0 · 11.5s


````output
✓ Local run started.
  dir  /nas_hcfc/vimarsh/hf-icml-agent-repo-2026/.orx/openresearch/local-runs/b807d368-b860-47f2-a974-6e74c77d12de
  run  b807d368-b860-47f2-a974-6e74c77d12de
  Follow it with `orx exp wait 3502d8e3-0d22-4328-8eeb-c21e0d6620a3` or `orx logs b807d368-b860-47f2-a974-6e74c77d12de`.

````


---
<!-- trackio-cell
{"type": "markdown", "id": "cell_602b42206eed", "created_at": "2026-07-17T07:12:57+00:00", "title": "Scaled local CUDA proxy (run b807d368-b860-47f2-a974-6e74c77d12de): RTX 4000 Ad…"}
-->
Scaled local CUDA proxy (run b807d368-b860-47f2-a974-6e74c77d12de): RTX 4000 Ada 20GB, PyTorch 2.6/CUDA 12.4, INT8 semi-structured GEMM, N=K=4096 and 6:8-equivalent K_slid=6144. M=64: 0.6488x; M=2048: 0.7327x. Speedup did not reproduce. This is a real sparse-Tensor-Core proxy but excludes the paper's cuSPARSELt 0.8.1 tuning, fused activation lifting, and A100/H100 hardware; the authors' sm89 binary cannot load because libcusparseLt.so.0 is absent.