File size: 3,758 Bytes
6f8f573 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | # Claim 4: Kernel performance
---
<!-- trackio-cell
{"type": "markdown", "id": "cell_c26acd888a0c", "created_at": "2026-07-17T07:05:33+00:00", "title": "Partly verified and one supplied attribution refuted. Released A100 INT8 Qwen2.…"}
-->
Partly verified and one supplied attribution refuted. Released A100 INT8 Qwen2.5-7B data gives 1.41x and 1.42x at M=8192/16384. The paper attributes the 1.35-1.37x FP8 range to RTX 4090, not H100; released H100 FP8 Qwen2.5-7B values at those M are 1.08x and 1.19x.
---
<!-- trackio-cell
{"type": "code", "id": "cell_2b0a6c889758", "created_at": "2026-07-17T07:06:48+00:00", "title": "Run: orx (exit 1)", "command": ["orx", "exp", "run", "eeb01f38-ca0c-49a6-baaf-5b68d91f7296", "--backend", "hf", "--flavor", "a100-large", "--timeout", "1h"], "exit_code": 1, "duration_s": 6.658}
-->
````bash
$ orx exp run eeb01f38-ca0c-49a6-baaf-5b68d91f7296 --backend hf --flavor a100-large --timeout 1h
````
exit 1 · 6.7s
````output
Hugging Face job submit failed (403): {"error":"You don't have the required permissions to complete this action on namespace vimarsh, missing permissions: job.write"}
````
---
<!-- trackio-cell
{"type": "code", "id": "cell_af7f0b303f64", "created_at": "2026-07-17T07:07:26+00:00", "title": "Run: orx (exit 0)", "command": ["orx", "exp", "run", "eeb01f38-ca0c-49a6-baaf-5b68d91f7296", "--backend", "local"], "exit_code": 0, "duration_s": 8.786}
-->
````bash
$ orx exp run eeb01f38-ca0c-49a6-baaf-5b68d91f7296 --backend local
````
exit 0 · 8.8s
````output
✓ Local run started.
dir /nas_hcfc/vimarsh/hf-icml-agent-repo-2026/.orx/openresearch/local-runs/d50c2df5-6b18-4ba4-a73f-e6e17a12bac2
run d50c2df5-6b18-4ba4-a73f-e6e17a12bac2
Follow it with `orx exp wait eeb01f38-ca0c-49a6-baaf-5b68d91f7296` or `orx logs d50c2df5-6b18-4ba4-a73f-e6e17a12bac2`.
````
---
<!-- trackio-cell
{"type": "markdown", "id": "cell_c165074d620a", "created_at": "2026-07-17T07:09:39+00:00", "title": "HF A100 submission was attempted but rejected before scheduling with HTTP 403:…"}
-->
HF A100 submission was attempted but rejected before scheduling with HTTP 403: token lacks job.write on namespace vimarsh. The first supervised local retry failed during PyTorch CUDA initialization because two devices were discovered before the run exposed one device; descendant retry isolates the free physical GPU 0 before Python import.
---
<!-- trackio-cell
{"type": "code", "id": "cell_eecf46361d8a", "created_at": "2026-07-17T07:10:52+00:00", "title": "Run: orx (exit 0)", "command": ["orx", "exp", "run", "3502d8e3-0d22-4328-8eeb-c21e0d6620a3", "--backend", "local"], "exit_code": 0, "duration_s": 11.546}
-->
````bash
$ orx exp run 3502d8e3-0d22-4328-8eeb-c21e0d6620a3 --backend local
````
exit 0 · 11.5s
````output
✓ Local run started.
dir /nas_hcfc/vimarsh/hf-icml-agent-repo-2026/.orx/openresearch/local-runs/b807d368-b860-47f2-a974-6e74c77d12de
run b807d368-b860-47f2-a974-6e74c77d12de
Follow it with `orx exp wait 3502d8e3-0d22-4328-8eeb-c21e0d6620a3` or `orx logs b807d368-b860-47f2-a974-6e74c77d12de`.
````
---
<!-- trackio-cell
{"type": "markdown", "id": "cell_602b42206eed", "created_at": "2026-07-17T07:12:57+00:00", "title": "Scaled local CUDA proxy (run b807d368-b860-47f2-a974-6e74c77d12de): RTX 4000 Ad…"}
-->
Scaled local CUDA proxy (run b807d368-b860-47f2-a974-6e74c77d12de): RTX 4000 Ada 20GB, PyTorch 2.6/CUDA 12.4, INT8 semi-structured GEMM, N=K=4096 and 6:8-equivalent K_slid=6144. M=64: 0.6488x; M=2048: 0.7327x. Speedup did not reproduce. This is a real sparse-Tensor-Core proxy but excludes the paper's cuSPARSELt 0.8.1 tuning, fused activation lifting, and A100/H100 hardware; the authors' sm89 binary cannot load because libcusparseLt.so.0 is absent.
|