README.md

metadata

title: cuda-kernels-live
emoji: ⚡
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
suggested_hardware: zero-a10g

CUDA Kernels — live on ZeroGPU

Runs real, JIT-compiled CUDA kernels on a free Hugging Face ZeroGPU allocation:

1. Sliding-window attention — a from-scratch kernel (online softmax, Longformer-style local window), compiled at request time with torch.utils.cpp_extension.load_inline, benchmarked against dense masked PyTorch attention.
2. Kernel fusion compiler — y = gelu(x*w + b) fused from 3 elementwise ops into 1 generated CUDA kernel by fusion_compiler, compiled and run, benchmarked against the naive 3-kernel-launch version.

Companion Spaces/repos:

• long-context-attention-kernels — the production tiled kernel this demo's simplified version is based on
• cuda-fusion-compiler — the fusion compiler used in tab 2

Why ZeroGPU

ZeroGPU attaches a GPU to the process only for the duration of a function decorated with @spaces.GPU, so all CUDA work here (JIT compile + kernel launch + benchmark) happens inside those functions. First call per session compiles the kernel (a few seconds); later calls reuse the on-disk build cache.

Local run (needs your own CUDA GPU)

pip install -r requirements.txt
python app.py

Locally spaces.GPU is a no-op decorator (the spaces package falls back gracefully off of a ZeroGPU Space), so this also runs on any machine with a CUDA-capable GPU and the toolchain installed.