README.md

---
title: cuda-kernels-live
emoji: ⚡
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
suggested_hardware: zero-a10g
---

# CUDA Kernels — live on ZeroGPU

Runs real, JIT-compiled CUDA kernels on a free Hugging Face ZeroGPU
allocation:

1. **Sliding-window attention** — a from-scratch kernel (online softmax,
   Longformer-style local window), compiled at request time with
   `torch.utils.cpp_extension.load_inline`, benchmarked against dense
   masked PyTorch attention.
2. **Kernel fusion compiler** — `y = gelu(x*w + b)` fused from 3 elementwise
   ops into 1 generated CUDA kernel by
   [`fusion_compiler`](../cuda-fusion-compiler), compiled and run, benchmarked
   against the naive 3-kernel-launch version.

Companion Spaces/repos:
- [long-context-attention-kernels](https://github.com/data-geek-astronomy/long-context-attention-kernels) — the production tiled kernel this demo's simplified version is based on
- [cuda-fusion-compiler](https://github.com/data-geek-astronomy/cuda-fusion-compiler) — the fusion compiler used in tab 2

## Why ZeroGPU

ZeroGPU attaches a GPU to the process only for the duration of a function
decorated with `@spaces.GPU`, so all CUDA work here (JIT compile + kernel
launch + benchmark) happens inside those functions. First call per session
compiles the kernel (a few seconds); later calls reuse the on-disk build
cache.

## Local run (needs your own CUDA GPU)

```bash
pip install -r requirements.txt
python app.py
```

Locally `spaces.GPU` is a no-op decorator (the `spaces` package falls back
gracefully off of a ZeroGPU Space), so this also runs on any machine with a
CUDA-capable GPU and the toolchain installed.