Kernels:

---

flashrt

/

fp8-gemm

like 2

Follow

flashrt 10

# flashrt/fp8-gemm

FlashRT native CUDA FP8 GEMV/GEMM kernels for low-latency transformer and
diffuser linear layers.

The block-128 scaled API supports Ada `sm_89` and Blackwell `sm_120a`.
The per-tensor API supports Blackwell `sm_110a` (Jetson AGX Thor) and
`sm_120a`. SM110 uses the production FlashRT Sq/T1/Wide CUTLASS family and has
been swept across PI0.5, GROOT, Cosmos Edge, and LingBot VLA projection shapes.

## Functions

- `fp8_linear_bf16(input, weight, alpha=1.0, out=None, variant=0)`
- `fp8_linear_residual_bf16(input, weight, residual, alpha=1.0, variant=0)`
- `fp8_linear_bias_bf16(input, weight, bias, alpha=1.0, out=None)`
- `fp8_linear_bias_residual_bf16(input, weight, bias, residual, alpha=1.0)`
- `fp8_linear_bias_gelu_bf16(input, weight, bias, alpha=1.0, out=None)`
- `fp8_blockwise_linear_bf16(input, weight, input_scale, weight_scale, out=None)`
- `select_fp8_linear_tile(m, n, k, variant=0)`

On SM110, keep `variant=0` for the tuned public dispatcher. Variants `1`, `2`,
and `3` force Sq, T1, and Wide respectively for diagnostics.

SM110 also provides BF16 bias, in-place bias+residual, and tanh-GELU+bias
epilogues for SigLIP-style projection and MLP shapes. The validated large-M
plain GEMM band is `M=65..1024`.

See the repository README for shape contracts, validation status, and examples.