| # flashrt/fp8-gemm |
|
|
| FlashRT native CUDA FP8 GEMV/GEMM kernels for low-latency transformer and |
| diffuser linear layers. |
|
|
| The block-128 scaled API supports Ada `sm_89` and Blackwell `sm_120a`. |
| The per-tensor API supports Blackwell `sm_110a` (Jetson AGX Thor) and |
| `sm_120a`. SM110 uses the production FlashRT Sq/T1/Wide CUTLASS family and has |
| been swept across PI0.5, GROOT, Cosmos Edge, and LingBot VLA projection shapes. |
|
|
| ## Functions |
|
|
| - `fp8_linear_bf16(input, weight, alpha=1.0, out=None, variant=0)` |
| - `fp8_linear_residual_bf16(input, weight, residual, alpha=1.0, variant=0)` |
| - `fp8_linear_bias_bf16(input, weight, bias, alpha=1.0, out=None)` |
| - `fp8_linear_bias_residual_bf16(input, weight, bias, residual, alpha=1.0)` |
| - `fp8_linear_bias_gelu_bf16(input, weight, bias, alpha=1.0, out=None)` |
| - `fp8_blockwise_linear_bf16(input, weight, input_scale, weight_scale, out=None)` |
| - `select_fp8_linear_tile(m, n, k, variant=0)` |
|
|
| On SM110, keep `variant=0` for the tuned public dispatcher. Variants `1`, `2`, |
| and `3` force Sq, T1, and Wide respectively for diagnostics. |
|
|
| SM110 also provides BF16 bias, in-place bias+residual, and tanh-GELU+bias |
| epilogues for SigLIP-style projection and MLP shapes. The validated large-M |
| plain GEMM band is `M=65..1024`. |
|
|
| See the repository README for shape contracts, validation status, and examples. |
|
|
