Promote latest kernel artifacts to main

README.md

@@ -1,9 +1,116 @@
-# flashrt/fp4-gemm
+# fp4-gemm
 
-This repository is a compatibility mirror for older `kernels` clients
-that resolve repositories through the default Hugging Face model repo API.
+FlashRT native Blackwell NVFP4 A4W4 GEMM kernels.
 
-Canonical Kernel Hub repo: https://huggingface.co/kernels/flashrt/fp4-gemm
+This package consumes packed FP4 E2M1 tensors plus CUTLASS Sm1xx SFA/SFB scale
+buffers and produces BF16 output. It is designed to pair with
+`flashrt/fp4-fused-ops` and other static low-bit transformer/diffuser runtime
+paths.
 
-Do not edit this mirror by hand. It is generated from the Kernel Hub
-`vN` branches and contains the same `build/**` artifacts.
+## Available Functions
+
+- `sfa_size_bytes(rows, dim)`
+- `quantize_fp4_sfa_fp16(x, packed=None, sfa=None, is_sfb=False)`
+- `quantize_fp4_sfa_bf16(x, packed=None, sfa=None, is_sfb=False)`
+- `dequantize_fp4_sfa_fp16(packed, sfa, out=None, is_sfb=False)`
+- `nvfp4_gemm_bf16(a_packed, b_packed, sfa, sfb, alpha=1.0, out=None, variant=-1)`
+- `nvfp4_gemm_bias_bf16(a_packed, b_packed, sfa, sfb, bias, out=None)`
+- `nvfp4_gemm_bias_residual_bf16(a_packed, b_packed, sfa, sfb, bias, residual, out=None)`
+- `nvfp4_gemm_residual_bf16(a_packed, b_packed, sfa, sfb, residual, alpha=1.0, out=None)`
+- `nvfp4_gemm_bias_gelu_bf16(a_packed, b_packed, sfa, sfb, bias, alpha=1.0, out=None)`
+- `nvfp4_gemm_bias_gelu_nvfp4(a_packed, b_packed, sfa, sfb, bias, alpha=1.0, out_packed=None, out_sfa=None)`
+- `nvfp4_gemm_streamk_bf16(a_packed, b_packed, sfa, sfb, alpha=1.0, out=None)`
+- `nvfp4_gemm_streamk_bias_bf16(a_packed, b_packed, sfa, sfb, bias, alpha=1.0, out=None)`
+- `fp4_w4a16_linear_bf16(...)` is retained as a compatibility alias
+
+## Tensor Contract
+
+- `a_packed`: `torch.uint8`, shape `(M, K / 2)`.
+- `b_packed`: `torch.uint8`, shape `(N, K / 2)`.
+- `sfa`: `torch.uint8`, CUTLASS SFA layout for `(M, K)`.
+- `sfb`: `torch.uint8`, CUTLASS SFB layout for `(N, K)`.
+- output: `torch.bfloat16`, shape `(M, N)`.
+- `K` must be divisible by 16.
+- Targets: Blackwell `sm_110a` (Jetson AGX Thor, CUDA 13+) and `sm_120a`
+  (RTX Blackwell, CUDA 12.8+).
+
+`variant` selects the CUTLASS schedule:
+
+- `-1`: architecture-aware auto-dispatch (public default).
+- `0`: default `<128,128,256>` cooperative schedule.
+- `1`: widen `<128,256,128>` schedule, intended for very large `N`.
+- `2`: pingpong schedule for A/B testing shape-specific wins.
+
+The canonical linear API and FP4/SFA quantize/dequantize helpers are available
+on both SM110 and SM120. SM110 additionally provides the GROOT N1.7 production
+epilogues `nvfp4_gemm_bias_bf16`, `nvfp4_gemm_bias_residual_bf16`, and
+`nvfp4_gemm_bias_gelu_nvfp4`. The latter emits packed FP4 plus CUTLASS SFA so
+the following projection can consume it without a BF16 materialization and a
+standalone quantization launch. Stream-K and the older BF16 GELU epilogue keep
+their existing SM120 dispatch and reject unsupported architectures explicitly.
+
+The SM110 release gate includes the production `(M,N,K)` shapes
+`(41,4608,1536)`, `(41,6144,1536)`, and `(41,1536,6144)`, plus the legacy
+`M=51` compatibility row. The kernels are the native sources used by FlashRT's
+GROOT N1.7 Thor NVFP4 pipeline.
+
+## Minimal Usage
+
+```python
+from kernels import get_kernel
+import torch
+
+ops = get_kernel("flashrt/fp4-gemm", version=1, trust_remote_code=True)
+
+x = torch.randn((32, 256), device="cuda", dtype=torch.float16)
+w = torch.randn((512, 256), device="cuda", dtype=torch.float16)
+
+a_packed, sfa = ops.quantize_fp4_sfa_fp16(x, is_sfb=False)
+b_packed, sfb = ops.quantize_fp4_sfa_fp16(w, is_sfb=True)
+
+y = ops.nvfp4_gemm_bf16(a_packed, b_packed, sfa, sfb, alpha=1.0)
+```
+
+For BF16 model activations, use the direct producer so the hot path does not
+materialize an intermediate FP16 tensor:
+
+```python
+x_bf16 = torch.randn((1, 5120), device="cuda", dtype=torch.bfloat16)
+a_packed, sfa = ops.quantize_fp4_sfa_bf16(x_bf16)
+```
+
+The BF16 entry writes the same E2M1 bytes and CUTLASS SFA/SFB layout as
+`quantize_fp4_sfa_fp16(x_bf16.to(torch.float16))` for finite FP16-range
+inputs. It is an additive API; the existing FP16 producer remains unchanged.
+
+The quantize/dequantize helpers are included for examples and validation. A
+production runtime should keep weights prepacked and should avoid quantizing in
+the hot path unless that producer kernel is part of the intended low-bit block.
+
+Use the bias/GELU and residual variants to avoid returning to BF16
+elementwise code between low-bit GEMMs. Stream-K variants are selected only
+for the validated large down-projection shapes; unsupported shapes reject
+rather than silently selecting a losing schedule.
+
+## Validation
+
+```bash
+python fp4-gemm/tests/test_fp4_gemm.py --backend source --mode full
+python fp4-gemm/tests/test_fp4_gemm.py --backend installed --mode full \
+  --artifact fp4-gemm/build/torch211-cxx11-cu128-x86_64-linux
+python fp4-gemm/benchmarks/benchmark.py --backend installed --mode headline \
+  --artifact fp4-gemm/build/torch211-cxx11-cu128-x86_64-linux
+
+# Thor model-shape gate
+python fp4-gemm/tests/test_fp4_gemm.py --backend installed \
+  --mode thor-models \
+  --artifact fp4-gemm/build/torch211-cxx11-cu130-aarch64-linux
+```
+
+The correctness reference dequantizes the same FP4/SFA and FP4/SFB inputs used
+by the kernel, then computes the PyTorch GEMM reference from those dequantized
+low-bit values.
+
+The producer gate also checks the BF16 direct entry byte-for-byte against the
+established FP16 compatibility chain at decode widths 5120, 6144 and 17408,
+plus multi-row activation and SFB layouts.

benchmarks/RESULTS.md

@@ -55,17 +55,3 @@ The direct entry is byte-exact against the package's established
 BF16-to-FP16 plus FP16-producer contract. The native timing is reported as a
 performance reference only because that producer uses a distinct quantization
 strategy.
-
-## NVIDIA Thor GROOT N1.7 artifact
-
-The SM110 additions from FlashRT
-`24df793f4fa2d50780aea03b644208c6e0cb4162` were rebuilt on NVIDIA Thor with
-PyTorch 2.13.0+cu130 as `torch213-cxx11-cu130-aarch64-linux`. The installed
-artifact passed 23/23 checks; BF16-to-FP4 output was exact and the fullgraph
-compile path had `max_abs=0`.
-
-The FP4 quantizer Tensor wrapper/raw registered-op measurement was
-`5.5812/5.0712 us` in direct mode. This eager delta includes Python-side
-allocation and dispatch. With caller-owned buffers under CUDA Graph, the
-measurement was `3.2988/3.3003 us` (`0.9995x`), which is the production GROOT
-hot-path contract.

build/torch212-cxx11-cu130-x86_64-linux/__init__.py

@@ -19,7 +19,8 @@ def sfa_size_bytes(rows: int, dim: int) -> int:
 def _alloc_fp4(rows: int, dim: int, device: torch.device | str):
     return (
         torch.empty((rows, dim // 2), device=device, dtype=torch.uint8),
-        torch.empty((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
+        # Tile-layout padding entries are not written by every quantizer.
+        torch.zeros((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
     )
 
 
@@ -36,6 +37,26 @@ def _linear_fake(
     return None
 
 
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_fp16"))
+def _linear_fp16_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, variant: int = -1) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_geglu_nvfp4_fp16"))
+def _geglu_fp4_fake(a, b, sfa, sfb, scratch, out_packed, out_sfa, skinny: bool = False) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_gelu_nvfp4_fp16"))
+def _bias_gelu_fp4_fp16_fake(a, b, sfa, sfb, bias, out_packed, out_sfa) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_residual_fp16"))
+def _bias_residual_fp16_fake(a, b, sfa, sfb, bias, residual, out) -> None:
+    return None
+
+
 @torch.library.register_fake(add_op_namespace_prefix("fp4_w4a4_gemv_warpsplit_bf16"))
 def _gemv_warpsplit_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, warps: int = 4, stages: int = 4) -> None:
     if a_packed.shape[0] != 1:
@@ -162,6 +183,100 @@ def nvfp4_gemm_bf16(
     return out
 
 
+def nvfp4_gemm_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    alpha: float = 1.0,
+    out: torch.Tensor | None = None,
+    variant: int = -1,
+) -> torch.Tensor:
+    """SM110 native NVFP4 GEMM with FP16 output."""
+    if out is None:
+        out = torch.empty(
+            (a_packed.shape[0], b_packed.shape[0]),
+            device=a_packed.device,
+            dtype=torch.float16,
+        )
+    ops.nvfp4_gemm_fp16(
+        a_packed, b_packed, sfa, sfb, out, float(alpha), int(variant)
+    )
+    return out
+
+
+def nvfp4_gemm_geglu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_interleaved_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    *,
+    skinny: bool = False,
+    scratch: torch.Tensor | None = None,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """GEMM with fused GeGLU and compact NVFP4 output on SM110.
+
+    ``b_interleaved_packed`` stores gate/up rows pairwise, so its first
+    dimension is twice the logical hidden width.
+    """
+    m, n_twice = a_packed.shape[0], b_interleaved_packed.shape[0]
+    hidden = n_twice // 2
+    if scratch is None:
+        scratch = torch.empty((m, hidden), device=a_packed.device, dtype=torch.uint8)
+    if out_packed is None:
+        out_packed = torch.empty((m, hidden // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, hidden),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_geglu_nvfp4_fp16(
+        a_packed, b_interleaved_packed, sfa, sfb, scratch,
+        out_packed, out_sfa, bool(skinny)
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_gelu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    *,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """FP16-bias GEMM with fused GELU and NVFP4 output on SM110."""
+    m, n = a_packed.shape[0], b_packed.shape[0]
+    if out_packed is None:
+        out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_bias_gelu_nvfp4_fp16(
+        a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_residual_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    residual: torch.Tensor,
+    *,
+    out: torch.Tensor | None = None,
+) -> torch.Tensor:
+    """FP16-output GEMM with fused FP16 bias and residual on SM110."""
+    if out is None:
+        out = torch.empty_like(residual)
+    ops.nvfp4_gemm_bias_residual_fp16(
+        a_packed, b_packed, sfa, sfb, bias, residual, out
+    )
+    return out
+
+
 def fp4_w4a4_gemv_warpsplit_bf16(
     a_packed: torch.Tensor,
     b_packed: torch.Tensor,
@@ -292,7 +407,7 @@ def nvfp4_gemm_bias_gelu_nvfp4(
     if out_packed is None:
         out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
     if out_sfa is None:
-        out_sfa = torch.empty((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
     ops.nvfp4_gemm_bias_gelu_nvfp4(
         a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa, float(alpha)
     )
@@ -345,6 +460,10 @@ __all__ = [
     "fp4_w4a16_linear_bf16",
     "fp4_w4a4_gemv_warpsplit_bf16",
     "nvfp4_gemm_bf16",
+    "nvfp4_gemm_fp16",
+    "nvfp4_gemm_geglu_nvfp4_fp16",
+    "nvfp4_gemm_bias_gelu_nvfp4_fp16",
+    "nvfp4_gemm_bias_residual_fp16",
     "nvfp4_gemm_bias_bf16",
     "nvfp4_gemm_bias_gelu_bf16",
     "nvfp4_gemm_bias_gelu_nvfp4",

build/torch212-cxx11-cu130-x86_64-linux/{_fp4_gemm_cuda_8a66d8b.abi3.so → _fp4_gemm_cuda_55c4885.abi3.so}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:26fce4c1404d996b8e9ab774125c787e47a568f79da35f734949bff0177a173a
-size 2848128
+oid sha256:c55a4f6233f699e6d2e69205e0f4c5e7753d0dc216129bf75e4477d675daee8e
+size 6350504

build/torch212-cxx11-cu130-x86_64-linux/_ops.py

@@ -1,9 +1,9 @@
 import torch
-from . import _fp4_gemm_cuda_8a66d8b
-ops = torch.ops._fp4_gemm_cuda_8a66d8b
+from . import _fp4_gemm_cuda_55c4885
+ops = torch.ops._fp4_gemm_cuda_55c4885
 
 def add_op_namespace_prefix(op_name: str):
     """
     Prefix op by namespace.
     """
-    return f"_fp4_gemm_cuda_8a66d8b::{op_name}"
+    return f"_fp4_gemm_cuda_55c4885::{op_name}"

build/torch212-cxx11-cu130-x86_64-linux/metadata.json

@@ -1,6 +1,6 @@
 {
   "name": "fp4-gemm",
-  "id": "_fp4_gemm_cuda_8a66d8b",
+  "id": "_fp4_gemm_cuda_55c4885",
   "version": 1,
   "license": "Apache-2.0",
   "python-depends": [],
@@ -14,19 +14,19 @@
   "digest": {
     "algorithm": "sha256",
     "files": {
-      "__init__.py": "+Kk/VNnIWwe9nszQYvL2qtU3DpKxbqHVfklg5WtVuB4=",
-      "_fp4_gemm_cuda_8a66d8b.abi3.so": "JvzkwUBNmWuOmrd0Elx4fkelaPedo19zSUm/8Bd6Fzo=",
-      "_ops.py": "mZSHUC0K9o9glzvGmRSdtPppOq4bmIaxNEjibkaWTk0="
+      "__init__.py": "4o968SKYWj8r1Ly7FqSPFK/AYdu4G4bXuVv1UcvYjk8=",
+      "_fp4_gemm_cuda_55c4885.abi3.so": "xVpPYjP2mebS5pIF4PTF53U9DcIWEpv3XkR31nXa7o4=",
+      "_ops.py": "IVtq+3pHXwuoCbgYGw3JP7/nKr3GEZqn8D97JJjf7ZE="
     }
   },
   "provenance": {
     "kernel-builder": {
       "version": "0.17.0-dev0",
-      "sha": "870e825d881664e39f9287a27a74ef63ff3c545e",
+      "sha": "81f55ea30fd8f819dcf93a3c934dd584c895bd2f",
       "dirty": false
     },
     "kernel": {
-      "sha": "8a66d8b7f79a6fde86aa6929db2b60ab097e2d55",
+      "sha": "55c4885251068f195418bcf9ae541f4b757a6ea0",
       "dirty": false
     }
   }

build/torch212-cxx11-cu132-x86_64-linux/__init__.py

@@ -19,7 +19,8 @@ def sfa_size_bytes(rows: int, dim: int) -> int:
 def _alloc_fp4(rows: int, dim: int, device: torch.device | str):
     return (
         torch.empty((rows, dim // 2), device=device, dtype=torch.uint8),
-        torch.empty((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
+        # Tile-layout padding entries are not written by every quantizer.
+        torch.zeros((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
     )
 
 
@@ -36,6 +37,26 @@ def _linear_fake(
     return None
 
 
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_fp16"))
+def _linear_fp16_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, variant: int = -1) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_geglu_nvfp4_fp16"))
+def _geglu_fp4_fake(a, b, sfa, sfb, scratch, out_packed, out_sfa, skinny: bool = False) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_gelu_nvfp4_fp16"))
+def _bias_gelu_fp4_fp16_fake(a, b, sfa, sfb, bias, out_packed, out_sfa) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_residual_fp16"))
+def _bias_residual_fp16_fake(a, b, sfa, sfb, bias, residual, out) -> None:
+    return None
+
+
 @torch.library.register_fake(add_op_namespace_prefix("fp4_w4a4_gemv_warpsplit_bf16"))
 def _gemv_warpsplit_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, warps: int = 4, stages: int = 4) -> None:
     if a_packed.shape[0] != 1:
@@ -162,6 +183,100 @@ def nvfp4_gemm_bf16(
     return out
 
 
+def nvfp4_gemm_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    alpha: float = 1.0,
+    out: torch.Tensor | None = None,
+    variant: int = -1,
+) -> torch.Tensor:
+    """SM110 native NVFP4 GEMM with FP16 output."""
+    if out is None:
+        out = torch.empty(
+            (a_packed.shape[0], b_packed.shape[0]),
+            device=a_packed.device,
+            dtype=torch.float16,
+        )
+    ops.nvfp4_gemm_fp16(
+        a_packed, b_packed, sfa, sfb, out, float(alpha), int(variant)
+    )
+    return out
+
+
+def nvfp4_gemm_geglu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_interleaved_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    *,
+    skinny: bool = False,
+    scratch: torch.Tensor | None = None,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """GEMM with fused GeGLU and compact NVFP4 output on SM110.
+
+    ``b_interleaved_packed`` stores gate/up rows pairwise, so its first
+    dimension is twice the logical hidden width.
+    """
+    m, n_twice = a_packed.shape[0], b_interleaved_packed.shape[0]
+    hidden = n_twice // 2
+    if scratch is None:
+        scratch = torch.empty((m, hidden), device=a_packed.device, dtype=torch.uint8)
+    if out_packed is None:
+        out_packed = torch.empty((m, hidden // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, hidden),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_geglu_nvfp4_fp16(
+        a_packed, b_interleaved_packed, sfa, sfb, scratch,
+        out_packed, out_sfa, bool(skinny)
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_gelu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    *,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """FP16-bias GEMM with fused GELU and NVFP4 output on SM110."""
+    m, n = a_packed.shape[0], b_packed.shape[0]
+    if out_packed is None:
+        out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_bias_gelu_nvfp4_fp16(
+        a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_residual_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    residual: torch.Tensor,
+    *,
+    out: torch.Tensor | None = None,
+) -> torch.Tensor:
+    """FP16-output GEMM with fused FP16 bias and residual on SM110."""
+    if out is None:
+        out = torch.empty_like(residual)
+    ops.nvfp4_gemm_bias_residual_fp16(
+        a_packed, b_packed, sfa, sfb, bias, residual, out
+    )
+    return out
+
+
 def fp4_w4a4_gemv_warpsplit_bf16(
     a_packed: torch.Tensor,
     b_packed: torch.Tensor,
@@ -292,7 +407,7 @@ def nvfp4_gemm_bias_gelu_nvfp4(
     if out_packed is None:
         out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
     if out_sfa is None:
-        out_sfa = torch.empty((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
     ops.nvfp4_gemm_bias_gelu_nvfp4(
         a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa, float(alpha)
     )
@@ -345,6 +460,10 @@ __all__ = [
     "fp4_w4a16_linear_bf16",
     "fp4_w4a4_gemv_warpsplit_bf16",
     "nvfp4_gemm_bf16",
+    "nvfp4_gemm_fp16",
+    "nvfp4_gemm_geglu_nvfp4_fp16",
+    "nvfp4_gemm_bias_gelu_nvfp4_fp16",
+    "nvfp4_gemm_bias_residual_fp16",
     "nvfp4_gemm_bias_bf16",
     "nvfp4_gemm_bias_gelu_bf16",
     "nvfp4_gemm_bias_gelu_nvfp4",

build/torch212-cxx11-cu132-x86_64-linux/{_fp4_gemm_cuda_8a66d8b.abi3.so → _fp4_gemm_cuda_55c4885.abi3.so}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:69c1903f5c54a54d13a0a27890d26ca0f17123dfceb463d44c1cc7185d2db1b6
-size 2843976
+oid sha256:9745e68973460d1d2fbb943c7ba63ab9fb23d8bb8a59c707bcf386e3891c69cb
+size 6342256

build/torch212-cxx11-cu132-x86_64-linux/_ops.py

@@ -1,9 +1,9 @@
 import torch
-from . import _fp4_gemm_cuda_8a66d8b
-ops = torch.ops._fp4_gemm_cuda_8a66d8b
+from . import _fp4_gemm_cuda_55c4885
+ops = torch.ops._fp4_gemm_cuda_55c4885
 
 def add_op_namespace_prefix(op_name: str):
     """
     Prefix op by namespace.
     """
-    return f"_fp4_gemm_cuda_8a66d8b::{op_name}"
+    return f"_fp4_gemm_cuda_55c4885::{op_name}"

build/torch212-cxx11-cu132-x86_64-linux/metadata.json

@@ -1,6 +1,6 @@
 {
   "name": "fp4-gemm",
-  "id": "_fp4_gemm_cuda_8a66d8b",
+  "id": "_fp4_gemm_cuda_55c4885",
   "version": 1,
   "license": "Apache-2.0",
   "python-depends": [],
@@ -14,19 +14,19 @@
   "digest": {
     "algorithm": "sha256",
     "files": {
-      "__init__.py": "+Kk/VNnIWwe9nszQYvL2qtU3DpKxbqHVfklg5WtVuB4=",
-      "_fp4_gemm_cuda_8a66d8b.abi3.so": "acGQP1xUpU0ToKJ4kNJsoPFxI9/OtGPUTBzHGF0tsbY=",
-      "_ops.py": "mZSHUC0K9o9glzvGmRSdtPppOq4bmIaxNEjibkaWTk0="
+      "__init__.py": "4o968SKYWj8r1Ly7FqSPFK/AYdu4G4bXuVv1UcvYjk8=",
+      "_fp4_gemm_cuda_55c4885.abi3.so": "l0XmiXNGDR0vu5Q8e6Y6ufsj2LuKWccHvPOG44kcacs=",
+      "_ops.py": "IVtq+3pHXwuoCbgYGw3JP7/nKr3GEZqn8D97JJjf7ZE="
     }
   },
   "provenance": {
     "kernel-builder": {
       "version": "0.17.0-dev0",
-      "sha": "870e825d881664e39f9287a27a74ef63ff3c545e",
+      "sha": "81f55ea30fd8f819dcf93a3c934dd584c895bd2f",
       "dirty": false
     },
     "kernel": {
-      "sha": "8a66d8b7f79a6fde86aa6929db2b60ab097e2d55",
+      "sha": "55c4885251068f195418bcf9ae541f4b757a6ea0",
       "dirty": false
     }
   }

build/torch213-cxx11-cu130-x86_64-linux/__init__.py

@@ -19,7 +19,8 @@ def sfa_size_bytes(rows: int, dim: int) -> int:
 def _alloc_fp4(rows: int, dim: int, device: torch.device | str):
     return (
         torch.empty((rows, dim // 2), device=device, dtype=torch.uint8),
-        torch.empty((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
+        # Tile-layout padding entries are not written by every quantizer.
+        torch.zeros((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
     )
 
 
@@ -36,6 +37,26 @@ def _linear_fake(
     return None
 
 
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_fp16"))
+def _linear_fp16_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, variant: int = -1) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_geglu_nvfp4_fp16"))
+def _geglu_fp4_fake(a, b, sfa, sfb, scratch, out_packed, out_sfa, skinny: bool = False) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_gelu_nvfp4_fp16"))
+def _bias_gelu_fp4_fp16_fake(a, b, sfa, sfb, bias, out_packed, out_sfa) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_residual_fp16"))
+def _bias_residual_fp16_fake(a, b, sfa, sfb, bias, residual, out) -> None:
+    return None
+
+
 @torch.library.register_fake(add_op_namespace_prefix("fp4_w4a4_gemv_warpsplit_bf16"))
 def _gemv_warpsplit_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, warps: int = 4, stages: int = 4) -> None:
     if a_packed.shape[0] != 1:
@@ -162,6 +183,100 @@ def nvfp4_gemm_bf16(
     return out
 
 
+def nvfp4_gemm_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    alpha: float = 1.0,
+    out: torch.Tensor | None = None,
+    variant: int = -1,
+) -> torch.Tensor:
+    """SM110 native NVFP4 GEMM with FP16 output."""
+    if out is None:
+        out = torch.empty(
+            (a_packed.shape[0], b_packed.shape[0]),
+            device=a_packed.device,
+            dtype=torch.float16,
+        )
+    ops.nvfp4_gemm_fp16(
+        a_packed, b_packed, sfa, sfb, out, float(alpha), int(variant)
+    )
+    return out
+
+
+def nvfp4_gemm_geglu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_interleaved_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    *,
+    skinny: bool = False,
+    scratch: torch.Tensor | None = None,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """GEMM with fused GeGLU and compact NVFP4 output on SM110.
+
+    ``b_interleaved_packed`` stores gate/up rows pairwise, so its first
+    dimension is twice the logical hidden width.
+    """
+    m, n_twice = a_packed.shape[0], b_interleaved_packed.shape[0]
+    hidden = n_twice // 2
+    if scratch is None:
+        scratch = torch.empty((m, hidden), device=a_packed.device, dtype=torch.uint8)
+    if out_packed is None:
+        out_packed = torch.empty((m, hidden // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, hidden),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_geglu_nvfp4_fp16(
+        a_packed, b_interleaved_packed, sfa, sfb, scratch,
+        out_packed, out_sfa, bool(skinny)
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_gelu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    *,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """FP16-bias GEMM with fused GELU and NVFP4 output on SM110."""
+    m, n = a_packed.shape[0], b_packed.shape[0]
+    if out_packed is None:
+        out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_bias_gelu_nvfp4_fp16(
+        a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_residual_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    residual: torch.Tensor,
+    *,
+    out: torch.Tensor | None = None,
+) -> torch.Tensor:
+    """FP16-output GEMM with fused FP16 bias and residual on SM110."""
+    if out is None:
+        out = torch.empty_like(residual)
+    ops.nvfp4_gemm_bias_residual_fp16(
+        a_packed, b_packed, sfa, sfb, bias, residual, out
+    )
+    return out
+
+
 def fp4_w4a4_gemv_warpsplit_bf16(
     a_packed: torch.Tensor,
     b_packed: torch.Tensor,
@@ -292,7 +407,7 @@ def nvfp4_gemm_bias_gelu_nvfp4(
     if out_packed is None:
         out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
     if out_sfa is None:
-        out_sfa = torch.empty((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
     ops.nvfp4_gemm_bias_gelu_nvfp4(
         a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa, float(alpha)
     )
@@ -345,6 +460,10 @@ __all__ = [
     "fp4_w4a16_linear_bf16",
     "fp4_w4a4_gemv_warpsplit_bf16",
     "nvfp4_gemm_bf16",
+    "nvfp4_gemm_fp16",
+    "nvfp4_gemm_geglu_nvfp4_fp16",
+    "nvfp4_gemm_bias_gelu_nvfp4_fp16",
+    "nvfp4_gemm_bias_residual_fp16",
     "nvfp4_gemm_bias_bf16",
     "nvfp4_gemm_bias_gelu_bf16",
     "nvfp4_gemm_bias_gelu_nvfp4",

build/torch213-cxx11-cu130-x86_64-linux/{_fp4_gemm_cuda_8a66d8b.abi3.so → _fp4_gemm_cuda_55c4885.abi3.so}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2fe7efef250feb7e8ec881de0919f010812cc5840a2b071993b4336cdfa9717e
-size 2847968
+oid sha256:0a35111ce16d53ac8a6300acc0ea58b3c560abc04ca16e3be4297b4e3bb09335
+size 6350344

build/torch213-cxx11-cu130-x86_64-linux/_ops.py

@@ -1,9 +1,9 @@
 import torch
-from . import _fp4_gemm_cuda_8a66d8b
-ops = torch.ops._fp4_gemm_cuda_8a66d8b
+from . import _fp4_gemm_cuda_55c4885
+ops = torch.ops._fp4_gemm_cuda_55c4885
 
 def add_op_namespace_prefix(op_name: str):
     """
     Prefix op by namespace.
     """
-    return f"_fp4_gemm_cuda_8a66d8b::{op_name}"
+    return f"_fp4_gemm_cuda_55c4885::{op_name}"

build/torch213-cxx11-cu130-x86_64-linux/metadata.json

@@ -1,6 +1,6 @@
 {
   "name": "fp4-gemm",
-  "id": "_fp4_gemm_cuda_8a66d8b",
+  "id": "_fp4_gemm_cuda_55c4885",
   "version": 1,
   "license": "Apache-2.0",
   "python-depends": [],
@@ -14,19 +14,19 @@
   "digest": {
     "algorithm": "sha256",
     "files": {
-      "__init__.py": "+Kk/VNnIWwe9nszQYvL2qtU3DpKxbqHVfklg5WtVuB4=",
-      "_fp4_gemm_cuda_8a66d8b.abi3.so": "L+fv7yUP636OyIHeCRnwEIEsxYQKKwcZk7QzbN+pcX4=",
-      "_ops.py": "mZSHUC0K9o9glzvGmRSdtPppOq4bmIaxNEjibkaWTk0="
+      "__init__.py": "4o968SKYWj8r1Ly7FqSPFK/AYdu4G4bXuVv1UcvYjk8=",
+      "_fp4_gemm_cuda_55c4885.abi3.so": "CjURHOFtU6yKYwCswOpYs8Vgq8BMoW475Cl7TjuwkzU=",
+      "_ops.py": "IVtq+3pHXwuoCbgYGw3JP7/nKr3GEZqn8D97JJjf7ZE="
     }
   },
   "provenance": {
     "kernel-builder": {
       "version": "0.17.0-dev0",
-      "sha": "870e825d881664e39f9287a27a74ef63ff3c545e",
+      "sha": "81f55ea30fd8f819dcf93a3c934dd584c895bd2f",
       "dirty": false
     },
     "kernel": {
-      "sha": "8a66d8b7f79a6fde86aa6929db2b60ab097e2d55",
+      "sha": "55c4885251068f195418bcf9ae541f4b757a6ea0",
       "dirty": false
     }
   }

build/torch213-cxx11-cu132-x86_64-linux/__init__.py

@@ -19,7 +19,8 @@ def sfa_size_bytes(rows: int, dim: int) -> int:
 def _alloc_fp4(rows: int, dim: int, device: torch.device | str):
     return (
         torch.empty((rows, dim // 2), device=device, dtype=torch.uint8),
-        torch.empty((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
+        # Tile-layout padding entries are not written by every quantizer.
+        torch.zeros((sfa_size_bytes(rows, dim),), device=device, dtype=torch.uint8),
     )
 
 
@@ -36,6 +37,26 @@ def _linear_fake(
     return None
 
 
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_fp16"))
+def _linear_fp16_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, variant: int = -1) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_geglu_nvfp4_fp16"))
+def _geglu_fp4_fake(a, b, sfa, sfb, scratch, out_packed, out_sfa, skinny: bool = False) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_gelu_nvfp4_fp16"))
+def _bias_gelu_fp4_fp16_fake(a, b, sfa, sfb, bias, out_packed, out_sfa) -> None:
+    return None
+
+
+@torch.library.register_fake(add_op_namespace_prefix("nvfp4_gemm_bias_residual_fp16"))
+def _bias_residual_fp16_fake(a, b, sfa, sfb, bias, residual, out) -> None:
+    return None
+
+
 @torch.library.register_fake(add_op_namespace_prefix("fp4_w4a4_gemv_warpsplit_bf16"))
 def _gemv_warpsplit_fake(a_packed, b_packed, sfa, sfb, out, alpha: float = 1.0, warps: int = 4, stages: int = 4) -> None:
     if a_packed.shape[0] != 1:
@@ -162,6 +183,100 @@ def nvfp4_gemm_bf16(
     return out
 
 
+def nvfp4_gemm_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    alpha: float = 1.0,
+    out: torch.Tensor | None = None,
+    variant: int = -1,
+) -> torch.Tensor:
+    """SM110 native NVFP4 GEMM with FP16 output."""
+    if out is None:
+        out = torch.empty(
+            (a_packed.shape[0], b_packed.shape[0]),
+            device=a_packed.device,
+            dtype=torch.float16,
+        )
+    ops.nvfp4_gemm_fp16(
+        a_packed, b_packed, sfa, sfb, out, float(alpha), int(variant)
+    )
+    return out
+
+
+def nvfp4_gemm_geglu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_interleaved_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    *,
+    skinny: bool = False,
+    scratch: torch.Tensor | None = None,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """GEMM with fused GeGLU and compact NVFP4 output on SM110.
+
+    ``b_interleaved_packed`` stores gate/up rows pairwise, so its first
+    dimension is twice the logical hidden width.
+    """
+    m, n_twice = a_packed.shape[0], b_interleaved_packed.shape[0]
+    hidden = n_twice // 2
+    if scratch is None:
+        scratch = torch.empty((m, hidden), device=a_packed.device, dtype=torch.uint8)
+    if out_packed is None:
+        out_packed = torch.empty((m, hidden // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, hidden),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_geglu_nvfp4_fp16(
+        a_packed, b_interleaved_packed, sfa, sfb, scratch,
+        out_packed, out_sfa, bool(skinny)
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_gelu_nvfp4_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    *,
+    out_packed: torch.Tensor | None = None,
+    out_sfa: torch.Tensor | None = None,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """FP16-bias GEMM with fused GELU and NVFP4 output on SM110."""
+    m, n = a_packed.shape[0], b_packed.shape[0]
+    if out_packed is None:
+        out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
+    if out_sfa is None:
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+    ops.nvfp4_gemm_bias_gelu_nvfp4_fp16(
+        a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa
+    )
+    return out_packed, out_sfa
+
+
+def nvfp4_gemm_bias_residual_fp16(
+    a_packed: torch.Tensor,
+    b_packed: torch.Tensor,
+    sfa: torch.Tensor,
+    sfb: torch.Tensor,
+    bias: torch.Tensor,
+    residual: torch.Tensor,
+    *,
+    out: torch.Tensor | None = None,
+) -> torch.Tensor:
+    """FP16-output GEMM with fused FP16 bias and residual on SM110."""
+    if out is None:
+        out = torch.empty_like(residual)
+    ops.nvfp4_gemm_bias_residual_fp16(
+        a_packed, b_packed, sfa, sfb, bias, residual, out
+    )
+    return out
+
+
 def fp4_w4a4_gemv_warpsplit_bf16(
     a_packed: torch.Tensor,
     b_packed: torch.Tensor,
@@ -292,7 +407,7 @@ def nvfp4_gemm_bias_gelu_nvfp4(
     if out_packed is None:
         out_packed = torch.empty((m, n // 2), device=a_packed.device, dtype=torch.uint8)
     if out_sfa is None:
-        out_sfa = torch.empty((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
+        out_sfa = torch.zeros((sfa_size_bytes(m, n),), device=a_packed.device, dtype=torch.uint8)
     ops.nvfp4_gemm_bias_gelu_nvfp4(
         a_packed, b_packed, sfa, sfb, bias, out_packed, out_sfa, float(alpha)
     )
@@ -345,6 +460,10 @@ __all__ = [
     "fp4_w4a16_linear_bf16",
     "fp4_w4a4_gemv_warpsplit_bf16",
     "nvfp4_gemm_bf16",
+    "nvfp4_gemm_fp16",
+    "nvfp4_gemm_geglu_nvfp4_fp16",
+    "nvfp4_gemm_bias_gelu_nvfp4_fp16",
+    "nvfp4_gemm_bias_residual_fp16",
     "nvfp4_gemm_bias_bf16",
     "nvfp4_gemm_bias_gelu_bf16",
     "nvfp4_gemm_bias_gelu_nvfp4",

build/torch213-cxx11-cu132-x86_64-linux/{_fp4_gemm_cuda_8a66d8b.abi3.so → _fp4_gemm_cuda_55c4885.abi3.so}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7043e89c27719838b2dfab5d8c82e8098528a792ba5d89861a5bfe45cecc022c
-size 2843824
+oid sha256:ce8cdbdd5f77404bb14b639d16ca5cdada3ddd43a40d6edeb5a4cdee2d6a260e
+size 6342104

build/torch213-cxx11-cu132-x86_64-linux/_ops.py

@@ -1,9 +1,9 @@
 import torch
-from . import _fp4_gemm_cuda_8a66d8b
-ops = torch.ops._fp4_gemm_cuda_8a66d8b
+from . import _fp4_gemm_cuda_55c4885
+ops = torch.ops._fp4_gemm_cuda_55c4885
 
 def add_op_namespace_prefix(op_name: str):
     """
     Prefix op by namespace.
     """
-    return f"_fp4_gemm_cuda_8a66d8b::{op_name}"
+    return f"_fp4_gemm_cuda_55c4885::{op_name}"

build/torch213-cxx11-cu132-x86_64-linux/metadata.json

@@ -1,6 +1,6 @@
 {
   "name": "fp4-gemm",
-  "id": "_fp4_gemm_cuda_8a66d8b",
+  "id": "_fp4_gemm_cuda_55c4885",
   "version": 1,
   "license": "Apache-2.0",
   "python-depends": [],
@@ -14,19 +14,19 @@
   "digest": {
     "algorithm": "sha256",
     "files": {
-      "__init__.py": "+Kk/VNnIWwe9nszQYvL2qtU3DpKxbqHVfklg5WtVuB4=",
-      "_fp4_gemm_cuda_8a66d8b.abi3.so": "cEPonCdxmDiy36tdjILoCYUop5K6XYmGGlv+Rc7MAiw=",
-      "_ops.py": "mZSHUC0K9o9glzvGmRSdtPppOq4bmIaxNEjibkaWTk0="
+      "__init__.py": "4o968SKYWj8r1Ly7FqSPFK/AYdu4G4bXuVv1UcvYjk8=",
+      "_fp4_gemm_cuda_55c4885.abi3.so": "zozb3V93QEuxS2OdFspc2to93UOkDW7etaTN7i1qJg4=",
+      "_ops.py": "IVtq+3pHXwuoCbgYGw3JP7/nKr3GEZqn8D97JJjf7ZE="
     }
   },
   "provenance": {
     "kernel-builder": {
       "version": "0.17.0-dev0",
-      "sha": "870e825d881664e39f9287a27a74ef63ff3c545e",
+      "sha": "81f55ea30fd8f819dcf93a3c934dd584c895bd2f",
       "dirty": false
     },
     "kernel": {
-      "sha": "8a66d8b7f79a6fde86aa6929db2b60ab097e2d55",
+      "sha": "55c4885251068f195418bcf9ae541f4b757a6ea0",
       "dirty": false
     }
   }