Gated Delta Attention
BF16 Gated DeltaNet recurrent/chunk/WY kernels from FlashRT, packaged for Hugging Face Kernel Hub. The v2 public profile covers Qwen3.6-style linear-attention decode recurrence and prefill WY building blocks.
Available functions
gated_delta_recurrent_bf16gated_delta_recurrent_inout_bf16gated_delta_recurrent_f32state_bf16iogated_delta_chunk_bf16gated_delta_chunk_smem_bf16lin_split_qkv_broadcast_bf16lin_split_qkv_gqa_bf16split_q_gate_bf16gdn_gating_bf16gdn_gating_strided_bf16gdn_chunk_from_conv_smem_bf16gdn_wy_norm_cumsum_pack_qk_bf16gdn_wy_kkt_b64_bf16gdn_wy_solve_tril_b64_f32gdn_wy_recompute_wu_b64_bf16gdn_wy_chunk_h_b64_bf16gdn_wy_output_o_b64_bf16
Usage
from kernels import get_kernel
gdn = get_kernel("flashrt/gated-delta-attention", version=2, trust_remote_code=True)
out = gdn.gated_delta_recurrent_bf16(q, k, v, g, beta, state)
The WY helpers use the Qwen3.6 profile: conv_out=(S,10240),
Q/K heads 16, value heads 48, head dimension 128, and 64-token WY
blocks.
- Downloads last month
- 172
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
# !pip install kernels from kernels import get_kernel kernel = get_kernel("flashrt/gated-delta-attention")