Merge pull request #22 from MotifTechnologies/jangwoong/mla-rope-fa4-port

cleanup: drop k_pe RoPE custom kernel (caller uses PyTorch native)

refactor: replace warp shuffle with CUB BlockReduce

fix: unify all backward kernels to input-based math + fix test import

style: fix yapf/isort/clang-format for CI --all-files

feat: update RMSNorm Python interface for optimized kernels

perf: optimize RMSNorm CUDA kernels for all dims

feat: dedicated _kv_rope_bwd_kernel (register-sum + copy-fused)

perf: remove autotune, hard-code per-kernel configs from live dump

cleanup: remove dead Phase 3 Q kernel + shrink autotune to hand-picked configs

review fixups: stride asserts, autotune split, intent comments

feat: MLA RoPE Triton kernels (port from llm-training)

style: fix yapf/isort formatting for CI --all-files check

grouped polynorm with padding aware (#19)

chore: remove pre-built binaries and add local build loader shim (#18)

style: apply yapf, isort, and clang-format

style: fix clang-format on torch_binding.h

fix: rename stale references and clean up Triton remnants

refactor: remove Triton kernels, add hidden_clamp to unscored ops

feat: add grouped poly norm CUDA kernel with scores and hidden_clamp fusion

refactor: rename grouped_fused_mul_poly_norm → fused_mul_grouped_poly_norm

feat: add GroupedFusedMulPolyNorm Triton kernel for MoE models (#16)

fix: support PyTorch 2.10 register_op_strategy import path change

fix: fix fused add rms norm sharding strategy

fix: fix rms norm sharding strategy

fix(rms_norm.py): add assertion for input gradients to handle unsupported cases in backward pass

feat: support sequence parallel with fused_add_rms_norm

refactor(activation): change fused_add_rms_norm and fused_add_rms_norm_backward to out-place operations

refactor(rms_norm): move RMS normalization logic to a new module for better organization and maintainability

feat: support sequence parallel with rms_norm

feat: make rms_norm as out-place

Fix fused add rms norm (#4)

Add fusion (#3)

Optimize kernel (#2)

feat: support reset_parameters()

feat(rms-norm): Impl fused RMSNorm

refactor(poly-norm): use const for immutable args

chore: use latest build image and misc

feat(poly-norm): add default value for eps argument

feat(poly-norm): Add PolyNorm