Add ApplyRoPE and RMSNorm kernels written in OpenAI Triton
#10
by Cheshire94 - opened
No description provided.
Cheshire94 changed pull request title from pr/9 to pr/10
This PR add kernels of ApplyRoPE and RMSNorm written in OpenAI Triton. These kernels offer better performance, support a wider range of GPU architectures (including V100 and T4), and require no pre-compilation, compared with flash-attn. They are enabled automatically if Triton is installed (usually bundled with PyTorch 2.x).
Cheshire94 changed pull request title from pr/10 to Add ApplyRoPE and RMSNorm kernels written in OpenAI Triton
Cheshire94 changed pull request status to closed