kernels/pr_595/en/integrating-kernels.md

Integrating kernels

This page shows how different projects use kernels.

autoresearch

karpathy/autoresearch uses kernels to integrate Flash-Attention 3 through the get_kernel() method.

AReaL

inclusionAI/AReaL uses kernels in an opt-in manner to integrate optimized attention mechanisms.

transformers

huggingface/transformers primarily depends on kernels for all optimizations related to optimized kernels, including optimized attention implementations, MoE blocks, and quantization. Besides get_kernel(), it also uses kernel layers to optimize the forward passes of common layers involved in the modeling blocks. Some references are available here and here.

Refer to the following posts to know more:

• Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers
• Mixture of Experts (MoEs) in Transformers

diffusers

Similar to transformers, huggingface/diffusers uses kernels for integrating optimized kernels to compute attention.

Besides leveraging pre-built compute kernels, different projects rely on kernels to also package, build, and distribute their kernels on the Hugging Face Hub platform. This is made possible by the "builder" component of kernels. Visit huggingface.co/kernels to browse the pre-built compute kernels available on the Hub.

Feel free to open a PR enlisting your project to show how kernels is leveraged there.