Hugging Face's logo

Kernels Tests
community
Activity Feed

AI & ML interests

None defined yet.

Recent Activity

danieldk  updated a model 19 days ago
kernels-test/versions
danieldk  updated a model about 1 month ago
kernels-test/python-dep
danieldk  updated a model about 2 months ago
kernels-test/silu-and-mul-noarch
View all activity

danieldk 
posted an update about 11 hours ago
view post
Post
345
kernels 0.12 is out! 🎉

Changes:

* Support for kernel version branches to gracefully roll out kernel API changes.
* Support for PyTorch 2.10.
* kernel-builder is now merged into the kernels repo.
* Initial support for standardized kernel benchmarks.

https://github.com/huggingface/kernels/releases/tag/v0.12.0
danieldk 
updated a model 19 days ago
danieldk 
updated a model about 1 month ago
danieldk 
updated a model about 2 months ago
danieldk 
published a model about 2 months ago
danieldk 
updated 2 models 2 months ago
danieldk 
published 2 models 2 months ago
danieldk 
published a model 3 months ago
danieldk 
posted an update 3 months ago
sayakpaul 
authored a paper 4 months ago
danieldk 
updated a model 4 months ago
danieldk 
published a model 4 months ago
danieldk 
updated a model 4 months ago
sayakpaul 
posted an update 6 months ago
view post
Post
2196
Fast LoRA inference for Flux with Diffusers and PEFT 🚨

There are great materials that demonstrate how to optimize inference for popular image generation models, such as Flux. However, very few cover how to serve LoRAs fast, despite LoRAs being an inseparable part of their adoption.

In our latest post, @BenjaminB and I show different techniques to optimize LoRA inference for the Flux family of models for image generation. Our recipe includes the use of:

1. torch.compile
2. Flash Attention 3 (when compatible)
3. Dynamic FP8 weight quantization (when compatible)
4. Hotswapping for avoiding recompilation during swapping new LoRAs 🤯

We have tested our recipe with Flux.1-Dev on both H100 and RTX 4090. We achieve at least a *2x speedup* in either of the GPUs. We believe our recipe is grounded in the reality of how LoRA-based use cases are generally served. So, we hope this will be beneficial to the community 🤗

Even though our recipe was tested primarily with NVIDIA GPUs, it should also work with AMD GPUs.

Learn the details and the full code here:
https://huggingface.co/blog/lora-fast
  • 3 replies
·
danieldk 
published a model 6 months ago
danieldk 
posted an update 7 months ago
view post
Post
2060
kernels 0.8.0 is out: https://github.com/huggingface/kernels/releases/tag/v0.8.0

This release refines kernel selection in the kernelize function:

• You can now register kernels for certain CUDA capability ranges.
• Rather than doing exact mating of modes, fall back to other compatible modes. If you are kernelizing for inference, but you only registered a training + torch.compile kernel, it will use that kernel since it is compatible with inference as well.
  • 1 reply
·
danieldk 
posted an update 7 months ago
danieldk 
posted an update 7 months ago
view post
Post
381
Kernels 0.7.0 is out: https://github.com/huggingface/kernels/releases/tag/v0.7.0 🚀

This release makes it possible to register multiple kernels for a layer. Do you have a super-fast kernel for inference and another kernel for training? Register them both and kernelize will pick the kernel depending on whether you are going to do training or inference.