Instructions to use kernels-community/flash-attn2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use kernels-community/flash-attn2 with Kernels:
# !pip install kernels from kernels import get_kernel kernel = get_kernel("kernels-community/flash-attn2") - Notebooks
- Google Colab
- Kaggle
Uploaded using `kernel-builder`.
Browse files
README.md
CHANGED
|
@@ -16,14 +16,3 @@ Function list not available.
|
|
| 16 |
## Benchmarks
|
| 17 |
|
| 18 |
Benchmarking script is available for this kernel. Run `kernels benchmark kernels-community/flash-attn2`.
|
| 19 |
-
|
| 20 |
-
### Performance
|
| 21 |
-
|
| 22 |
-
<img class="dark:hidden border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_light_animation.svg" />
|
| 23 |
-
<img class="hidden dark:block border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_dark_animation.svg" />
|
| 24 |
-
|
| 25 |
-
<img class="dark:hidden border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_light_latency.svg" />
|
| 26 |
-
<img class="hidden dark:block border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_dark_latency.svg" />
|
| 27 |
-
|
| 28 |
-
<img class="dark:hidden border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_light_throughput.svg" />
|
| 29 |
-
<img class="hidden dark:block border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_dark_throughput.svg" />
|
|
|
|
| 16 |
## Benchmarks
|
| 17 |
|
| 18 |
Benchmarking script is available for this kernel. Run `kernels benchmark kernels-community/flash-attn2`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|