Instructions to use kernels-community/flash-attn2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use kernels-community/flash-attn2 with Kernels:
# !pip install kernels from kernels import get_kernel kernel = get_kernel("kernels-community/flash-attn2") - Notebooks
- Google Colab
- Kaggle
drbh commited on
Commit ·
6c3907a
1
Parent(s): 3099d65
fix: adjust comma typos
Browse files
torch-ext/torch_binding.h
CHANGED
|
@@ -10,11 +10,11 @@ mha_fwd(torch::Tensor &q,
|
|
| 10 |
c10::optional<torch::Tensor> alibi_slopes_,
|
| 11 |
const double p_dropout,
|
| 12 |
const double softmax_scale,
|
| 13 |
-
bool is_causal
|
| 14 |
-
const int64_t window_size_left
|
| 15 |
const int64_t window_size_right,
|
| 16 |
const double softcap,
|
| 17 |
-
const bool return_softmax
|
| 18 |
c10::optional<at::Generator> gen_);
|
| 19 |
|
| 20 |
std::vector<torch::Tensor>
|
|
|
|
| 10 |
c10::optional<torch::Tensor> alibi_slopes_,
|
| 11 |
const double p_dropout,
|
| 12 |
const double softmax_scale,
|
| 13 |
+
bool is_causal,
|
| 14 |
+
const int64_t window_size_left,
|
| 15 |
const int64_t window_size_right,
|
| 16 |
const double softcap,
|
| 17 |
+
const bool return_softmax,
|
| 18 |
c10::optional<at::Generator> gen_);
|
| 19 |
|
| 20 |
std::vector<torch::Tensor>
|