On Windows, the official CUDA builds of torch ship without flash attention. `_fused` only checks the GPU generation, so any Ampere or newer card takes the flash path and the first `system_one` call fails:
```
RuntimeError: USE_FLASH_ATTENTION was not enabled for build.
attention.py, line 80, in causal_attention
out = torch.ops.aten._flash_attention_forward(...)
```
Setup: Windows 11, RTX laptop GPU (Ampere or newer), torch 2.6.0+cu124, transformers 5.22.2, bf16.
This PR adds `torch.backends.cuda.is_flash_attention_available()` to `_fused`. When flash is missing, every call site already falls back to its SDPA path (`causal_attention`, `_windowed` and `Sees.attend`), so nothing else changes. On builds that have flash, `_fused` returns the same value as before.
```python
def _fused(q: torch.Tensor) -> bool:
"""Whether the flash kernels take these queries: a GPU in half precision, ROCm or CUDA from Ampere on."""
if not q.is_cuda or q.dtype not in (torch.float16, torch.bfl…
5 days ago
Inline nav icons so they load with the page and read in dark mode
transformers 5.22 calls causal_conv1d_fn with the layer's keywords (sliding_window); like transformers' own torch path, the convolution now ignores them. Without it, an audio clip raised a TypeError on 5.22.
fuse_projections: view a group already in one storage instead of copying it (#3)
- fuse_projections: view a group already in one storage instead of copying it (33b2fffb16d2e24874a7dafee0c1d4b16fe975ab)
Co-authored-by: Chao-Chun (Joe) Hsu <joe32140@users.noreply.huggingface.co>
Questions whose instructions or option descriptions are JSON objects, as some Decision Index benchmarks write them, are read as they are; only a media dict is a spoken question.
