FlashAttention 2.8.3 — Windows Wheel for PyTorch 2.12 / CUDA 13
A community-built Windows wheel for FlashAttention 2.8.3, compiled from the official FlashAttention v2.8.3 source.
This build was created because upgrading to PyTorch 2.12.1+cu130 can leave existing Windows FlashAttention builds incompatible.
Compatibility
| Component | Tested configuration |
|---|---|
| Operating System | Windows 11 x64 |
| Python | 3.12 |
| PyTorch | 2.12.1+cu130 |
| CUDA Toolkit | 13.0.88 |
| FlashAttention | 2.8.3 |
| MSVC | Visual Studio 2022 / MSVC 19.44 |
| GPU | NVIDIA GeForce RTX 4070 Ti SUPER |
| GPU Architecture | SM89 / Compute Capability 8.9 |
| C++11 ABI | TRUE |
| Build System | Ninja 1.13.2 |
Download
Download flash_attn-2.8.3-cp312-cp312-win_amd64.whl from the Files tab above.
Installation
After downloading the wheel:
python -m pip install flash_attn-2.8.3-cp312-cp312-win_amd64.whl --no-deps
If using a specific Python installation:
D:\apps\Python\Python312\python.exe -m pip install flash_attn-2.8.3-cp312-cp312-win_amd64.whl --no-deps
Verification
The compiled extension was tested by executing an actual FlashAttention CUDA kernel on an NVIDIA GeForce RTX 4070 Ti SUPER.
Test result:
SUCCESS - Flash Attention CUDA kernel executed
Output: torch.Size([1, 128, 8, 128]) torch.float16 cuda:0
GPU: NVIDIA GeForce RTX 4070 Ti SUPER
The build was subsequently tested with ComfyUI:
PyTorch: 2.12.1+cu130
Device: NVIDIA GeForce RTX 4070 Ti SUPER
Using Flash Attention
ComfyUI successfully started with FlashAttention enabled.
Build Information
Source:
FlashAttention v2.8.3
Build configuration:
CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0
FLASH_ATTENTION_FORCE_CXX11_ABI=TRUE
FLASH_ATTN_CUDA_ARCHS=89
MAX_JOBS=4
NVCC_THREADS=2
DISTUTILS_USE_SDK=1
Built using:
python -m pip wheel . --no-build-isolation --no-deps -w .\wheels
Important — Read Before Installing
⚠️ This is an unofficial community build.
It is not an official FlashAttention release and is not affiliated with or endorsed by the FlashAttention developers.
The wheel has been tested on the configuration listed above.
In particular, this build was compiled for:
NVIDIA SM89 / Ada Lovelace
Other GPU architectures, Python versions, PyTorch versions, or CUDA versions may not be compatible.
Use at your own risk.
If this works on your hardware, please report your configuration so compatibility can be expanded.
Why does this exist?
An existing Windows FlashAttention 2.8.3 installation may stop working after upgrading PyTorch.
For example, a previously working build may produce:
ImportError: DLL load failed while importing flash_attn_2_cuda:
The specified procedure could not be found.
This wheel was rebuilt against:
PyTorch 2.12.1+cu130 / CUDA 13.0
and verified with an actual CUDA kernel execution.
SHA-256
2d858cd3b3b1aacd48d5edec6e85db933eaef23ef2b47a81873968e4f8f6d913
Credits
FlashAttention: https://github.com/Dao-AILab/flash-attention
Original project by Tri Dao and contributors.
This repository contains a community-built Windows binary based on FlashAttention v2.8.3.