| --- |
| license: other |
| library_name: flash-attn |
| tags: |
| - flash-attention |
| - pytorch |
| - cuda |
| - windows |
| - comfyui |
| - custom-wheel |
| - inference |
| --- |
| |
| # FlashAttention 2.8.3 — Windows Wheel for PyTorch 2.12 / CUDA 13 |
|
|
| A community-built Windows wheel for **FlashAttention 2.8.3**, compiled from the official FlashAttention v2.8.3 source. |
|
|
| This build was created because upgrading to **PyTorch 2.12.1+cu130** can leave existing Windows FlashAttention builds incompatible. |
|
|
| ## Compatibility |
|
|
| | Component | Tested configuration | |
| |---|---| |
| | Operating System | Windows 11 x64 | |
| | Python | 3.12 | |
| | PyTorch | 2.12.1+cu130 | |
| | CUDA Toolkit | 13.0.88 | |
| | FlashAttention | 2.8.3 | |
| | MSVC | Visual Studio 2022 / MSVC 19.44 | |
| | GPU | NVIDIA GeForce RTX 4070 Ti SUPER | |
| | GPU Architecture | SM89 / Compute Capability 8.9 | |
| | C++11 ABI | TRUE | |
| | Build System | Ninja 1.13.2 | |
|
|
| ## Download |
|
|
| Download `flash_attn-2.8.3-cp312-cp312-win_amd64.whl` from the **Files** tab above. |
|
|
| ## Installation |
|
|
| After downloading the wheel: |
|
|
| python -m pip install flash_attn-2.8.3-cp312-cp312-win_amd64.whl --no-deps |
| |
| If using a specific Python installation: |
|
|
| D:\apps\Python\Python312\python.exe -m pip install flash_attn-2.8.3-cp312-cp312-win_amd64.whl --no-deps |
| |
| ## Verification |
|
|
| The compiled extension was tested by executing an actual FlashAttention CUDA kernel on an NVIDIA GeForce RTX 4070 Ti SUPER. |
|
|
| Test result: |
|
|
| SUCCESS - Flash Attention CUDA kernel executed |
| Output: torch.Size([1, 128, 8, 128]) torch.float16 cuda:0 |
| GPU: NVIDIA GeForce RTX 4070 Ti SUPER |
| |
| The build was subsequently tested with ComfyUI: |
|
|
| PyTorch: 2.12.1+cu130 |
| Device: NVIDIA GeForce RTX 4070 Ti SUPER |
| Using Flash Attention |
| |
| ComfyUI successfully started with FlashAttention enabled. |
|
|
| ## Build Information |
|
|
| Source: |
|
|
| **FlashAttention v2.8.3** |
|
|
| Build configuration: |
|
|
| CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0 |
| FLASH_ATTENTION_FORCE_CXX11_ABI=TRUE |
| FLASH_ATTN_CUDA_ARCHS=89 |
| MAX_JOBS=4 |
| NVCC_THREADS=2 |
| DISTUTILS_USE_SDK=1 |
| |
| Built using: |
|
|
| python -m pip wheel . --no-build-isolation --no-deps -w .\wheels |
| |
| ## Important — Read Before Installing |
|
|
| ⚠️ **This is an unofficial community build.** |
|
|
| It is **not an official FlashAttention release** and is not affiliated with or endorsed by the FlashAttention developers. |
|
|
| The wheel has been tested on the configuration listed above. |
|
|
| In particular, this build was compiled for: |
|
|
| **NVIDIA SM89 / Ada Lovelace** |
|
|
| Other GPU architectures, Python versions, PyTorch versions, or CUDA versions may not be compatible. |
|
|
| Use at your own risk. |
|
|
| If this works on your hardware, please report your configuration so compatibility can be expanded. |
|
|
| ## Why does this exist? |
|
|
| An existing Windows FlashAttention 2.8.3 installation may stop working after upgrading PyTorch. |
|
|
| For example, a previously working build may produce: |
|
|
| ImportError: DLL load failed while importing flash_attn_2_cuda: |
| The specified procedure could not be found. |
| |
| This wheel was rebuilt against: |
|
|
| **PyTorch 2.12.1+cu130 / CUDA 13.0** |
|
|
| and verified with an actual CUDA kernel execution. |
|
|
| ## SHA-256 |
|
|
| 2d858cd3b3b1aacd48d5edec6e85db933eaef23ef2b47a81873968e4f8f6d913 |
| |
| ## Credits |
|
|
| FlashAttention: |
| https://github.com/Dao-AILab/flash-attention |
|
|
| Original project by Tri Dao and contributors. |
|
|
| This repository contains a community-built Windows binary based on FlashAttention v2.8.3. |
