README.md

---
license: apache-2.0
language:
- en
- zh
tags:
- comfyui
- blackwell
- pytorch
- cuda
- attention
---

# Blackwell4ComfyUI

**English** | [简体中文](README_zh-CN.md)

A collection of prebuilt attention-acceleration extensions for **ComfyUI on Windows with NVIDIA Blackwell (RTX 50-series) GPUs**. The project currently provides Windows x64 wheels built for Python 3.11, PyTorch 2.9.1, and CUDA 13.0, avoiding the need to configure Visual Studio, a CUDA build toolchain, and lengthy local compilation.

## Included Components

| Component | Package | Version | Purpose |
| --- | --- | --- | --- |
| Block Sparse Attention | `block_sparse_attn` | 0.0.1 | Dense, streaming, and block-sparse attention kernels for reducing the cost of long-sequence attention |
| SpargeAttention | `spas_sage_attn` | 0.1.0 | Training-free sparse attention based on SageAttention2 for language, image, and video model inference |
| SageAttention | `sageattention` | 2.2.0 | Quantized attention kernels designed to accelerate inference while preserving accuracy |
| FlashAttention | `flash_attn` | 2.8.4 | Memory-efficient exact attention kernels for faster training and inference |

## Current Compatibility

- Operating system: Windows x64
- Python: CPython 3.11
- PyTorch: 2.9.1
- CUDA: 13.0
- GPU: Primarily intended for NVIDIA Blackwell / RTX 50-series GPUs

These wheels are native binaries tied closely to their target runtime. A mismatch in Python, PyTorch, CUDA, or system architecture may prevent installation or cause DLL and CUDA kernel errors during import or execution.

## Directory Layout

```text
Blackwell4ComfyUI/
├─ Block-Sparse-Attention/
│  └─ cp311-torch2.9.1-cu130-win/
│     └─ block_sparse_attn-0.0.1-...-win_amd64.whl
├─ FlashAttention/
│  └─ cp311-torch2.9.1-cu130-win/
│     └─ flash_attn-2.8.4-...-win_amd64.whl
├─ SageAttention/
│  └─ cp311-torch2.9.1-cu130-win/
│     └─ sageattention-2.2.0-...-win_amd64.whl
└─ SpargeAttn/
   └─ cp311-torch2.9.1-cu130-win/
      └─ spas_sage_attn-0.1.0-...-win_amd64.whl
```

## Installation

Run the installation with the **Python interpreter used by ComfyUI**, not another Python installation on the system. Check the active environment first:

```powershell
python -c "import sys, torch; print(sys.version); print(torch.__version__); print(torch.version.cuda)"
```

After confirming that the output matches the compatibility requirements above, install the required wheels:

```powershell
python -m pip install ".\Block-Sparse-Attention\cp311-torch2.9.1-cu130-win\block_sparse_attn-0.0.1-cp311-cp311-torch2.9.1-cu130-win_amd64.whl"

python -m pip install ".\SpargeAttn\cp311-torch2.9.1-cu130-win\spas_sage_attn-0.1.0-cp311-cp311-torch2.9.1-cu130-win_amd64.whl"

python -m pip install ".\SageAttention\cp311-torch2.9.1-cu130-win\sageattention-2.2.0-cp311-cp311-torch2.9.1-cu130-win_amd64.whl"

python -m pip install ".\FlashAttention\cp311-torch2.9.1-cu130-win\flash_attn-2.8.4-cp311-cp311-torch2.9.1-cu130-win_amd64.whl"
```

Block Sparse Attention also declares `torch`, `einops`, `packaging`, and `ninja` as dependencies. pip will normally resolve missing dependencies automatically. Review its proposed dependency changes before proceeding to avoid replacing the PyTorch installation already configured for ComfyUI.

Verify that the installed modules can be imported:

```powershell
python -c "import block_sparse_attn; print('block_sparse_attn: OK')"
python -c "import spas_sage_attn; print('spas_sage_attn: OK')"
python -c "import sageattention; print('sageattention: OK')"
python -c "import flash_attn; print('flash_attn: OK')"
```

## Usage Notes

- Back up the ComfyUI Python environment before installing or upgrading packages.
- Do not allow pip to upgrade or downgrade existing PyTorch/CUDA components unless you have confirmed their compatibility.
- A successful import only confirms that the extension can be loaded. Run an actual target workflow to validate GPU architecture and CUDA kernel compatibility.
- These packages are low-level Python/CUDA extensions, not ComfyUI custom nodes. They take effect only when a node or model implementation calls their APIs.

## Upstream Projects

- [Block Sparse Attention](https://github.com/mit-han-lab/Block-Sparse-Attention)
- [SpargeAttention](https://github.com/thu-ml/SpargeAttn)
- [SageAttention](https://github.com/thu-ml/SageAttention)
- [FlashAttention](https://github.com/Dao-AILab/flash-attention)

Refer to the respective upstream projects for source code, licenses, API documentation, and paper citations. This repository only collects prebuilt artifacts for a specific Windows/ComfyUI environment.

## License

Original content in this repository is licensed under the [Apache License 2.0](https://huggingface.co/FengCoding/BlackwellWheels/blob/main/LICENSE).

The bundled third-party wheels and their contents remain subject to their respective upstream licenses. Apache-2.0 does not replace or modify those licenses; consult the package metadata and upstream projects before redistribution.