Did anyone have any success running this on a 5090?
I get a mismatch between the cu121 and my GPU.
For an RTX 5090 (Blackwell architecture) GPU, you need CUDA 12.8 as a minimum.
e.g. A Torch (and matching Torchaudio) version with '+cu128' appended, and which matches your Python version.
(The installation instructions use Python 3.10.)
Update:
I have it working with PyTorch2.10+cu130 and Python 3.12, but had to drop the version of Transformers right down to 4.37.2 to have it work correctly.
I'm using an RTX 3090 GPU, on Windows11 25H2.
I don't have a correctly built Nemotron or Megatron, and SageAttention won't work no matter what version I try, but it's very fast processing on the GPU still, taking only a few seconds to generate an output.
It has taken me several hours to get everything installed and working, including trashing the venv file and starting over at one point.
The documentation is lacking in a few details, such as with regard to installing nltk_data. And the requirements.txt, as it is, will not give you a working app, at least not in my situation.
Good luck with it. It's fun to play with.