memory allocation failed with OOM
Please help. I just can't get this model to run on my Asus GX10, no matter which recipe I use. Meanwhile, nvidia/Qwen3.8-Flash-Next-NVFP4 runs without any issues.
(EngineCore pid=178) INFO 10-20 18:21:49 [default_loader.py:575] Loading weights took 152.17 seconds
[rank0]:[W1020 18:22:26.325337083 CUDACachingAllocator.cpp:3933] memory allocation failed with OOM on device 0 while trying to allocate 20971520 bytes (free: 10585366528, total: 130594156544).
[rank0]:[W1020 18:22:27.801097111 CUDACachingAllocator.cpp:3933] memory allocation failed with OOM on device 0 while trying to allocate 20971520 bytes (free: 10586898432, total: 130594156544).
I figured out the problem. For some reason, the permissions for ~/.cache/b12x were incorrect, and the backend couldn't write anything there at all.