What are memory requirements for loading the Ternary weights?
Tested with 8GB VRAM, getting following error —
Thanks for helping to test! I've mainly tested on my M4 Max, so there are a lot of devices that are untested. This is actually something I am actively working on, and will hopefully share more soon.
In the meantime, can you let me know more of your browser/hardware specs?
Doesn't work on Intel HD Graphics 730 either. Windows 10, 16 GB RAM, Chromium 148 with new profile with default settings.
Memory usage at the time the error happens is about 8 GB, i.e. there's about 8 GB free left.
index-Bf-HmMxp.js:8128 pipeline preload failed: RangeError: Failed to execute 'createBuffer' on 'GPUDevice': createBuffer failed, size (512) is too large for the implementation when mappedAtCreation == true
at Object.createBuffer (index-Bf-HmMxp.js:1:8998)
at s.tensorFromTypedArray (index-Bf-HmMxp.js:1:2648)
at index-Bf-HmMxp.js:3964:2517
at Object.receive (index-Bf-HmMxp.js:2:29289)
at i (index-Bf-HmMxp.js:2:29783)
at b (index-Bf-HmMxp.js:1:15448)
at async x (index-Bf-HmMxp.js:1:15825)
at async Promise.all (index 0)
at async M.streamAll (index-Bf-HmMxp.js:1:15894)
at async e.fromBf16SafeTensors (index-Bf-HmMxp.js:3964:3778)
my @ index-Bf-HmMxp.js:8128
The 1-bit version does work, but it's useless crap because the right half of the image looks like it didn't get enough steps (just your typical color blocks) no matter how many steps you use. I tried 4, 8, 12 steps, and the right half stays the same undercooked mess. Here's an example for 8 steps: https://cdn-uploads.huggingface.co/production/uploads/noauth/hNFr2htcXIgut5JuHDkl7.png
I just learned that an Intel iGPU won't use the entire RAM. In my case, it only uses 7 GB out of 16 (even though the Task Manager thinks the maximum is 7.9 GB), and when it needs more, it just crashes. So my previous comment still falls under the 8 GB VRAM category (7 GB even).