If I have 16GB of VRAM, which Q should I choose?
#10
by Alfredo182 - opened
My setup is in my profile. I hope someone can answer me, I'm new to this.
You want to end up with at least 1GB free after loading all models. That leaves room for overhead. If you put your hardware in your profile, many model pages will show you what they think you can use. For instance, here are little pics of what it thinks I can get away with with my 16GB 5070Ti, my 32GB dual-5060Ti setup (If you want to know how I did this, look here: https://www.reddit.com/r/LocalLLaMA/comments/1r1qpdv/dual_rtx_5060_ti_32gb_pooled_vram_vs_single_rtx/), my 16GB Mac Pro, and my 8GB Jetson Nano - which it says can't run any of them. :( (But that's okay, it has other things to do. ;) )
