Deepseek-V3.2 https://huggingface.co/deepseek-ai/DeepSeek-V3.2
Would appreciate any Quants for Deepseek V3.2.
https://huggingface.co/deepseek-ai/DeepSeek-V3.2
Preferably Q4, though whatever will fit on 336 GB RAM / 92GB VRAM (Combined)
Thanks for all you do.
Too big? 😟
Would appreciate any Quants for Deepseek V3.2.
So would I. What a shame that llama.cpp does not yet support Deepseek V3.2. Please follow https://github.com/ggml-org/llama.cpp/issues/16331 and let us know once it is supported but given that development started 29th of September and is still far from done I would not expect it to be supported anytime soon. Deepseek V3.2 requires a ton of work for llama.cpp to support and createthis is giving his best.
Too big? 😟
Not at all. I love doing massive models using my RPC imatrix computation setup. We did far larger models in the past. Our biggest one so far being FATLLAMA-1.7T-Instruct (https://huggingface.co/RichardErkhov/FATLLAMA-1.7T-Instruct).