Valkyrie is an amazing model. This version seems not to 'think' with Ollama though.
#3
by
mirage335
- opened
Unfortunately, at least GGUF quants of this model, do not 'think'.
ollama run hf.co/bartowski/TheDrummer_Valkyrie-49B-v2-GGUF:IQ2_XXS Please tell me a short story.
This reflects some issues getting the upstream Llama-3_3-Nemotron-Super-49B-v1_5 model to 'think'. Only the unsloth quant seems to correctly 'think'.
ollama run hf.co/unsloth/Llama-3_3-Nemotron-Super-49B-v1_5-GGUF:IQ2_XXS Please tell me a short story.
Yes, larger quants can be noticeably better, the IQ2_XXS quant just suffices for quick testing and certain less creative but still important use cases.
Valkyrie is truly still a leading model even for some general purpose things like choosing synonyms, naming things, etc. Would be nice to get the reasoning working with the newer version, the 1.5 Nemotron version seems to address some weaknesses of using the previous 'Super' 49B parameter model instead of 'Ultra' 253B parameter model.