How to load gguf file?
#1
by Niel - opened
How do you load the GGUF model (Q4_K_M)? Im following the sample code, it works fine loading the default model.
https://github.com/flowaicom/flow-judge/blob/main/examples/2_custom_evaluation_criteria.ipynb
Niel changed discussion title from Loading up gguf file to How to load gguf file?
Niel changed discussion status to closed
Niel changed discussion status to open