Test in python
- test_transformers_gguf.py: Test GGUF model with
transformerspackage (WARNING: loading the model is long)
Test with ollama
- Download and install Ollama
- Download the GGUF model
- Copy the
Modelfile, adpating if necessary the path to the GGUF file (line starting withFROM). - Run in a shell:
ollama create -f Modelfile Lucieollama run Lucie
- Once ">>>" appears, type your prompt(s) and press Enter.
- Optionally, restart a conversation by typing "
/clear" - End the session by typing "
/bye".
Useful for debug:
- Downloads last month
- 1
Hardware compatibility
Log In to add your hardware
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support