rparkr commited on
Commit
0e053c9
·
verified ·
1 Parent(s): 163c3dc

chore: Add Ollama Modelfile

Browse files

# Summary
Add an Ollama Modelfile demonstrating how to run this fine-tuned model locally using Ollama.

Files changed (1) hide show
  1. Modelfile +28 -0
Modelfile ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Ollama Modelfile to use this fine-tuned model with Ollama.
2
+ # TODO (Ryan Parker, 7-May-2026): this Modelfile currently doesn't work.
3
+ # When using safetensors, the base model must be downloaded first, and
4
+ # Ollama raises an error stating that MLX is required for quantization.
5
+ # Note that the base model can be downloaded like this:
6
+ # uv tool run --from huggingface-hub -- hf download LiquidAI/LFM2.5-1.2B-Instruct
7
+ # When using GGUF, Ollama successfully creates the model, but then raises
8
+ # an error that LoRA is not supported.
9
+
10
+ # safetensors version
11
+ # Uses the adapter_model.safetensors trained in this repository.
12
+ # Usage:
13
+ # ollama create lfm-coder --file Modelfile --quantize q4_K_M --experimental
14
+
15
+ # FROM hf.co/LiquidAI/LFM2.5-1.2B-Instruct
16
+ # ADAPTER adapter_model.safetensors
17
+
18
+ # GGUF version
19
+ # Usage:
20
+ # ollama create lfm-coder --file Modelfile
21
+ FROM hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:Q4_K_M
22
+ ADAPTER hf.co/rparkr/LFM2.5-1.2B-Instruct-Coding-F16-GGUF
23
+
24
+ # Recommended settings from: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct#:~:text=Generation%20parameters%3A
25
+ PARAMETER temperature 0.1
26
+ PARAMETER top_k 50
27
+ PARAMETER repeat_penalty 1.05
28
+ PARAMETER num_ctx 128000