Mikrodev commited on
Commit
dca9346
·
verified ·
1 Parent(s): 8507ec0

Upload Modelfile with huggingface_hub

Browse files
Files changed (1) hide show
  1. Modelfile +10 -19
Modelfile CHANGED
@@ -10,31 +10,22 @@
10
  # 2) ollama create stcoder-qwen35-9b:<quant> -f Modelfile e.g. ollama create stcoder-qwen35-9b:q8_0 -f Modelfile
11
  # 3) ollama run stcoder-qwen35-9b:<quant> "Motor starts 5 seconds after the start button; stop and E-stop drop it."
12
  #
13
- # Requires Ollama 0.32.3 or newer.
 
 
 
 
 
 
 
 
14
  # num_ctx 8192 matches the sequence length this model was fine-tuned at.
15
  # Published numbers were measured greedily (temperature 0, seed 42) at num_ctx 16384 / num_predict 8192. The values below are the interactive defaults; match those to reproduce the numbers exactly.
16
- # Ollama 0.32.3 or newer is REQUIRED: earlier builds (0.24 tested) fail to load this architecture. The GGUF is fine - the loader was not.
17
  # This family reasons before answering and the thinking is drawn from the same budget, which is why num_predict is 8192 here and not 2048. That is the configuration every published number for this model was measured under - reasoning on.
18
- # One reply in fifteen spent the whole budget thinking and returned no code. If you would rather trade that risk for speed, you can suppress the reasoning by adding an empty <think></think> pair after the final <|im_start|>assistant line in the TEMPLATE above - but note that we did not evaluate the model in that mode.
19
 
20
  FROM ./qwen35_9b-tc.q8_0.gguf
21
 
22
- TEMPLATE """{{- if .System }}<|im_start|>system
23
- {{ .System }}<|im_end|>
24
- {{ end }}
25
- {{- range .Messages }}
26
- {{- if eq .Role "user" }}<|im_start|>user
27
- {{ .Content }}<|im_end|>
28
- {{ else if eq .Role "assistant" }}<|im_start|>assistant
29
- {{ .Content }}<|im_end|>
30
- {{ end }}
31
- {{- end }}<|im_start|>assistant
32
- """
33
-
34
- PARAMETER stop "<|im_end|>"
35
- PARAMETER stop "<|endoftext|>"
36
- PARAMETER stop "<|im_start|>"
37
-
38
  PARAMETER temperature 0.2
39
  PARAMETER top_p 0.95
40
  PARAMETER top_k 20
 
10
  # 2) ollama create stcoder-qwen35-9b:<quant> -f Modelfile e.g. ollama create stcoder-qwen35-9b:q8_0 -f Modelfile
11
  # 3) ollama run stcoder-qwen35-9b:<quant> "Motor starts 5 seconds after the start button; stop and E-stop drop it."
12
  #
13
+ # Requires Ollama 0.32.3 or newer; verified on 0.32.5.
14
+ #
15
+ # No TEMPLATE line here, deliberately: Ollama uses the chat template stored inside the GGUF
16
+ # - the template of the tokenizer this model was trained with, so it cannot drift out of
17
+ # sync with the weights. We measured this: adding a TEMPLATE to this file does not change
18
+ # the prompt the model receives (identical prompt token counts with, without, and with a
19
+ # deliberately wrong template). With llama.cpp directly, pass --jinja so llama-cli and
20
+ # llama-server use that same embedded template.
21
+ #
22
  # num_ctx 8192 matches the sequence length this model was fine-tuned at.
23
  # Published numbers were measured greedily (temperature 0, seed 42) at num_ctx 16384 / num_predict 8192. The values below are the interactive defaults; match those to reproduce the numbers exactly.
 
24
  # This family reasons before answering and the thinking is drawn from the same budget, which is why num_predict is 8192 here and not 2048. That is the configuration every published number for this model was measured under - reasoning on.
25
+ # One reply in fifteen spent the whole budget thinking and returned no code. To trade that risk for speed, turn the reasoning off at the API rather than by editing this file: send `"think": false` to /api/chat, or type `/set nothink` inside `ollama run`. Both were verified against this build (reasoning block empty, answer produced). Note we did not evaluate the model in that mode.
26
 
27
  FROM ./qwen35_9b-tc.q8_0.gguf
28
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
  PARAMETER temperature 0.2
30
  PARAMETER top_p 0.95
31
  PARAMETER top_k 20