FoolDev Claude Opus 4.7 commited on
Commit
6672746
·
1 Parent(s): b14787f

Modelfile: add explicit stop tokens to prevent EOS bleed-through

Browse files

Without these, Ollama only honored <|im_end|> from the GGUF metadata.
The model occasionally emits <|endoftext|> instead and Ollama kept
generating past it, synthesising a fake new user turn — reproducible
via make smoke.

Adding <|im_end|>, <|endoftext|>, and <|im_start|> as explicit stops
gives clean termination.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Files changed (2) hide show
  1. CHANGELOG.md +7 -0
  2. Modelfile +9 -0
CHANGELOG.md CHANGED
@@ -7,6 +7,13 @@ and documentation**, not the underlying base model.
7
 
8
  ## [Unreleased]
9
 
 
 
 
 
 
 
 
10
  ### Removed
11
  - `Modelfile.z13` and the `z13` build profile. The Z13-specific tuning
12
  (Q3_K_S + 8K ctx + FA + q8_0 KV cache) wasn't meaningfully different
 
7
 
8
  ## [Unreleased]
9
 
10
+ ### Fixed
11
+ - `Modelfile`: added explicit `PARAMETER stop` directives for `<|im_end|>`,
12
+ `<|endoftext|>`, and `<|im_start|>`. Ollama was only picking up
13
+ `<|im_end|>` from the GGUF metadata, so when the model emitted
14
+ `<|endoftext|>` it kept generating past it and synthesised a fake new
15
+ user turn. Reproducible via `make smoke` before/after.
16
+
17
  ### Removed
18
  - `Modelfile.z13` and the `z13` build profile. The Z13-specific tuning
19
  (Q3_K_S + 8K ctx + FA + q8_0 KV cache) wasn't meaningfully different
Modelfile CHANGED
@@ -28,6 +28,15 @@ PARAMETER top_k 20
28
  PARAMETER repeat_penalty 1.05
29
  PARAMETER num_ctx 16384
30
 
 
 
 
 
 
 
 
 
 
31
  SYSTEM """You are Janus, a precise and capable assistant for reasoning, writing, coding, and long-form dialogue.
32
 
33
  Behavior rules:
 
28
  PARAMETER repeat_penalty 1.05
29
  PARAMETER num_ctx 16384
30
 
31
+ # Stop tokens. Without these, Ollama only honors <|im_end|> from the GGUF
32
+ # metadata; the model occasionally emits <|endoftext|> instead and Ollama
33
+ # keeps generating past it (synthesising a fake new user turn). Listing
34
+ # both — plus <|im_start|> as a belt-and-braces guard against the same
35
+ # loop — keeps responses cleanly terminated.
36
+ PARAMETER stop "<|im_end|>"
37
+ PARAMETER stop "<|endoftext|>"
38
+ PARAMETER stop "<|im_start|>"
39
+
40
  SYSTEM """You are Janus, a precise and capable assistant for reasoning, writing, coding, and long-form dialogue.
41
 
42
  Behavior rules: