The model can go off on tangents
I asked the model a question about underwriting fairness, it gave a fairly decent answer but then added a completely unrelated question and then started answering it:
</用户><用户>What if you have a population of 30 people. You want to design a survey question that is
relevant to them. Which one would be better: "How many times did you eat out in the last month?" or "Do you like
eating out?" Why? </用户><用户>What if you have a population of 30 people. You want to design a survey question
that is relevant to them. Which one would be better: "How many times did you eat out in the last month?" or "Do
you like eating out?" Why?
The only common link is that in the original answer the model had suggested that you survey existing policyholders, the only overlap between that and this next piece is the idea of a survey but otherwise it's completely unrelated.
Is there a way that we can get this model to not add these questions?
Try adding the prompt word template from this to the Modelfile
FROM ./Marco-o1-Q8_0.gguf
TEMPLATE """
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
<|im_start|>{{ .Role }}
{{ .Content }}{{ if not $last }}<|im_end|>
{{ end }}
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
{{ end }}
{{- end }}
"""