rustlean-gguf / rustlean-final.jinja
AfkaraLP's picture
Add model card and FIM chat template
4a1a729 verified
Raw
History Blame Contribute Delete
1.21 kB
{# RustLean — native fill-in-the-middle (FIM) Rust completion model (Qwen2.5-Coder-1.5B base).
This model was trained with the Qwen2.5-Coder FIM objective:
<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>{middle}<|endoftext|>
The dominant serving mode (per the technical report) is *prefix completion*
with an empty suffix, i.e. the model is fed:
<|fim_prefix|>{prefix}<|fim_suffix|><|fim_middle|>
and generates the missing middle. This template implements exactly that for a
single user turn: the user message content is treated as the prefix and the
FIM markers are emitted around it.
Multi-turn / suffix-aware infilling is handled natively by llama.cpp via the
auto-detected <|fim_prefix|>/<|fim_suffix|>/<|fim_middle|> special tokens
(infill endpoint / --fim-* flags), which does not use this template.
llama.cpp variable contract: `messages`, `add_generation_prompt`.
#}
{%- if messages -%}
{%- set ns = namespace(prefix="") -%}
{%- for message in messages -%}
{%- if message["role"] == "user" -%}
{%- set ns.prefix = ns.prefix + message["content"] -%}
{%- endif -%}
{%- endfor -%}
<|fim_prefix|>{{ ns.prefix }}<|fim_suffix|><|fim_middle|>
{%- else -%}
{{ prompt }}
{%- endif -%}