AfkaraLP/rustlean-gguf

RustLean v4 (GGUF)

RustLean v4 is a Rust-specialized, native fill-in-the-middle (FIM) completion model based on Qwen/Qwen2.5-Coder-1.5B. This repository ships a merged Q8_0 GGUF with no runtime LoRA dependency.

The release uses 50% of the selected step-1100 LoRA delta. The full adapter had strong completion metrics but regressed deterministic HumanEvalPack-Rust. The interpolated release retained its measured AST and out-of-distribution exact match while recovering the base model's compiler-tested pass rate.

Model Details

Property	Value
Base	Qwen/Qwen2.5-Coder-1.5B
Architecture	Qwen2 decoder, 28 layers, width 1,536
Attention	12 query heads, 2 KV heads (GQA)
Parameters	Approximately 1.54B
Context	32,768 tokens native; trained and evaluated at 1,024
Adapter	Rank-32 LoRA on attention and MLP projections
Training	1,200 optimizer steps, approximately 10.5M processed tokens
Release delta	0.5 times the step-1100 adapter delta, merged into fp16
Artifact	rustlean-v4.Q8_0.gguf, approximately 1.64 GB

Training Data

The licensed Rust corpus contains 58,407 training chunks from 1,048 repository families and 3,044 holdout chunks from 73 unseen families. Repository families are disjoint between training and evaluation. HumanEvalPack-Rust and MultiPL-E prompts, tests, and canonical solutions were excluded from training.

Each source training chunk produced three FIM views and one left-to-right replay row. The final mixture contains:

Objective	Rows
Total	217,727
FIM	159,320
Left-to-right	58,407
AST-boundary holes	121,835
Random editor holes	37,485
Empty-suffix prefix completions	49,644

There are no duplicate FIM objectives. Exact duplicates were also removed across the train and holdout source splits.

Prompt Format

Use Qwen's PSM FIM format:

<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>

For ordinary cursor completion, leave the suffix empty:

<|fim_prefix|>fn add(a: i32, b: i32) -> i32 {
    <|fim_suffix|><|fim_middle|>

Stop at <|endoftext|>, <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|>, or <|fim_pad|>.

The same prefix-completion format is embedded as the GGUF chat template.

Evaluation

All completion results below use greedy generation, a 1,024-token input crop, a fixed 256-token generation budget, and control-token trimming. Private results use the first 200 deterministic examples from repository-family- disjoint holdouts.

Model	Private AST exact	Similarity	Parse rate*
Qwen2.5-Coder-1.5B	22.5%	0.465	71.88%
Previous RustLean	31.5%	0.647	81.25%
RustLean v4	35.0%	0.680	84.38%

Model	OOD exact, 95 tasks	Similarity	Parse rate*
Previous RustLean	30.53%	0.793	95.24%
RustLean v4	35.79%	0.781	100%

* Parse rate is measured only where the original reconstructed source is parseable, so its denominator is smaller than the row count.

Deterministic HumanEvalPack-Rust uses 164 tasks, one greedy empty-suffix FIM completion per task, Rust 1.95, and execution-backed tests:

Model	Passing tasks	pass@1
Qwen2.5-Coder-1.5B	48/164	29.27%
Full step-1100 adapter	35/164	21.34%
RustLean v4, 0.5 delta	48/164	29.27%

HumanEvalPack is evaluation-only. No synthesis improvement over the base model is claimed.

Usage

llama.cpp

llama-server -m rustlean-v4.Q8_0.gguf

For raw CLI completion:

llama-cli -m rustlean-v4.Q8_0.gguf --no-conversation -e \
  -p '<|fim_prefix|>fn main() {<|fim_suffix|><|fim_middle|>'

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="rustlean-v4.Q8_0.gguf", n_ctx=8192, n_gpu_layers=-1)
prompt = "<|fim_prefix|>fn add(a: i32, b: i32) -> i32 {\n    <|fim_suffix|><|fim_middle|>"
result = llm(
    prompt,
    max_tokens=96,
    temperature=0.2,
    stop=["<|endoftext|>", "<|fim_prefix|>", "<|fim_suffix|>", "<|fim_middle|>", "<|fim_pad|>"],
)
print(result["choices"][0]["text"])

Files

• rustlean-v4.Q8_0.gguf: merged Q8_0 model with embedded FIM template.
• rustlean-v4.jinja: standalone copy of the FIM template.

GGUF SHA-256: 47dc21750adee427aae050c0c44c7bd78e63c2792b9230e18bc35471453ec694

License

Apache-2.0, inherited from Qwen/Qwen2.5-Coder-1.5B.