README.md

---
license: apache-2.0
tags:
- fill-in-the-middle
- code
- rust
- llama-cpp
- gguf
base_model: Qwen/Qwen2.5-Coder-1.5B
library_name: transformers
---

# RustLean v4 (GGUF)

RustLean v4 is a Rust-specialized, native fill-in-the-middle (FIM) completion
model based on
[`Qwen/Qwen2.5-Coder-1.5B`](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B).
This repository ships a merged `Q8_0` GGUF with no runtime LoRA dependency.

The release uses 50% of the selected step-1100 LoRA delta. The full adapter had
strong completion metrics but regressed deterministic HumanEvalPack-Rust. The
interpolated release retained its measured AST and out-of-distribution exact
match while recovering the base model's compiler-tested pass rate.

## Model Details

| Property | Value |
|---|---|
| Base | `Qwen/Qwen2.5-Coder-1.5B` |
| Architecture | Qwen2 decoder, 28 layers, width 1,536 |
| Attention | 12 query heads, 2 KV heads (GQA) |
| Parameters | Approximately 1.54B |
| Context | 32,768 tokens native; trained and evaluated at 1,024 |
| Adapter | Rank-32 LoRA on attention and MLP projections |
| Training | 1,200 optimizer steps, approximately 10.5M processed tokens |
| Release delta | 0.5 times the step-1100 adapter delta, merged into fp16 |
| Artifact | `rustlean-v4.Q8_0.gguf`, approximately 1.64 GB |

## Training Data

The licensed Rust corpus contains 58,407 training chunks from 1,048 repository
families and 3,044 holdout chunks from 73 unseen families. Repository families
are disjoint between training and evaluation. HumanEvalPack-Rust and MultiPL-E
prompts, tests, and canonical solutions were excluded from training.

Each source training chunk produced three FIM views and one left-to-right replay
row. The final mixture contains:

| Objective | Rows |
|---|---:|
| Total | 217,727 |
| FIM | 159,320 |
| Left-to-right | 58,407 |
| AST-boundary holes | 121,835 |
| Random editor holes | 37,485 |
| Empty-suffix prefix completions | 49,644 |

There are no duplicate FIM objectives. Exact duplicates were also removed
across the train and holdout source splits.

## Prompt Format

Use Qwen's PSM FIM format:

```text
<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>
```

For ordinary cursor completion, leave the suffix empty:

```text
<|fim_prefix|>fn add(a: i32, b: i32) -> i32 {
    <|fim_suffix|><|fim_middle|>
```

Stop at `<|endoftext|>`, `<|fim_prefix|>`, `<|fim_suffix|>`,
`<|fim_middle|>`, or `<|fim_pad|>`.

The same prefix-completion format is embedded as the GGUF chat template.

## Evaluation

All completion results below use greedy generation, a 1,024-token input crop,
a fixed 256-token generation budget, and control-token trimming. Private
results use the first 200 deterministic examples from repository-family-
disjoint holdouts.

| Model | Private AST exact | Similarity | Parse rate* |
|---|---:|---:|---:|
| Qwen2.5-Coder-1.5B | 22.5% | 0.465 | 71.88% |
| Previous RustLean | 31.5% | 0.647 | 81.25% |
| **RustLean v4** | **35.0%** | **0.680** | **84.38%** |

| Model | OOD exact, 95 tasks | Similarity | Parse rate* |
|---|---:|---:|---:|
| Previous RustLean | 30.53% | **0.793** | 95.24% |
| **RustLean v4** | **35.79%** | 0.781 | **100%** |

`*` Parse rate is measured only where the original reconstructed source is
parseable, so its denominator is smaller than the row count.

Deterministic HumanEvalPack-Rust uses 164 tasks, one greedy empty-suffix FIM
completion per task, Rust 1.95, and execution-backed tests:

| Model | Passing tasks | pass@1 |
|---|---:|---:|
| Qwen2.5-Coder-1.5B | 48/164 | 29.27% |
| Full step-1100 adapter | 35/164 | 21.34% |
| **RustLean v4, 0.5 delta** | **48/164** | **29.27%** |

HumanEvalPack is evaluation-only. No synthesis improvement over the base model
is claimed.

## Usage

### llama.cpp

```bash
llama-server -m rustlean-v4.Q8_0.gguf
```

For raw CLI completion:

```bash
llama-cli -m rustlean-v4.Q8_0.gguf --no-conversation -e \
  -p '<|fim_prefix|>fn main() {<|fim_suffix|><|fim_middle|>'
```

### llama-cpp-python

```python
from llama_cpp import Llama

llm = Llama(model_path="rustlean-v4.Q8_0.gguf", n_ctx=8192, n_gpu_layers=-1)
prompt = "<|fim_prefix|>fn add(a: i32, b: i32) -> i32 {\n    <|fim_suffix|><|fim_middle|>"
result = llm(
    prompt,
    max_tokens=96,
    temperature=0.2,
    stop=["<|endoftext|>", "<|fim_prefix|>", "<|fim_suffix|>", "<|fim_middle|>", "<|fim_pad|>"],
)
print(result["choices"][0]["text"])
```

## Files

- `rustlean-v4.Q8_0.gguf`: merged Q8_0 model with embedded FIM template.
- `rustlean-v4.jinja`: standalone copy of the FIM template.

GGUF SHA-256:
`47dc21750adee427aae050c0c44c7bd78e63c2792b9230e18bc35471453ec694`

## License

Apache-2.0, inherited from `Qwen/Qwen2.5-Coder-1.5B`.