---
title: RustLean v4 FIM Playground
emoji: 🦀
colorFrom: indigo
colorTo: red
sdk: gradio
sdk_version: "6.20.0"
app_file: app.py
pinned: false
license: apache-2.0
models:
  - AfkaraLP/rustlean-gguf
short_description: Cursor-aware Rust FIM code completion
---

# RustLean v4 FIM Playground

Cursor-aware Rust code completion using
[`AfkaraLP/rustlean-gguf`](https://huggingface.co/AfkaraLP/rustlean-gguf) — a
Qwen2.5-Coder-1.5B model fine-tuned for **fill-in-the-middle (FIM)** on Rust,
served via `llama-cpp-python`. RustLean v4 improves private AST exact match
from the base model's 22.5% to 35.0% while matching its deterministic
HumanEvalPack-Rust pass rate.

## How it works

Place a **`<|cursor|>`** marker anywhere in the editor. The app splits the file
into a prefix and a suffix at that marker and builds the Qwen PSM prompt:

```
<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>
```

The generated middle is spliced back in. If there is no suffix, the model runs
in pure prefix-completion mode.

## Hardware (ZeroGPU)

Set the Space hardware to **ZeroGPU** in *Settings → Hardware*.

## Secrets

Add a Space **Secret** named `HF_TOKEN` with a read token
(<https://huggingface.co/settings/tokens>). The Space only needs it if the model
repo is private or to avoid rate limits while downloading the GGUF.

Optional environment overrides (Space Variables or local `.env`):

| Variable         | Default                    | Meaning                                |
| ---------------- | -------------------------- | -------------------------------------- |
| `MODEL_REPO`     | `AfkaraLP/rustlean-gguf`   | HF repo id                             |
| `MODEL_FILE`     | `rustlean-v4.Q8_0.gguf`    | GGUF filename                          |
| `RUSTLEAN_CTX`   | `8192`                     | Context window                         |
| `N_GPU_LAYERS`   | `-1`                       | `-1` offloads all layers to the GPU    |

## Local development (NixOS / CUDA)

This repo ships a `flake.nix` for a CUDA dev shell (built for the RTX 2070 Super,
compute capability `sm_75` / Turing):

```bash
nix develop         # sets up CUDA + builds llama-cpp-python with GGML_CUDA
cp .env.example .env && edit .env   # put your HF_TOKEN here
python app.py
```

On a 2070 Super expect ~40 tok/s decode (per the model card).