title: RustLean v4 FIM Playground
emoji: 🦀
colorFrom: indigo
colorTo: red
sdk: gradio
sdk_version: 6.20.0
app_file: app.py
pinned: false
license: apache-2.0
models:
- AfkaraLP/rustlean-gguf
short_description: Cursor-aware Rust FIM code completion
RustLean v4 FIM Playground
Cursor-aware Rust code completion using
AfkaraLP/rustlean-gguf — a
Qwen2.5-Coder-1.5B model fine-tuned for fill-in-the-middle (FIM) on Rust,
served via llama-cpp-python. RustLean v4 improves private AST exact match
from the base model's 22.5% to 35.0% while matching its deterministic
HumanEvalPack-Rust pass rate.
How it works
Place a <|cursor|> marker anywhere in the editor. The app splits the file
into a prefix and a suffix at that marker and builds the Qwen PSM prompt:
<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>
The generated middle is spliced back in. If there is no suffix, the model runs in pure prefix-completion mode.
Hardware (ZeroGPU)
Set the Space hardware to ZeroGPU in Settings → Hardware.
Secrets
Add a Space Secret named HF_TOKEN with a read token
(https://huggingface.co/settings/tokens). The Space only needs it if the model
repo is private or to avoid rate limits while downloading the GGUF.
Optional environment overrides (Space Variables or local .env):
| Variable | Default | Meaning |
|---|---|---|
MODEL_REPO |
AfkaraLP/rustlean-gguf |
HF repo id |
MODEL_FILE |
rustlean-v4.Q8_0.gguf |
GGUF filename |
RUSTLEAN_CTX |
8192 |
Context window |
N_GPU_LAYERS |
-1 |
-1 offloads all layers to the GPU |
Local development (NixOS / CUDA)
This repo ships a flake.nix for a CUDA dev shell (built for the RTX 2070 Super,
compute capability sm_75 / Turing):
nix develop # sets up CUDA + builds llama-cpp-python with GGML_CUDA
cp .env.example .env && edit .env # put your HF_TOKEN here
python app.py
On a 2070 Super expect ~40 tok/s decode (per the model card).