File size: 2,328 Bytes
ef83975 d634d7e ef83975 1e63572 ef83975 d634d7e ef83975 d634d7e ef83975 d634d7e ef83975 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
title: RustLean v4 FIM Playground
emoji: 🦀
colorFrom: indigo
colorTo: red
sdk: gradio
sdk_version: "6.20.0"
app_file: app.py
pinned: false
license: apache-2.0
models:
- AfkaraLP/rustlean-gguf
short_description: Cursor-aware Rust FIM code completion
---
# RustLean v4 FIM Playground
Cursor-aware Rust code completion using
[`AfkaraLP/rustlean-gguf`](https://huggingface.co/AfkaraLP/rustlean-gguf) — a
Qwen2.5-Coder-1.5B model fine-tuned for **fill-in-the-middle (FIM)** on Rust,
served via `llama-cpp-python`. RustLean v4 improves private AST exact match
from the base model's 22.5% to 35.0% while matching its deterministic
HumanEvalPack-Rust pass rate.
## How it works
Place a **`<|cursor|>`** marker anywhere in the editor. The app splits the file
into a prefix and a suffix at that marker and builds the Qwen PSM prompt:
```
<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>
```
The generated middle is spliced back in. If there is no suffix, the model runs
in pure prefix-completion mode.
## Hardware (ZeroGPU)
Set the Space hardware to **ZeroGPU** in *Settings → Hardware*.
## Secrets
Add a Space **Secret** named `HF_TOKEN` with a read token
(<https://huggingface.co/settings/tokens>). The Space only needs it if the model
repo is private or to avoid rate limits while downloading the GGUF.
Optional environment overrides (Space Variables or local `.env`):
| Variable | Default | Meaning |
| ---------------- | -------------------------- | -------------------------------------- |
| `MODEL_REPO` | `AfkaraLP/rustlean-gguf` | HF repo id |
| `MODEL_FILE` | `rustlean-v4.Q8_0.gguf` | GGUF filename |
| `RUSTLEAN_CTX` | `8192` | Context window |
| `N_GPU_LAYERS` | `-1` | `-1` offloads all layers to the GPU |
## Local development (NixOS / CUDA)
This repo ships a `flake.nix` for a CUDA dev shell (built for the RTX 2070 Super,
compute capability `sm_75` / Turing):
```bash
nix develop # sets up CUDA + builds llama-cpp-python with GGML_CUDA
cp .env.example .env && edit .env # put your HF_TOKEN here
python app.py
```
On a 2070 Super expect ~40 tok/s decode (per the model card).
|