| --- |
| title: RustLean v4 FIM Playground |
| emoji: 🦀 |
| colorFrom: indigo |
| colorTo: red |
| sdk: gradio |
| sdk_version: "6.20.0" |
| app_file: app.py |
| pinned: false |
| license: apache-2.0 |
| models: |
| - AfkaraLP/rustlean-gguf |
| short_description: Cursor-aware Rust FIM code completion |
| --- |
| |
| # RustLean v4 FIM Playground |
|
|
| Cursor-aware Rust code completion using |
| [`AfkaraLP/rustlean-gguf`](https://huggingface.co/AfkaraLP/rustlean-gguf) — a |
| Qwen2.5-Coder-1.5B model fine-tuned for **fill-in-the-middle (FIM)** on Rust, |
| served via `llama-cpp-python`. RustLean v4 improves private AST exact match |
| from the base model's 22.5% to 35.0% while matching its deterministic |
| HumanEvalPack-Rust pass rate. |
|
|
| ## How it works |
|
|
| Place a **`<|cursor|>`** marker anywhere in the editor. The app splits the file |
| into a prefix and a suffix at that marker and builds the Qwen PSM prompt: |
|
|
| ``` |
| <|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|> |
| ``` |
|
|
| The generated middle is spliced back in. If there is no suffix, the model runs |
| in pure prefix-completion mode. |
|
|
| ## Hardware (ZeroGPU) |
|
|
| Set the Space hardware to **ZeroGPU** in *Settings → Hardware*. |
|
|
| ## Secrets |
|
|
| Add a Space **Secret** named `HF_TOKEN` with a read token |
| (<https://huggingface.co/settings/tokens>). The Space only needs it if the model |
| repo is private or to avoid rate limits while downloading the GGUF. |
|
|
| Optional environment overrides (Space Variables or local `.env`): |
|
|
| | Variable | Default | Meaning | |
| | ---------------- | -------------------------- | -------------------------------------- | |
| | `MODEL_REPO` | `AfkaraLP/rustlean-gguf` | HF repo id | |
| | `MODEL_FILE` | `rustlean-v4.Q8_0.gguf` | GGUF filename | |
| | `RUSTLEAN_CTX` | `8192` | Context window | |
| | `N_GPU_LAYERS` | `-1` | `-1` offloads all layers to the GPU | |
|
|
| ## Local development (NixOS / CUDA) |
|
|
| This repo ships a `flake.nix` for a CUDA dev shell (built for the RTX 2070 Super, |
| compute capability `sm_75` / Turing): |
|
|
| ```bash |
| nix develop # sets up CUDA + builds llama-cpp-python with GGML_CUDA |
| cp .env.example .env && edit .env # put your HF_TOKEN here |
| python app.py |
| ``` |
|
|
| On a 2070 Super expect ~40 tok/s decode (per the model card). |
|
|
