Instructions to use alan13367/Qwen3.5-4B-RustCoder-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use alan13367/Qwen3.5-4B-RustCoder-MLX-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit") config = load_config("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use alan13367/Qwen3.5-4B-RustCoder-MLX-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "alan13367/Qwen3.5-4B-RustCoder-MLX-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "alan13367/Qwen3.5-4B-RustCoder-MLX-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use alan13367/Qwen3.5-4B-RustCoder-MLX-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "alan13367/Qwen3.5-4B-RustCoder-MLX-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default alan13367/Qwen3.5-4B-RustCoder-MLX-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use alan13367/Qwen3.5-4B-RustCoder-MLX-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "alan13367/Qwen3.5-4B-RustCoder-MLX-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "alan13367/Qwen3.5-4B-RustCoder-MLX-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3.5-4B-RustCoder-MLX-8bit
Qwen/Qwen3.5-4B fine-tuned to write Rust that compiles, passes tests and is clippy-clean. It keeps its thinking mode. This is the MLX 8-bit checkpoint for Apple Silicon, the one that was trained and evaluated.
Other versions: Qwen3.5-4B-RustCoder (bf16 safetensors, for transformers and other non-MLX tools) and Qwen3.5-4B-RustCoder-GGUF (for llama.cpp).
Version 2 (2026-11-20). A weighted merge of two LoRA fine-tunes (see Training), +3.2 points over the first release. The first release is still available at revision 225e85b.
Every training answer was checked by a gate that builds the code, runs hidden tests and lints it. Answers that failed it were not used.
Results
Pass rate: the share of answers whose code passes the task's hidden tests. Each task was sampled 4 times in each of two evaluation runs, with thinking on and no system prompt. The test tiers were never trained on.
| Pass rate | Easy (29 tasks) | Hard (40) | Quality (29) | Dev (82) | All (180) |
|---|---|---|---|---|---|
| Qwen3.5-4B (8-bit) | 43.5% | 6.9% | 8.6% | 20.1% | 19.1% |
| Qwen3.5-4B-RustCoder v2 | 53.0% | 8.1% | 15.9% | 23.6% | 23.7% |
- Over the base: +4.6 points pooled over all 180 tasks (95% CI +1.8 to +7.4). 52 tasks improved and 25 got worse.
- Over the first release: +3.2 points (95% CI +1.0 to +5.3), measured the same way. The first release's card reported one evaluation run; every number on this card is the average of two.
- The tiers are small, so a single tier has a wide margin. Quality (+7.3, CI +1.3 to +14.2) is the clearest per-tier gain, easy (+9.5) is probably real, and hard (+1.2) did not move.
- Quality covers fixing bugs, refactoring, writing tests and new domains. A refactor counts only if it is also lint-clean. Dev is the tier used to pick checkpoints.
- At least 1 of 4 attempts passes on 86% of easy tasks and 21% of hard tasks.
With a compiler in the loop
It does much better when the compiler checks its answers before one is kept. Same tiers; the hidden tests only grade the answer that was kept:
| Setup | Easy | Hard | Quality | Dev | All |
|---|---|---|---|---|---|
| First answer (above) | 53.0% | 8.1% | 15.9% | 23.6% | 23.7% |
| 4 answers, keep one that builds and is clippy-clean, up to two retries with the compiler's errors | 81.2% | 22.4% | 33.1% | 34.0% | 38.9% |
| The same with 8 answers, ranked by unit tests the model writes itself | 84.5% | 26.2% | 36.0% | 39.0% | 43.0% |
For the last row, the model first writes a #[cfg(test)] module from the task description alone (without the hidden tests), and among the clean answers the one that passes the most of those tests is kept.
General ability, against the base (subsets of each benchmark):
| MMLU (200) | ARC-Challenge (150) | GSM8K (100) | HellaSwag (150) | |
|---|---|---|---|---|
| Qwen3.5-4B (8-bit) | 69.5% | 89.3% | 91.0% | 80.7% |
| Qwen3.5-4B-RustCoder v2 | 69.5% | 90.0% | 94.0% | 80.7% |
On general text, its token distribution stays close to the base: KL 0.024 nats/token and 94.3% top-1 agreement.
Usage
With mlx-vlm (the checkpoint keeps the base's Qwen3.5 layout, vision tower included):
pip install -U mlx-vlm
python -m mlx_vlm.generate --model alan13367/Qwen3.5-4B-RustCoder-MLX-8bit --enable-thinking \
--max-tokens 4096 --temperature 0.6 --top-p 0.95 --top-k 20 \
--prompt "Write in Rust: pub fn median(xs: &mut [i64]) -> Option<f64>"
from mlx_vlm import load, generate
model, processor = load("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit")
messages = [{"role": "user", "content": "Write in Rust: pub fn median(xs: &mut [i64]) -> Option<f64>"}]
prompt = processor.tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
out = generate(model, processor, prompt, max_tokens=4096, temperature=0.6, top_p=0.95, top_k=20)
print(out.text)
Recommended:
- Use no system prompt. Training and evaluation used none. A long style prompt sent the base model into runaway reasoning.
- Leave thinking on and cap the output at about 4096 tokens. Answers take about 1.2k (easy tasks) to 1.8k tokens (hard ones) on average. An attempt that runs to the cap is usually stuck in a reasoning loop, so resampling helps more than raising the cap.
- Use temperature 0.6, top-p 0.95 and top-k 20. These are the settings behind the numbers above.
- Check its code with the compiler when you can. Picking an answer that builds and is clippy-clean, and retrying with the error output, nearly doubles its pass rate (see With a compiler in the loop).
Training
Two LoRA fine-tunes of the 8-bit MLX model, merged by weight: 0.7 times the first fine-tune's weight update plus 0.3 times the second's. The merge is exact (the two low-rank updates are concatenated, rank 32 in all) and involves no further training. Both fine-tunes used LoRA rank 16, alpha 32, dropout 0.05, on the last 16 decoder layers, one epoch at learning rate 1e-4, batches of up to 4096 tokens, and the base's own answers to general, non-Rust prompts as replay.
- First fine-tune (the first release): 739 samples, 784k tokens. 238 of the base model's own passing attempts at the training tasks, 277 self-repairs (the model fixing its own failed attempt, given the compiler or test output as the next turn), 175 passing traces from DeepSeek V4.1 Flash kept only when they matched the 4B's own length and style, and 88 replay answers. Validation loss 0.411.
- Second fine-tune: 3,442 samples, 4.04M tokens, with 595 more generated training tasks weighted toward fixing, refactoring and test writing. About half are the model's own answers and repairs (874 and 881), half DeepSeek V4.1 Flash records on the tasks the 4B could not solve (812 answers, 815 repairs), with the teacher's reasoning weighted 0.3 in the loss so the model keeps its own way of reasoning. On its own it gained on easy and hard tasks and lost on dev; the 0.7 / 0.3 merge kept the gains without the loss.
- Two merge weights were tried (0.5 and 0.7) and the better one kept, judged on these test tiers, so the result includes a small selection.
- The loss covers only the final reply: in repair records, the failed attempt and the feedback are context, not training targets.
What the experiments behind this checkpoint showed:
- The model learns best from its own style. Reference solutions with model-written reasoning made it worse (easy 41% to 27%): it imitated their dense style and compiled less often.
- Unfiltered teacher traces also made it worse. Their long, second-guessing reasoning taught it to never leave thinking. Filtered to its own length and style, or with their reasoning down-weighted, they were safe.
- More rounds of the same stopped helping. Training again on its own answers, on twice the data, or on the compiler loop's fixed answers gave no further gain. RL (GRPO, 60 steps with the test gate as the reward) raised the training reward without improving held-out tasks.
Limitations
- It is a 4B model and Rust is hard for it. Many of its first answers still fail to compile, mostly with type, trait-bound and method errors in its own code (E0308, E0277, E0599), and harder multi-part tasks rarely pass (hard tier 8.1%). Use it with a compiler in the loop.
- It still over-thinks sometimes. Cap the output, and resample when an answer gets stuck.
- Only the text side was trained. The vision tower is the base's, unchanged and untested here.
- The evaluation sets are small (29 to 82 tasks per tier), so each per-tier number has a wide margin. Treat the gains as modest.
- It was trained and evaluated on standalone tasks: a function, a type or a small module checked by hidden tests, plus bug fixes, refactors and test-writing. It has not been measured on multi-file projects or long-context editing.
License
Apache 2.0, the license of the base model Qwen/Qwen3.5-4B.
- Downloads last month
- 35
8-bit
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit") config = load_config("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output)