How to use from the
Use from the
MLX library
# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit")
config = load_config("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Qwen3.5-4B-RustCoder-MLX-8bit

Qwen/Qwen3.5-4B fine-tuned to write Rust that compiles, passes tests and is clippy-clean. It keeps its thinking mode. This is the MLX 8-bit checkpoint for Apple Silicon, the one that was trained and evaluated.

Other versions: Qwen3.5-4B-RustCoder (bf16 safetensors, for transformers and other non-MLX tools) and Qwen3.5-4B-RustCoder-GGUF (for llama.cpp).

Version 2 (2026-11-20). A weighted merge of two LoRA fine-tunes (see Training), +3.2 points over the first release. The first release is still available at revision 225e85b.

Every training answer was checked by a gate that builds the code, runs hidden tests and lints it. Answers that failed it were not used.

Results

Pass rate: the share of answers whose code passes the task's hidden tests. Each task was sampled 4 times in each of two evaluation runs, with thinking on and no system prompt. The test tiers were never trained on.

Pass rate Easy (29 tasks) Hard (40) Quality (29) Dev (82) All (180)
Qwen3.5-4B (8-bit) 43.5% 6.9% 8.6% 20.1% 19.1%
Qwen3.5-4B-RustCoder v2 53.0% 8.1% 15.9% 23.6% 23.7%
  • Over the base: +4.6 points pooled over all 180 tasks (95% CI +1.8 to +7.4). 52 tasks improved and 25 got worse.
  • Over the first release: +3.2 points (95% CI +1.0 to +5.3), measured the same way. The first release's card reported one evaluation run; every number on this card is the average of two.
  • The tiers are small, so a single tier has a wide margin. Quality (+7.3, CI +1.3 to +14.2) is the clearest per-tier gain, easy (+9.5) is probably real, and hard (+1.2) did not move.
  • Quality covers fixing bugs, refactoring, writing tests and new domains. A refactor counts only if it is also lint-clean. Dev is the tier used to pick checkpoints.
  • At least 1 of 4 attempts passes on 86% of easy tasks and 21% of hard tasks.

With a compiler in the loop

It does much better when the compiler checks its answers before one is kept. Same tiers; the hidden tests only grade the answer that was kept:

Setup Easy Hard Quality Dev All
First answer (above) 53.0% 8.1% 15.9% 23.6% 23.7%
4 answers, keep one that builds and is clippy-clean, up to two retries with the compiler's errors 81.2% 22.4% 33.1% 34.0% 38.9%
The same with 8 answers, ranked by unit tests the model writes itself 84.5% 26.2% 36.0% 39.0% 43.0%

For the last row, the model first writes a #[cfg(test)] module from the task description alone (without the hidden tests), and among the clean answers the one that passes the most of those tests is kept.

General ability, against the base (subsets of each benchmark):

MMLU (200) ARC-Challenge (150) GSM8K (100) HellaSwag (150)
Qwen3.5-4B (8-bit) 69.5% 89.3% 91.0% 80.7%
Qwen3.5-4B-RustCoder v2 69.5% 90.0% 94.0% 80.7%

On general text, its token distribution stays close to the base: KL 0.024 nats/token and 94.3% top-1 agreement.

Usage

With mlx-vlm (the checkpoint keeps the base's Qwen3.5 layout, vision tower included):

pip install -U mlx-vlm
python -m mlx_vlm.generate --model alan13367/Qwen3.5-4B-RustCoder-MLX-8bit --enable-thinking \
  --max-tokens 4096 --temperature 0.6 --top-p 0.95 --top-k 20 \
  --prompt "Write in Rust: pub fn median(xs: &mut [i64]) -> Option<f64>"
from mlx_vlm import load, generate

model, processor = load("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit")
messages = [{"role": "user", "content": "Write in Rust: pub fn median(xs: &mut [i64]) -> Option<f64>"}]
prompt = processor.tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
out = generate(model, processor, prompt, max_tokens=4096, temperature=0.6, top_p=0.95, top_k=20)
print(out.text)

Recommended:

  • Use no system prompt. Training and evaluation used none. A long style prompt sent the base model into runaway reasoning.
  • Leave thinking on and cap the output at about 4096 tokens. Answers take about 1.2k (easy tasks) to 1.8k tokens (hard ones) on average. An attempt that runs to the cap is usually stuck in a reasoning loop, so resampling helps more than raising the cap.
  • Use temperature 0.6, top-p 0.95 and top-k 20. These are the settings behind the numbers above.
  • Check its code with the compiler when you can. Picking an answer that builds and is clippy-clean, and retrying with the error output, nearly doubles its pass rate (see With a compiler in the loop).

Training

Two LoRA fine-tunes of the 8-bit MLX model, merged by weight: 0.7 times the first fine-tune's weight update plus 0.3 times the second's. The merge is exact (the two low-rank updates are concatenated, rank 32 in all) and involves no further training. Both fine-tunes used LoRA rank 16, alpha 32, dropout 0.05, on the last 16 decoder layers, one epoch at learning rate 1e-4, batches of up to 4096 tokens, and the base's own answers to general, non-Rust prompts as replay.

  • First fine-tune (the first release): 739 samples, 784k tokens. 238 of the base model's own passing attempts at the training tasks, 277 self-repairs (the model fixing its own failed attempt, given the compiler or test output as the next turn), 175 passing traces from DeepSeek V4.1 Flash kept only when they matched the 4B's own length and style, and 88 replay answers. Validation loss 0.411.
  • Second fine-tune: 3,442 samples, 4.04M tokens, with 595 more generated training tasks weighted toward fixing, refactoring and test writing. About half are the model's own answers and repairs (874 and 881), half DeepSeek V4.1 Flash records on the tasks the 4B could not solve (812 answers, 815 repairs), with the teacher's reasoning weighted 0.3 in the loss so the model keeps its own way of reasoning. On its own it gained on easy and hard tasks and lost on dev; the 0.7 / 0.3 merge kept the gains without the loss.
  • Two merge weights were tried (0.5 and 0.7) and the better one kept, judged on these test tiers, so the result includes a small selection.
  • The loss covers only the final reply: in repair records, the failed attempt and the feedback are context, not training targets.

What the experiments behind this checkpoint showed:

  • The model learns best from its own style. Reference solutions with model-written reasoning made it worse (easy 41% to 27%): it imitated their dense style and compiled less often.
  • Unfiltered teacher traces also made it worse. Their long, second-guessing reasoning taught it to never leave thinking. Filtered to its own length and style, or with their reasoning down-weighted, they were safe.
  • More rounds of the same stopped helping. Training again on its own answers, on twice the data, or on the compiler loop's fixed answers gave no further gain. RL (GRPO, 60 steps with the test gate as the reward) raised the training reward without improving held-out tasks.

Limitations

  • It is a 4B model and Rust is hard for it. Many of its first answers still fail to compile, mostly with type, trait-bound and method errors in its own code (E0308, E0277, E0599), and harder multi-part tasks rarely pass (hard tier 8.1%). Use it with a compiler in the loop.
  • It still over-thinks sometimes. Cap the output, and resample when an answer gets stuck.
  • Only the text side was trained. The vision tower is the base's, unchanged and untested here.
  • The evaluation sets are small (29 to 82 tasks per tier), so each per-tier number has a wide margin. Treat the gains as modest.
  • It was trained and evaluated on standalone tasks: a function, a type or a small module checked by hidden tests, plus bug fixes, refactors and test-writing. It has not been measured on multi-file projects or long-context editing.

License

Apache 2.0, the license of the base model Qwen/Qwen3.5-4B.

Downloads last month
35
Safetensors
Model size
5B params
Tensor type
U32
路
BF16
路
F32
路
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for alan13367/Qwen3.5-4B-RustCoder-MLX-8bit

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(1,549)
this model