coder-lfm3-1.7b

LoRA SFT of LiquidAI/LFM3-1.7B on 70 reasoning traces, merged to bfloat16 and dequantized from the official 4-bit MLX checkpoint. One of three planned subsets (Coder / IndieDev / GenAgent) of a 250-row distill set.

This is epoch 1 of 3 — the checkpoint with the best held-out loss, not the last one.

Training

base	LiquidAI/LFM3-1.7B-MLX-4bit (4-bit affine g64, dequantized at merge time)
method	LoRA, rank 8, 8 of 24 layers, 2.753M trainable (0.159%)
data	xtools-at/Xiaomi-MiMo-2.6-Pro-High-Reasoning-250x, rows labelled code / CS / stats
rows	70 train prompts → 282 examples; 7 held-out prompts → 32 examples
sequence	2048 tokens, batch 1, lr 1e-5 (Adam), 282 iters (epoch 1 of a 846-iter run)
hardware	MacBook Air M1, 8 GB unified memory
throughput	194–211 tok/s, 2h11m for all 3 epochs

Long assistant traces were chunked, not truncated: each row's rendered conversation was split into 2048-token windows with 256 tokens of the previous window replayed as context, so every token of the source data was trained on exactly once per epoch. Base-model NLL on the raw data would otherwise only cover 24% of the rows.

Evaluation

Held-out token NLL (lower is better), all 32 held-out chunks:

model	Coder held-out	math rows	health rows
base (4-bit)	1.430	1.959	1.889
this model	1.288	1.045	1.649
epoch 3 (not released)	1.294	0.982	1.670

Code smoke test — generated code executed in a subprocess, single-argument functions probed against fixed cases: prime check 12/12, vowel removal 5/5, email regex 5/5. Base matches these scores exactly; the tasks are too easy to separate the two, so the gain here is style and trace format, not code ability.

No benchmark suite was run.

Usage

from mlx_lm import load, generate

model, tokenizer = load("AJ-Gonzalez/coder-lfm3-1.7b")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Write a Python function to check if a number is prime."}],
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=2048))

Expect long normative reasoning (The user asks… We need to…) inside <think> before the answer, matching the teacher traces. Requires mlx-lm >= 0.32.2; LFM3 support was merged upstream only recently — with an older release, drop mlx_lm/models/lfm3.py from ml-explore/mlx-lm#2016 into site-packages.

Weights are MLX-format safetensors as written by mlx_lm.fuse --dequantize (the repo keeps the base model's modeling_lfm3.py / configuration_lfm3.py, but no torch conversion was produced). Load with mlx-lm on Apple silicon.

GGUF

coder-lfm3-1.7b-Q4_K_M.gguf (1.04 GiB, 5.05 bpw) is in this repo. It was converted from the merged MLX bf16 with llama.cpp's convert_hf_to_gguf.py and quantized with llama-quantize.

llama-cli -m coder-lfm3-1.7b-Q4_K_M.gguf \
  -p "Write a Python function to check if a number is prime." -n 512 --temp 0

Measured on the M1: 38.4 tok/s for a 96-token greedy smoke test (short run, not a benchmark). The MLX source needs the LFM3 converter's short-conv fix (squeeze() rather than squeeze(1)) so blk.N.shortconv.conv.weight reaches the expected 2D shape.

Limitations

• 70 source prompts, 3 topics, one epoch. This will not teach a coding model; it biases an existing one toward a reasoning-then-answer format.
• The validation set is 7 prompts (32 chunks); the loss differences between epochs 1–3 are within noise. Epoch 3 scored slightly better on math, worse on Coder.
• Traces were produced by xiaomi/mimo-v2.6-pro (the data card names TeichAI/claude-4.5-opus-high-reasoning-250x as the origin), so the target style is second-hand.
• No safety tuning, no tool-calling training, no long-context (>2048) training.

Licence

Base model: LFM Open License v1.0 (LiquidAI/LFM3-1.7B). Training data: xtools-at/Xiaomi-MiMo-2.6-Pro-High-Reasoning-250x, listed as licence "other" on the Hub — check that dataset's terms before redistributing the weights commercially.