la-math-v9 — Qwen3-4B fine-tuned for linear algebra

A LoRA fine-tune of Qwen3-4B specialized in linear algebra: determinants, inverses, eigenvalues, systems of equations, rank/spaces, factorizations, matrix operations, and linear algebra theory (definitions, theorems, examples).

Best LA specialist so far in the la-math series (v5 → fix2 → v9).

Evaluation

416-pair held-out linear algebra set, SymPy-graded with an identical harness across all models (284 items computationally gradable; the rest are theory/definitional).

Model	Accuracy (regraded)
la-math-v9	75.0%
la-math-v5-fix2	73.5%
la-math-v5	72.8%
la-math-v5-fix1	71.4%
Qwen3-4B base	67.8%

Category breakdown (v9): matrix_ops 100%, orthogonality 100%, lin_trans 88%, inverses 85%, determinants 84%, rank_spaces 67%, systems 65%, least_squares 60%, eigen 58%.

Training

• Base: Qwen3-4B (MLX, 4-bit quantized)
• Data: la_data_v8 — 9,703 training pairs (curated from v5 + gap-fill items + PDF-extracted theorems/definitions/examples from 8 standard linear algebra textbooks), audited self-consistent, answers capped at 1,024 tokens
• Method: LoRA, 16 layers, rank from mlx_lm defaults (0.182% trainable params)
• Fresh run: 9,703 iters (1 epoch), batch size 1, LR 1e-5, max seq 1,024, grad-checkpoint, mask-prompt
• Fused from the iter-7000 checkpoint (val-loss floor 0.047) — late-epoch overfit avoided
• Fused + requantized to 4-bit (group size 64)

Usage

MLX (mlx-lm):

from mlx_lm import load, generate
model, tokenizer = load("j01001100/la-math-v9")
response = generate(model, tokenizer, prompt="Find the inverse of [[2,1],[5,3]]", max_tokens=768)

LM Studio: local model, 2.28 GB, runs on Apple Silicon. For a tutor that verifies its own answers, wrap with inference-time SymPy verification (generate → verify → retry).

License & attribution

Apache-2.0. Base model: Qwen/Qwen3-4B (Apache-2.0). Fine-tune by jason (j01001100).