la-math-v5-fix2 — Linear Algebra Specialist (MLX)

A Qwen3-4B fine-tuned with LoRA (3 rounds, 7.5k + 1.5k + 1.5k iters) into a linear-algebra specialist. Fused and requantized to a standalone MLX model (2.1 GB, 4-bit, group size 64) — loads with mlx_lm.load(path), no adapter file.

Benchmark

416 held-out linear algebra pairs, SymPy-verified grading, ~67% auto-graded computational core (rest are theory/concept items):

Category	Base Qwen3-4B	This model
Overall graded	67.8%	73.5%
determinants	79%	88%
inverses	70%	82%
matrix ops	93%	100%
least squares	40%	70%
orthogonality	89%	100%
linear transforms	75%	88%
systems	47%	50%
eigen	58%	60%

Training data

• 8,337 base pairs (SymPy-verified generated, textbook extraction from 11 books, hand-crafted) — zero benchmark test-set contamination
• +1,866 targeted gap-fill items from failure analysis (eigen char-poly procedure, LU, 4x4 determinants, 3x3 Gauss-Jordan, parametric systems, null-space bases)

Usage

from mlx_lm import load, generate

model, tokenizer = load("jason/la-math-v5-fix2")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Compute the determinant of \\[ A = \\begin{bmatrix} 1 & 2 \\\\ 3 & 4 \\end{bmatrix} \\]"}],
    tokenize=False, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=1024))

Answers use LaTeX matrices (\begin{bmatrix}), show step-by-step work, and finish with a boxed final answer (\boxed{...}).