---
language:
- en
license: apache-2.0
tags:
- mlx
- linear-algebra
- math
- qwen3
- lora
base_model: Qwen/Qwen3-4B
---

# la-math-v9 — Qwen3-4B fine-tuned for linear algebra

A LoRA fine-tune of Qwen3-4B specialized in linear algebra: determinants, inverses,
eigenvalues, systems of equations, rank/spaces, factorizations, matrix operations,
and linear algebra theory (definitions, theorems, examples).

**Best LA specialist so far in the la-math series (v5 → fix2 → v9).**

## Evaluation

416-pair held-out linear algebra set, SymPy-graded with an identical harness across all models
(284 items computationally gradable; the rest are theory/definitional).

| Model | Accuracy (regraded) |
|---|---|
| **la-math-v9** | **75.0%** |
| la-math-v5-fix2 | 73.5% |
| la-math-v5 | 72.8% |
| la-math-v5-fix1 | 71.4% |
| Qwen3-4B base | 67.8% |

Category breakdown (v9): matrix_ops 100%, orthogonality 100%, lin_trans 88%, inverses 85%,
determinants 84%, rank_spaces 67%, systems 65%, least_squares 60%, eigen 58%.

## Training

- Base: Qwen3-4B (MLX, 4-bit quantized)
- Data: `la_data_v8` — 9,703 training pairs (curated from v5 + gap-fill items + PDF-extracted
  theorems/definitions/examples from 8 standard linear algebra textbooks), audited self-consistent,
  answers capped at 1,024 tokens
- Method: LoRA, 16 layers, rank from mlx_lm defaults (0.182% trainable params)
- Fresh run: 9,703 iters (1 epoch), batch size 1, LR 1e-5, max seq 1,024, grad-checkpoint, mask-prompt
- Fused from the **iter-7000 checkpoint** (val-loss floor 0.047) — late-epoch overfit avoided
- Fused + requantized to 4-bit (group size 64)

## Usage

MLX (mlx-lm):

```python
from mlx_lm import load, generate
model, tokenizer = load("j01001100/la-math-v9")
response = generate(model, tokenizer, prompt="Find the inverse of [[2,1],[5,3]]", max_tokens=768)
```

LM Studio: local model, 2.28 GB, runs on Apple Silicon. For a tutor that verifies its own
answers, wrap with inference-time SymPy verification (generate → verify → retry).

## License & attribution

Apache-2.0. Base model: Qwen/Qwen3-4B (Apache-2.0). Fine-tune by jason (j01001100).