json-extractor-demo

LoRA adapter for mlx-community/Qwen2.5-0.5B-Instruct-4bit, trained locally on Apple Silicon with mlx-lm by the local-model-factory pipeline (prepare -> train -> eval-base -> eval-tuned -> gate -> publish).

Task: Reference task: free-text note -> strict JSON by fixed rules. Synthetic + seeded; exists to prove the pipeline end to end.

Gate decision: PROMOTE

• field_accuracy: 0.1667 -> 1.0 (+0.8333) meets min improvement 0.05
• json_validity: 1.0 -> 1.0 (no meaningful regression)
• exact_match: 0.0 -> 1.0 (no meaningful regression)

Held-out evaluation (same test split, deterministic)

metric	base model	base + this adapter
exact_match	0.000	1.000
examples	40.000	40.000
field_accuracy	0.167	1.000
json_examples	40.000	40.000
json_validity	1.000	1.000
mean_latency_s	0.133	0.164

Training

• iterations: 200, batch size: 4, learning rate: 0.0001, LoRA layers: 8, seed: 7
• final validation loss: 0.093
• data: 400 raw examples → 320 train / 40 valid / 40 test (seed 20260701, 0 duplicates dropped)
• data source: json-extractor-demo task (tasks/json-extractor-demo)
• data ledger: json-extractor-demo-synthetic

Use it

pip install mlx-lm

mlx_lm.generate --model mlx-community/Qwen2.5-0.5B-Instruct-4bit \
  --adapter-path <local-download-of-this-repo> \
  --prompt "..."

The full run artifacts (train config, both eval reports, gate decision, data manifest) are included in this repo so the numbers above can be audited against exactly what produced them.