README: what each pack is and how it scores
Browse files
README.md
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- gguf
|
| 5 |
+
- lars
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# lars packs
|
| 9 |
+
|
| 10 |
+
Private backup of the lars-sdk dev packs. They were rebuilt on 2026-10-08 and 09, after the
|
| 11 |
+
machine that held the originals died. Each pack is in `packs/<name>/<version>/`:
|
| 12 |
+
|
| 13 |
+
- `model.gguf`: the fused LoRA model, imatrix-quantized.
|
| 14 |
+
- `calibration.json`: the calibrator, fitted on 4,120 jev-bench validation rows.
|
| 15 |
+
- `pack.json`: the manifest, with the base, the llama.cpp commit, sha256 sums and per-config scores.
|
| 16 |
+
|
| 17 |
+
Scores are on the jev-bench test split (22 configs, 200 rows each), through the pack:
|
| 18 |
+
|
| 19 |
+
| Pack | Base, checkpoint | Quant | Macro accuracy | Macro Brier | Lost original |
|
| 20 |
+
|---|---|---|---|---|---|
|
| 21 |
+
| `lars-0.6b-dev` 0.0.2 | Qwen/Qwen3-0.6B, final step 4,449 | Q4_K_M | 0.637 | 0.418 | 0.650 / 0.419 |
|
| 22 |
+
| `lars-0.6b-dev-step2500` 0.0.2 | Qwen/Qwen3-0.6B, step 2,500 (selected) | Q4_K_M | 0.625 | 0.433 | |
|
| 23 |
+
| `lars-2b-dev` 0.0.1 | Qwen/Qwen3.5-2B, final step 5,370 | Q4_K_M | 0.713 | 0.331 | 0.716 / 0.333 |
|
| 24 |
+
|
| 25 |
+
- **Training:** LoRA on corpus E. The 2B also trained on the typed-decisions train split; it
|
| 26 |
+
scores 0.757 on that benchmark's test split, against 0.759 for the lost pack.
|
| 27 |
+
- **The step-2,500 pack:** kept for comparison. The packs build from the final step.
|
| 28 |
+
- **`lars-8b-dev` 0.0.2** (Qwen/Qwen3-8B, Q3_K_M, was 0.748) is not rebuilt yet. It needs a
|
| 29 |
+
machine with more than 64 GB; lars-sdk's `HANDOFF.md` has the steps.
|
| 30 |
+
|
| 31 |
+
Apache-2.0, like the base models.
|