File size: 4,404 Bytes
bf45b91 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 | ---
language:
- en
license: other
library_name: llama.cpp
pipeline_tag: text-generation
base_model: prism-ml/Bonsai-27B-gguf
tags:
- llama.cpp
- lora
- q1
- q1_0
- packed-quantization
- native-training
- checkpoint-resume
- reproducibility
- cuda
---
# Bonsai-27B packed-Q1 native LoRA training release
This is a reproducible engineering release for native LoRA training directly against the packed Q1_0 Bonsai-27B base model in a pinned Prism/Mintplex llama.cpp tree.
The repository contains adapters, deterministic checkpoints, dataset shards, source changes, clean-build evidence, verification scripts, reports and an optional Linux/CUDA build. The 3.8 GB base GGUF is not redistributed.
## Release status
| Stage | Result |
|---|---|
| Step 8: native multi-target packed-Q1 LoRA | PASS |
| Step 9: deterministic dataset pipeline | PASS |
| Step 10: native training and exact checkpointing | PASS |
| Step 11: correctness and quality acceptance | PASS |
| Step 12: performance and stability acceptance | PASS |
| Step 13: clean reproducibility build | PASS |
## What is proven
- Packed Q1_0 base weights remain immutable.
- Six LoRA tensors are optimized without a persistent expanded base model.
- Checkpoint save/reload is exact.
- Interrupted accumulation resumes deterministically.
- Independent resume branches produced byte-identical checkpoints, adapters and state fingerprints.
- Load/unload, quick soak, RAM and VRAM checks passed.
- The patch applies to commit `7529fdaaf99ffdc5ca71ace9c7409a56b27ad92f` and selected targets build in a clean worktree.
## LoRA targets
| Base tensor | Family | Rank | Alpha | Base shape |
|---|---:|---:|---:|---:|
| `blk.0.ssm_alpha.weight` | SSM | 4 | 8 | `48 × 5120` |
| `blk.11.attn_k.weight` | attention | 4 | 8 | `1024 × 5120` |
| `blk.0.ffn_down.weight` | FFN | 4 | 8 | `5120 × 17408` |
Total trainable LoRA values: **135,360**.
## Quality result
This was a tiny proof run, not a quality fine-tune.
- Train mean-token NLL, full minus zero adapter: `0.00021834747786`
- Validation mean-token NLL, full minus zero adapter: `9.73401665583e-06`
- Validation improved: **no**
The adapter is numerically connected and stable, but the proof run is too small to establish a quality gain.
## Performance snapshot
| Measurement | Result |
|---|---:|
| Base end-to-end generation | `3.692` tokens/s |
| Trained-adapter end-to-end generation | `3.379` tokens/s |
| Base fresh-process load | `0.817` GiB/s |
| Trained fresh-process load | `0.748` GiB/s |
| Peak process RSS | `3980.6` MiB |
| Peak observed GPU memory | `9972.0` MiB |
| Load/unload cycles | `3` |
| Quick soak | `192` requested tokens |
| Training microsteps/minute | `0.843330` |
| Training updates/minute | `0.562220` |
## Primary artifacts
| Artifact | Size | SHA256 |
|---|---:|---|
| `step10_adapter_final.gguf` | 529.50 KiB | `6ba256d4e1836a177a8c7519d05a95639d47f4ee181fb6fcd438eee40805549f` |
| `multi_target_adapter_initial.gguf` | 529.50 KiB | `84d835c964c4a0ae32fed9be2106124ff9319b74317ba9cdb0ea21069150edb5` |
| `checkpoint_latest.p10ck` | 2.07 MiB | `1ba5545af5c33aacd7c1d2b395afe0e53e4fac5f1fd07ea56353c5a0c9fa64b9` |
| `quick_resume_adapter.gguf` | 529.50 KiB | `c6c163276a50c144315543a49ecdccd725608b5e3a653733a4a589369e0233d2` |
| `quick_resume_checkpoint.p10ck` | 2.07 MiB | `7c8e9b000eac8fa3ca60883b76f6a6c2ffe6126d97b3c861dfef70dcc4e3080c` |
## Base model identity
- Repository: `prism-ml/Bonsai-27B-gguf`
- Revision: `0469926cd8878dbddb5b883740ab3df060058696`
- Required file: `Bonsai-27B-Q1_0.gguf`
- Bytes: `3803452480`
- SHA256: `17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0`
```bash
python -m pip install -r reproduce/requirements.txt
python reproduce/download_models.py
python reproduce/restore_and_build.py
python reproduce/verify_release.py
```
## Load the adapter
```bash
./binaries/linux-colab-cuda/llama-completion -m /path/to/Bonsai-27B-Q1_0.gguf --lora adapters/step10_adapter_final.gguf -p "User: Explain deterministic checkpoint resume.\nAssistant:" -n 64 --temp 0 -ngl 999 -no-cnv
```
## Limitations
- This is a proof-of-training release, not a broadly trained production LoRA.
- Validation NLL was slightly worse by `9.73401665583e-06`.
- The Linux build is environment-specific.
- The quick soak is not an hours-long production endurance test.
- Upstream source and base-model licenses and terms apply.
|