| --- |
| language: |
| - en |
| license: other |
| library_name: llama.cpp |
| pipeline_tag: text-generation |
| base_model: prism-ml/Bonsai-27B-gguf |
| tags: |
| - llama.cpp |
| - lora |
| - q1 |
| - q1_0 |
| - packed-quantization |
| - native-training |
| - checkpoint-resume |
| - reproducibility |
| - cuda |
| --- |
| |
| # Bonsai-27B packed-Q1 native LoRA training release |
|
|
| This is a reproducible engineering release for native LoRA training directly against the packed Q1_0 Bonsai-27B base model in a pinned Prism/Mintplex llama.cpp tree. |
| |
| The repository contains adapters, deterministic checkpoints, dataset shards, source changes, clean-build evidence, verification scripts, reports and an optional Linux/CUDA build. The 3.8 GB base GGUF is not redistributed. |
| |
| ## Release status |
| |
| | Stage | Result | |
| |---|---| |
| | Step 8: native multi-target packed-Q1 LoRA | PASS | |
| | Step 9: deterministic dataset pipeline | PASS | |
| | Step 10: native training and exact checkpointing | PASS | |
| | Step 11: correctness and quality acceptance | PASS | |
| | Step 12: performance and stability acceptance | PASS | |
| | Step 13: clean reproducibility build | PASS | |
| |
| ## What is proven |
| |
| - Packed Q1_0 base weights remain immutable. |
| - Six LoRA tensors are optimized without a persistent expanded base model. |
| - Checkpoint save/reload is exact. |
| - Interrupted accumulation resumes deterministically. |
| - Independent resume branches produced byte-identical checkpoints, adapters and state fingerprints. |
| - Load/unload, quick soak, RAM and VRAM checks passed. |
| - The patch applies to commit `7529fdaaf99ffdc5ca71ace9c7409a56b27ad92f` and selected targets build in a clean worktree. |
|
|
| ## LoRA targets |
|
|
| | Base tensor | Family | Rank | Alpha | Base shape | |
| |---|---:|---:|---:|---:| |
| | `blk.0.ssm_alpha.weight` | SSM | 4 | 8 | `48 × 5120` | |
| | `blk.11.attn_k.weight` | attention | 4 | 8 | `1024 × 5120` | |
| | `blk.0.ffn_down.weight` | FFN | 4 | 8 | `5120 × 17408` | |
|
|
| Total trainable LoRA values: **135,360**. |
|
|
| ## Quality result |
|
|
| This was a tiny proof run, not a quality fine-tune. |
|
|
| - Train mean-token NLL, full minus zero adapter: `0.00021834747786` |
| - Validation mean-token NLL, full minus zero adapter: `9.73401665583e-06` |
| - Validation improved: **no** |
|
|
| The adapter is numerically connected and stable, but the proof run is too small to establish a quality gain. |
|
|
| ## Performance snapshot |
|
|
| | Measurement | Result | |
| |---|---:| |
| | Base end-to-end generation | `3.692` tokens/s | |
| | Trained-adapter end-to-end generation | `3.379` tokens/s | |
| | Base fresh-process load | `0.817` GiB/s | |
| | Trained fresh-process load | `0.748` GiB/s | |
| | Peak process RSS | `3980.6` MiB | |
| | Peak observed GPU memory | `9972.0` MiB | |
| | Load/unload cycles | `3` | |
| | Quick soak | `192` requested tokens | |
| | Training microsteps/minute | `0.843330` | |
| | Training updates/minute | `0.562220` | |
|
|
| ## Primary artifacts |
|
|
| | Artifact | Size | SHA256 | |
| |---|---:|---| |
| | `step10_adapter_final.gguf` | 529.50 KiB | `6ba256d4e1836a177a8c7519d05a95639d47f4ee181fb6fcd438eee40805549f` | |
| | `multi_target_adapter_initial.gguf` | 529.50 KiB | `84d835c964c4a0ae32fed9be2106124ff9319b74317ba9cdb0ea21069150edb5` | |
| | `checkpoint_latest.p10ck` | 2.07 MiB | `1ba5545af5c33aacd7c1d2b395afe0e53e4fac5f1fd07ea56353c5a0c9fa64b9` | |
| | `quick_resume_adapter.gguf` | 529.50 KiB | `c6c163276a50c144315543a49ecdccd725608b5e3a653733a4a589369e0233d2` | |
| | `quick_resume_checkpoint.p10ck` | 2.07 MiB | `7c8e9b000eac8fa3ca60883b76f6a6c2ffe6126d97b3c861dfef70dcc4e3080c` | |
|
|
| ## Base model identity |
|
|
| - Repository: `prism-ml/Bonsai-27B-gguf` |
| - Revision: `0469926cd8878dbddb5b883740ab3df060058696` |
| - Required file: `Bonsai-27B-Q1_0.gguf` |
| - Bytes: `3803452480` |
| - SHA256: `17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0` |
|
|
| ```bash |
| python -m pip install -r reproduce/requirements.txt |
| python reproduce/download_models.py |
| python reproduce/restore_and_build.py |
| python reproduce/verify_release.py |
| ``` |
|
|
| ## Load the adapter |
|
|
| ```bash |
| ./binaries/linux-colab-cuda/llama-completion -m /path/to/Bonsai-27B-Q1_0.gguf --lora adapters/step10_adapter_final.gguf -p "User: Explain deterministic checkpoint resume.\nAssistant:" -n 64 --temp 0 -ngl 999 -no-cnv |
| ``` |
|
|
| ## Limitations |
|
|
| - This is a proof-of-training release, not a broadly trained production LoRA. |
| - Validation NLL was slightly worse by `9.73401665583e-06`. |
| - The Linux build is environment-specific. |
| - The quick soak is not an hours-long production endurance test. |
| - Upstream source and base-model licenses and terms apply. |
|
|
