File size: 2,353 Bytes
5c251fb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# Training and evaluation kit for TernaryPilot v2

By Adrian Ciprian Iancu. These are the exact scripts used for the adapter in this repo.

| file | what it does |
|---|---|
| `bonsai_hf.py` | rebuilds Ternary-Bonsai-2-9B exactly in PyTorch from the PQ2_0 GGUF (2-bit codes + Hadamard rotation), LoRA on top, GGUF LoRA export |
| `tern_unpack.py` | unpacks the ternary codes |
| `bonsai_bw.py` | large-GPU mode: unpack every ternary weight once (BF16 with `BF16=1`), checked against the packed path |
| `vision_cache.py` | the vision tower and image processor (live encoding, or a cache) |
| `make_data.py` | v1 data builder (AgentNet, Multimodal-Mind2Web), perceptual-hash exclusion of benchmark images |
| `make_data_v2.py` | v2 data builder: GUI-360 + AgentNet Windows and macOS + the v1 file, parallel exclusion filter |
| `train.py` | the trainer: time budget (`HOURS`), WSD schedule, shared image prefix for examples of the same screenshot |
| `banco_pubblico.py` | ScreenSpot-v2, ScreenSpot-Pro, GUI-360 grounding and actions; base and any number of adapters on one llama.cpp server |

Environment: `BONSAI_GGUF` (the PQ2_0 GGUF), `MMPROJ` (BF16 vision projector), `QWEN_SRC` (folder with the Qwen3.5-9B
config and tokenizer), `GGUF_PY` (the `gguf-py` folder of the PrismML llama.cpp fork).

```bash
# data
python make_data.py --mind2web "mind2web/data/train-*.parquet" --images m2w-img --exclude "bench/**/*.png" --out data/m2w.jsonl
python make_data_v2.py --gui360 GUI-360 --agentnet-jsonl agentnet_win_mac_18k.jsonl --agentnet-images agentnet/imgs \
    --v1-jsonl data/m2w.jsonl --exclude "bench/**/*.png" --out data/train_v2.jsonl --max-gui360 15000 --max-agentnet 10000
# training (one 96 GB GPU)
DATA=data/train_v2.jsonl RUN_DIR=run/pilot-v2 HOURS=2.5 RANK=64 ALPHA=128 LR=1e-4 ACCUM=8 \
    DENSE=1 BF16=1 VISION_LIVE=1 EMPTY_CACHE=0 python train.py
# benchmarks: base, v2 and v1 on one server
python banco_pubblico.py --server llama-server --model Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf \
    --mmproj Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf --lora v2.gguf v1.gguf --names v2 v1 \
    --bench ss2 --data ScreenSpot-v2 --out banco-ss2.jsonl --parallel 4
```

The same kit trains on a GTX 1080 (8 GB) without `DENSE`; the 1080 version and its CUDA kernel are in
[bonsai2-ternary-pascal](https://github.com/ArtyomITA/bonsai2-ternary-pascal).