adriandj3's picture
TernaryPilot v2: card and training kit (weights after training)
5c251fb verified
|
Raw History Blame Contribute Delete
2.35 kB

Training and evaluation kit for TernaryPilot v2

By Adrian Ciprian Iancu. These are the exact scripts used for the adapter in this repo.

file what it does
bonsai_hf.py rebuilds Ternary-Bonsai-2-9B exactly in PyTorch from the PQ2_0 GGUF (2-bit codes + Hadamard rotation), LoRA on top, GGUF LoRA export
tern_unpack.py unpacks the ternary codes
bonsai_bw.py large-GPU mode: unpack every ternary weight once (BF16 with BF16=1), checked against the packed path
vision_cache.py the vision tower and image processor (live encoding, or a cache)
make_data.py v1 data builder (AgentNet, Multimodal-Mind2Web), perceptual-hash exclusion of benchmark images
make_data_v2.py v2 data builder: GUI-360 + AgentNet Windows and macOS + the v1 file, parallel exclusion filter
train.py the trainer: time budget (HOURS), WSD schedule, shared image prefix for examples of the same screenshot
banco_pubblico.py ScreenSpot-v2, ScreenSpot-Pro, GUI-360 grounding and actions; base and any number of adapters on one llama.cpp server

Environment: BONSAI_GGUF (the PQ2_0 GGUF), MMPROJ (BF16 vision projector), QWEN_SRC (folder with the Qwen3.5-9B config and tokenizer), GGUF_PY (the gguf-py folder of the PrismML llama.cpp fork).

# data
python make_data.py --mind2web "mind2web/data/train-*.parquet" --images m2w-img --exclude "bench/**/*.png" --out data/m2w.jsonl
python make_data_v2.py --gui360 GUI-360 --agentnet-jsonl agentnet_win_mac_18k.jsonl --agentnet-images agentnet/imgs \
    --v1-jsonl data/m2w.jsonl --exclude "bench/**/*.png" --out data/train_v2.jsonl --max-gui360 15000 --max-agentnet 10000
# training (one 96 GB GPU)
DATA=data/train_v2.jsonl RUN_DIR=run/pilot-v2 HOURS=2.5 RANK=64 ALPHA=128 LR=1e-4 ACCUM=8 \
    DENSE=1 BF16=1 VISION_LIVE=1 EMPTY_CACHE=0 python train.py
# benchmarks: base, v2 and v1 on one server
python banco_pubblico.py --server llama-server --model Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf \
    --mmproj Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf --lora v2.gguf v1.gguf --names v2 v1 \
    --bench ss2 --data ScreenSpot-v2 --out banco-ss2.jsonl --parallel 4

The same kit trains on a GTX 1080 (8 GB) without DENSE; the 1080 version and its CUDA kernel are in bonsai2-ternary-pascal.