File size: 2,943 Bytes
ddf8c5b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | # k3-test — rental validation scripts for Kimi K3 local-inference
Companion to `../k3-rental-test-plan.md` (§4 runbook). Read **`PREP_REPORT.md` first**
— it contains the pre-flight verification results and two important deltas from the plan
(trunk VRAM reality + how "RAM-resident" is reinterpreted under mmap streaming).
## Box setup (vast.ai, ~15 min)
```bash
# rent per plan §2, ssh in, then:
git clone <this-repo or rsync the dir> k3-test && cd k3-test
tmux new -s k3 # everything long-running inside tmux
./run_all.sh # setup → K3 download (bg) → recon → smoke gate → E0–E8 → pack
```
Useful variants:
```bash
SKIP_DOWNLOAD=1 ./run_all.sh # model already on disk
EXP_LIST="E1 E2 E3" ./20_k3_experiments.sh # subset of experiments
K3_INCLUDE="*UD-IQ1_S*" K3_SUBDIR=UD-IQ1_S ./01_download_k3.sh # 610 GB fallback quant
```
## File map
| file | what it does |
|---|---|
| `config.env` | all knobs: paths, pinned llama.cpp commit, quant selector, TSPLIT, filler list |
| `setup.sh` | apt deps, hf CLI + hf_transfer, clone+build llama.cpp (pinned, CUDA) |
| `00_recon.sh` | dmidecode/lscpu/NUMA/fio/sysbench RAM bw/GPU topo — free home-build data |
| `01_download_k3.sh` | K3 GGUF download w/ retries + verification (backgroundable) |
| `02_download_smoke.sh` | Qwen3-30B-A3B Q4_K_M (~18.6 GB) |
| `10_smoke.sh` | 5-case pipeline gate (GPU-resident, -cmoe, -ncmoe, NUMA×2, CPU-only bw) |
| `20_k3_experiments.sh` | E0–E8 per plan §4 Phase 2 (see header comment for mmap reinterpretation) |
| `30_teardown.sh` | tar logs; prints off-box copy + destroy reminders (never destroys itself) |
| `run_all.sh` | the full §4 runbook with smoke gate |
| `prompts/` | fixed chat / agentic-coding / long-doc prompts (reproducibility) |
| `lib/common.sh` | monitors (free/iostat/nvidia-smi dmon), drop_caches, RAM filler, runner+parser |
## Results
All runs land under `logs/<name>/` with `cmd.txt`, `run.log` (full llama.cpp output),
`summary.txt` (exit code, wall time, pp/tg timings), `mem.log`, `iostat.log`, `nvidia.log`.
`30_teardown.sh` tars everything; score it against plan §6 success criteria.
## Interpreting the headline numbers
- `k3_E1b_warm` tokens/s ≈ **dual-socket DDR4 ceiling** for hot-set-resident decode.
Home (8ch, ~150–180 GB/s) ≈ 55–70% of the rental's dual-socket number — see plan §5 note.
- `k3_E1a_cold vs k3_E1b_warm` = SSD streaming penalty with default readahead.
- `k3_E4_filler*_warm` sweep = hit-rate/cache-size curve (the E4 knee).
- `iostat.log` bytes-read delta ÷ tokens decoded = **real bytes/token** — check it
against the plan's ~40–45 GB/token estimate; it re-derives every speed prediction.
## Never-destroy-automatically
Billing is per-minute and the script will never destroy the instance.
`30_teardown.sh` prints the exact manual steps. Disk does not persist — pull the
tarball off-box **before** destroying.
|