| # k3-test — rental validation scripts for Kimi K3 local-inference |
|
|
| Companion to `../k3-rental-test-plan.md` (§4 runbook). Read **`PREP_REPORT.md` first** |
| — it contains the pre-flight verification results and two important deltas from the plan |
| (trunk VRAM reality + how "RAM-resident" is reinterpreted under mmap streaming). |
| |
| ## Box setup (vast.ai, ~15 min) |
| |
| ```bash |
| # rent per plan §2, ssh in, then: |
| git clone <this-repo or rsync the dir> k3-test && cd k3-test |
| tmux new -s k3 # everything long-running inside tmux |
| ./run_all.sh # setup → K3 download (bg) → recon → smoke gate → E0–E8 → pack |
| ``` |
| |
| Useful variants: |
| |
| ```bash |
| SKIP_DOWNLOAD=1 ./run_all.sh # model already on disk |
| EXP_LIST="E1 E2 E3" ./20_k3_experiments.sh # subset of experiments |
| K3_INCLUDE="*UD-IQ1_S*" K3_SUBDIR=UD-IQ1_S ./01_download_k3.sh # 610 GB fallback quant |
| ``` |
| |
| ## File map |
| |
| | file | what it does | |
| |---|---| |
| | `config.env` | all knobs: paths, pinned llama.cpp commit, quant selector, TSPLIT, filler list | |
| | `setup.sh` | apt deps, hf CLI + hf_transfer, clone+build llama.cpp (pinned, CUDA) | |
| | `00_recon.sh` | dmidecode/lscpu/NUMA/fio/sysbench RAM bw/GPU topo — free home-build data | |
| | `01_download_k3.sh` | K3 GGUF download w/ retries + verification (backgroundable) | |
| | `02_download_smoke.sh` | Qwen3-30B-A3B Q4_K_M (~18.6 GB) | |
| | `10_smoke.sh` | 5-case pipeline gate (GPU-resident, -cmoe, -ncmoe, NUMA×2, CPU-only bw) | |
| | `20_k3_experiments.sh` | E0–E8 per plan §4 Phase 2 (see header comment for mmap reinterpretation) | |
| | `30_teardown.sh` | tar logs; prints off-box copy + destroy reminders (never destroys itself) | |
| | `run_all.sh` | the full §4 runbook with smoke gate | |
| | `prompts/` | fixed chat / agentic-coding / long-doc prompts (reproducibility) | |
| | `lib/common.sh` | monitors (free/iostat/nvidia-smi dmon), drop_caches, RAM filler, runner+parser | |
| |
| ## Results |
| |
| All runs land under `logs/<name>/` with `cmd.txt`, `run.log` (full llama.cpp output), |
| `summary.txt` (exit code, wall time, pp/tg timings), `mem.log`, `iostat.log`, `nvidia.log`. |
| `30_teardown.sh` tars everything; score it against plan §6 success criteria. |
| |
| ## Interpreting the headline numbers |
| |
| - `k3_E1b_warm` tokens/s ≈ **dual-socket DDR4 ceiling** for hot-set-resident decode. |
| Home (8ch, ~150–180 GB/s) ≈ 55–70% of the rental's dual-socket number — see plan §5 note. |
| - `k3_E1a_cold vs k3_E1b_warm` = SSD streaming penalty with default readahead. |
| - `k3_E4_filler*_warm` sweep = hit-rate/cache-size curve (the E4 knee). |
| - `iostat.log` bytes-read delta ÷ tokens decoded = **real bytes/token** — check it |
| against the plan's ~40–45 GB/token estimate; it re-derives every speed prediction. |
|
|
| ## Never-destroy-automatically |
|
|
| Billing is per-minute and the script will never destroy the instance. |
| `30_teardown.sh` prints the exact manual steps. Disk does not persist — pull the |
| tarball off-box **before** destroying. |
|
|
