Instructions to use TessaCoil/K3-Stuff with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use TessaCoil/K3-Stuff with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
llama cli -hf TessaCoil/K3-Stuff:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
llama cli -hf TessaCoil/K3-Stuff:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf TessaCoil/K3-Stuff:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf TessaCoil/K3-Stuff:Q8_0

Use Docker

docker model run hf.co/TessaCoil/K3-Stuff:Q8_0

LM Studio
Jan
Ollama
How to use TessaCoil/K3-Stuff with Ollama:
```
ollama run hf.co/TessaCoil/K3-Stuff:Q8_0
```
Unsloth Desktop

How to use TessaCoil/K3-Stuff with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TessaCoil/K3-Stuff:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "TessaCoil/K3-Stuff:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use TessaCoil/K3-Stuff with Docker Model Runner:
```
docker model run hf.co/TessaCoil/K3-Stuff:Q8_0
```

Lemonade

How to use TessaCoil/K3-Stuff with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull TessaCoil/K3-Stuff:Q8_0

Run and chat with the model

lemonade run user.K3-Stuff-Q8_0

List all available models

lemonade list

Hermes Agent

How to use TessaCoil/K3-Stuff with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TessaCoil/K3-Stuff:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default TessaCoil/K3-Stuff:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use TessaCoil/K3-Stuff with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TessaCoil/K3-Stuff:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "TessaCoil/K3-Stuff:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

K3-Stuff / scripts /README.md

TessaCoil

Upload folder using huggingface_hub

ddf8c5b verified 3 days ago

preview code

Raw

History Blame Contribute Delete

2.94 kB

k3-test — rental validation scripts for Kimi K3 local-inference

Companion to ../k3-rental-test-plan.md (§4 runbook). Read PREP_REPORT.md first — it contains the pre-flight verification results and two important deltas from the plan (trunk VRAM reality + how "RAM-resident" is reinterpreted under mmap streaming).

Box setup (vast.ai, ~15 min)

# rent per plan §2, ssh in, then:
git clone <this-repo or rsync the dir> k3-test && cd k3-test
tmux new -s k3          # everything long-running inside tmux
./run_all.sh            # setup → K3 download (bg) → recon → smoke gate → E0–E8 → pack

Useful variants:

SKIP_DOWNLOAD=1 ./run_all.sh                 # model already on disk
EXP_LIST="E1 E2 E3" ./20_k3_experiments.sh   # subset of experiments
K3_INCLUDE="*UD-IQ1_S*" K3_SUBDIR=UD-IQ1_S ./01_download_k3.sh   # 610 GB fallback quant

File map

file	what it does
`config.env`	all knobs: paths, pinned llama.cpp commit, quant selector, TSPLIT, filler list
`setup.sh`	apt deps, hf CLI + hf_transfer, clone+build llama.cpp (pinned, CUDA)
`00_recon.sh`	dmidecode/lscpu/NUMA/fio/sysbench RAM bw/GPU topo — free home-build data
`01_download_k3.sh`	K3 GGUF download w/ retries + verification (backgroundable)
`02_download_smoke.sh`	Qwen3-30B-A3B Q4_K_M (~18.6 GB)
`10_smoke.sh`	5-case pipeline gate (GPU-resident, -cmoe, -ncmoe, NUMA×2, CPU-only bw)
`20_k3_experiments.sh`	E0–E8 per plan §4 Phase 2 (see header comment for mmap reinterpretation)
`30_teardown.sh`	tar logs; prints off-box copy + destroy reminders (never destroys itself)
`run_all.sh`	the full §4 runbook with smoke gate
`prompts/`	fixed chat / agentic-coding / long-doc prompts (reproducibility)
`lib/common.sh`	monitors (free/iostat/nvidia-smi dmon), drop_caches, RAM filler, runner+parser

Results

All runs land under logs/<name>/ with cmd.txt, run.log (full llama.cpp output), summary.txt (exit code, wall time, pp/tg timings), mem.log, iostat.log, nvidia.log. 30_teardown.sh tars everything; score it against plan §6 success criteria.

Interpreting the headline numbers

k3_E1b_warm tokens/s ≈ dual-socket DDR4 ceiling for hot-set-resident decode. Home (8ch, ~150–180 GB/s) ≈ 55–70% of the rental's dual-socket number — see plan §5 note.
k3_E1a_cold vs k3_E1b_warm = SSD streaming penalty with default readahead.
k3_E4_filler*_warm sweep = hit-rate/cache-size curve (the E4 knee).
iostat.log bytes-read delta ÷ tokens decoded = real bytes/token — check it against the plan's ~40–45 GB/token estimate; it re-derives every speed prediction.

Never-destroy-automatically

Billing is per-minute and the script will never destroy the instance. 30_teardown.sh prints the exact manual steps. Disk does not persist — pull the tarball off-box before destroying.