Instructions to use TessaCoil/K3-Stuff with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use TessaCoil/K3-Stuff with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
llama cli -hf TessaCoil/K3-Stuff:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
llama cli -hf TessaCoil/K3-Stuff:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf TessaCoil/K3-Stuff:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf TessaCoil/K3-Stuff:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf TessaCoil/K3-Stuff:Q8_0

Use Docker

docker model run hf.co/TessaCoil/K3-Stuff:Q8_0

LM Studio
Jan
Ollama
How to use TessaCoil/K3-Stuff with Ollama:
```
ollama run hf.co/TessaCoil/K3-Stuff:Q8_0
```
Unsloth Desktop

How to use TessaCoil/K3-Stuff with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TessaCoil/K3-Stuff:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "TessaCoil/K3-Stuff:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use TessaCoil/K3-Stuff with Docker Model Runner:
```
docker model run hf.co/TessaCoil/K3-Stuff:Q8_0
```

Lemonade

How to use TessaCoil/K3-Stuff with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull TessaCoil/K3-Stuff:Q8_0

Run and chat with the model

lemonade run user.K3-Stuff-Q8_0

List all available models

lemonade list

Hermes Agent

How to use TessaCoil/K3-Stuff with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TessaCoil/K3-Stuff:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default TessaCoil/K3-Stuff:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use TessaCoil/K3-Stuff with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TessaCoil/K3-Stuff:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "TessaCoil/K3-Stuff:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

K3-Stuff / scripts /TESTONE_RESULTS.md

TessaCoil

Upload folder using huggingface_hub

ddf8c5b verified 3 days ago

preview code

Raw

History Blame Contribute Delete

3.98 kB

Test One results — vast.ai instance 48699333 (2026-08-25 evening)

Box: dual E5-2660 v3 (40 threads), 503 GB RAM, 2× RTX 3090 24 GB, 32 GB disk, ~930/845 Mbps. Image: vast llama.cpp derivative. Purpose: SSH + scripts + build pipeline validation on the cheap box. All passed.

SSH connectivity (the thing Test One answered)

Works end-to-end: ssh -p 50990 root@173.75.47.72 with the account ed25519 key.
Key is passphrase-protected → must be in ssh-agent (ssh-add ~/.ssh/id_ed25519) for scripted access; plain ssh -i in non-interactive shells fails silently as "Permission denied (publickey)". This was the only connectivity false alarm.
rsync/scp both directions verified (scripts pushed, logs pulled).

Build & run

Prebuilt /opt/llama.cpp/cuda-12.8/llama-cli is v1 — useless; built pinned master d222767c7 (10630) successfully. One extra step needed: vast's llama.cpp image lacks cuBLAS dev headers → apt install libcublas-dev-12-9 (now baked into setup.sh, version auto-detected from nvcc).
Smoke gate PASSED on Qwen3-4B Q4_K_M stand-in (disk only had 20 GB free; the real 30B smoke gate still runs on the primary box).
- GPU decode ~185 tok/s; prompt eval ~3.3–3.4K tok/s; llama-bench pp512 8.4K, pp2048 12.5K, tg32 214 t/s. CPU-only decode 8.2 t/s.
New metrics plumbing works: logs/metrics.tsv now records per-run exit, wall, pp_tps, tg_tps, disk_read_mb (cgroup io.stat — real storage reads, cache hits excluded), and model page-cache residency via mincore. llama-cli's compact [ Prompt: X t/s | Generation: Y t/s ] format is parsed.

Vast container constraints discovered (affect the K3 runbook!)

Unprivileged Docker, no kernel control:
- ❌ drop_caches (/proc/sys read-only) → "cold" expert-streaming states can't be forced repeatedly. E1a-cold is only truly cold ONCE, right after model load. --load-mode dio (E3) is the controllable proxy; scripts now log CONTAINER_LIMITED instead of failing.
- ❌ numactl mempolicy (S4b skipped: set_mempolicy: Operation not permitted) and /sys readahead writes (E2) — both now guarded with CONTAINER_LIMITED fallback.
- ✅ /dev/shm is tmpfs sized to half RAM (251 GB here) → E4 RAM-filler trick works.
- ✅ cgroup v2 io.stat gives per-run disk-read bytes (the "how often do experts come from disk" metric).
If true VM instances exist at similar prices for the primary rental, they'd remove the drop_caches/numactl limits — worth checking "VM" instances on vast.ai; the chemistry of E1/E2 is better there. Not required: E1-warm/E3-dio still answer the cache questions.

Numbers recorded (home-build recon, this box)

sysbench memory: ~52.5 GB/s (load-store, underestimates stream BW).
fio (file-based, container overlay): randread-1M ≈ 1.06 GB/s, seqread-4M ≈ 2.9 GB/s. Host SSD was advertised ~3.2 GB/s — overlay + shared host trims it.
dmidecode unavailable in container (no /dev/mem) — DIMM recon impossible on vast docker instances. (lscpu works: sockets visible.)

Machine state at teardown-decision time

Pinned llama.cpp built at /root/llama.cpp; k3-test at /root/k3-test; Qwen3-4B Q4_K_M (2.4 GB) at /root/models/qwen4b. Disk ~95% full (32 GB alloc).
Nothing on the box persists through recycle/destroy (no volume).
Logs archived locally: k3-test/logs-testone-48699333/.

Fixes made during test one (all committed)

setup.sh: env-assignment-before-$SUDO bug; libcublas-dev install.
lib/common.sh: drop_caches CONTAINER_LIMITED; io.stat+mincore run metrics; compact t/s parsing; metrics.tsv.
10_smoke.sh: S4b container-limit skip; S6 pp-scaling bench.
run_exp name-passing bug; -st replaces unregistered conversation flags.
00_recon.sh: file-based fio with free-space sizing.
New: 15_pp_scaling.sh, prompts/gen_prompts.sh (512/2K/8K graded prompts), E9 (K3 pp scaling), E1 length-graded decode runs.