---
title: THOX Mesh Node
emoji: 🕸️
colorFrom: indigo
colorTo: purple
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
short_description: OpenAI-compatible THOX node in the MeshStack mesh
---

# THOX Mesh Node — HF CPU Space

A free-tier CPU node in the THOX mesh. It serves an OpenAI-compatible endpoint
from a THOX GGUF and self-registers so ThoxRoute can route traffic to it.

## Endpoints

| Path | Purpose |
| --- | --- |
| `POST /v1/chat/completions` | OpenAI-compatible chat completion |
| `GET /v1/models` | Advertised model |
| `GET /health` | Liveness, mesh registration state, live telemetry |

## Quick check

```bash
curl -s https://<space-host>/health | jq .

curl -s https://<space-host>/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hello from the THOX mesh."}],"max_tokens":48}'
```

## Configuration

Set these as Space **Variables**, except `HF_TOKEN` which must be a **Secret**.

| Variable | Default | Meaning |
| --- | --- | --- |
| `THOX_MODEL_REPO` | `Thox-ai/ThoxMini-3B` | Hub repo holding the GGUF |
| `THOX_MODEL_FILE` | `thoxmini-3b-Q4_K_M.gguf` | GGUF filename |
| `THOX_MODEL_ID` | `thoxmini-3b` | Model id advertised to the mesh |
| `THOX_ALIAS` | `thoxmini-3b-space` | Endpoint alias, unique per device |
| `THOX_CAPABILITIES` | `chat,completion` | Capability flags ThoxRoute matches on |
| `THOX_CONTROLLER_URL` | *(unset)* | Mesh controller to register with |
| `THOX_MESH_DEVICE_ID` | *(unset)* | Paired MeshStack device UUID |
| `THOX_MESH_DEVICE_KEY` | *(unset)* | **Secret.** Base64 Ed25519 seed |
| `THOX_N_GPU_LAYERS` | `auto` | `auto` \| `0` (force CPU) \| `-1` (all) \| an integer |
| `HF_TOKEN` | *(unset)* | **Secret.** Required — the model repo is private |

## CPU and GPU paths

The same agent runs on both. Nothing needs reconfiguring to move between them —
`THOX_N_GPU_LAYERS=auto` resolves per machine.

| | CPU (this Space, free Colab) | GPU (Colab T4/L4/A100) |
| --- | --- | --- |
| Model | ThoxMini-3B Q4_K_M (~2 GB) | ThoxMythos-9B Q4_K_M (~6.2 GB) |
| Offload | `n_gpu_layers=0` | `n_gpu_layers=-1` (all, clamped to VRAM) |
| llama.cpp build | CPU wheel/build | rebuilt with `-DGGML_CUDA=ON` |
| Throughput | ~1.5 tok/s | roughly an order of magnitude faster |

**A CPU-only environment is fully supported, not a degraded one.** GPU detection
goes through `thoxmesh_node.gpu`, which treats a missing `nvidia-smi`, a broken
driver and a hung probe all as "no GPU" and never raises. If a CUDA build is
asked to offload and cannot, the node retries CPU-only rather than refusing to
serve — a slower mesh member beats an absent one.

Check any machine before starting:

```bash
python -m thoxmesh_node doctor
```

It reports Python, the crypto/web stack, llama-cpp-python, `HF_TOKEN`, GPU state,
cloudflared and the resolved config. Exit 0 means the node can serve.

## Why this Space registers with the controller, not MeshStack v2 directly

`meshstack-device-v2` validates every `base_url` against `isPrivateHost` and
accepts only RFC1918, loopback, link-local, CGNAT and `.local` addresses. An
`*.hf.space` origin is public and is refused by that rule — deliberately, as an
anti-SSRF control on the mesh.

The node detects that specific refusal and falls back to the controller
transport rather than retrying a request that can never succeed. See
`ARCHITECTURE.md` for the contract and the proposed resolution.

## Cost

Runs on the free `cpu-basic` tier and must stay there. ThoxMini-3B Q4_K_M is
~2 GB and fits the 16 GB / 2 vCPU allocation; nothing here requests a GPU.

Expect roughly **1.5 tokens/second**. That is the honest free-CPU number, and it
is why `THOX_MAX_TOKENS_CAP` defaults to 256 — the node reports its real latency
as telemetry, so ThoxRoute ranks it behind faster nodes rather than being
surprised by it.

`ThoxLLM-327M-v2` was tried first because it is far smaller. It loads and serves
correctly but its output is degenerate (`"The capital of France is"` →
`"]]]]]]]]]]"`): a 327M base model with no instruction tuning and no chat
template. Mechanically fine, useless for a demo.

`thox-micro-125m` is not published on the Hub at all. It was the package default
until v1.1.0, which meant anything relying on defaults pointed at nothing.