Run more models than your GPU can hold.
CapyCTL deploys, parks, wakes and switches models on vLLM, SGLang and TensorFold, on GPU machines you own, behind one OpenAI-compatible endpoint.
v0.1.2 · Apache-2.0 · one binary, written in Rust · Linux x86-64 and ARM64 · NVIDIA GPUs
$ capyctl list deployments
NAME STATE READY REVISION HOSTS
coding-small ready 1/1 3 gpu-box
chat-large ready 1/1 1 workstation
vision-small parked 0/1 2 gpu-box
Example output. A parked model releases its GPU memory and wakes on the next request for it.
- More models than memoryMost GPU machines hold one or two models at a time. CapyCTL parks the idle ones and wakes the one a request asks for, making room by parking or stopping an idle model.
- One endpointOpenAI-compatible chat completions with streaming and tool calls, behind an API key. Your apps keep one base URL and pick a model by name.
- Your engines, your weightsRegister the vLLM, SGLang or TensorFold installation you already have. Models come from a directory or from the Hub, pinned to a commit when you deploy.
## Quick start
On Linux with an NVIDIA GPU and a vLLM, SGLang or TensorFold environment:
```bash
curl -fsSL https://edurdias.github.io/capyctl/install.sh | sh
capyctl engine add ~/venvs/vllm # a vLLM, SGLang or TensorFold environment
capyctl start standalone
```
In another shell, deploy a model from the Hub. Three fields are enough:
```yaml
# qwen3-4b.yaml
name: qwen3-4b
engine: vllm
model: {hf: Qwen/Qwen3-4B-Instruct-2507}
```
```bash
capyctl deploy model --file qwen3-4b.yaml --activate --wait
```
Any OpenAI client can now send `qwen3-4b` as the model name ([Make a request](https://edurdias.github.io/capyctl/docs/requests/)). To free its GPU memory, run `capyctl park deployment qwen3-4b`; the next request wakes it. One server can also drive several GPU machines ([Run on several machines](https://edurdias.github.io/capyctl/docs/several-machines/)).
## Measured recipes
Deployment files run through CapyCTL, each sized so the model fits in 32 GiB once ready. Measured on a GB10 on 2026-10-07.
| Model and checkpoint | Engine | Decode, one stream | More streams |
|---|---|---|---|
| Qwen3.6-35B-A3B, MLX 4-bit, MTP drafts | [TensorFold 0.6.5](https://github.com/edurdias/capyctl-recipes/tree/main/tensorfold/qwen3.6-35b-a3b-mlx-4bit-32gib-gb10) | 164 tokens/s | 368 tokens/s at 8 streams |
| Qwen3.6-35B-A3B, NVFP4 | [vLLM 0.30.0](https://github.com/edurdias/capyctl-recipes/tree/main/vllm/qwen3.6-35b-a3b-nvfp4-32gib-gb10) | 76.7 tokens/s | 180 tokens/s at 4 streams |
| Qwen3.6-35B-A3B, NVFP4 | [SGLang 0.5.21](https://github.com/edurdias/capyctl-recipes/tree/main/sglang/qwen3.6-35b-a3b-nvfp4-32gib-gb10) | 83.5 tokens/s | 186 tokens/s at 4 streams |
| Nemotron 3.5 Lightning 30B-A3B, MLX 4-bit, MTP drafts | [TensorFold 0.6.5](https://github.com/edurdias/capyctl-recipes/tree/main/tensorfold/nemotron-3.5-lightning-30b-a3b-mlx-4bit-32gib-gb10) | 137 tokens/s | one request at a time |
| Nemotron 3.5 Lightning 30B-A3B, NVFP4 | [vLLM 0.30.0](https://github.com/edurdias/capyctl-recipes/tree/main/vllm/nemotron-3.5-lightning-30b-a3b-nvfp4-32gib-gb10) | 69.9 tokens/s | 164 tokens/s at 4 streams |
| Nemotron 3.5 Lightning 30B-A3B, NVFP4 | [SGLang 0.5.21](https://github.com/edurdias/capyctl-recipes/tree/main/sglang/nemotron-3.5-lightning-30b-a3b-nvfp4-32gib-gb10) | 69.9 tokens/s | 163 tokens/s at 4 streams |
The TensorFold and NVFP4 rows serve different checkpoints, so they are not a one-to-one engine comparison. Each recipe lists the model revision, the commands, ready times, memory and a capyctl-bench report; [the recipes repository](https://github.com/edurdias/capyctl-recipes) has gpt-oss-20b, Gemma 4, Qwen3.8-27B and more.
## In this organization
Checkpoints converted so a recipe can serve them. Each card says what changed from the original and how it was made.
- [FrogNano-4B-2609-MLX-4bit](https://huggingface.co/capyctl/FrogNano-4B-2609-MLX-4bit): MLX 4-bit, text only, for TensorFold on NVIDIA GPUs.
- [FrogNano-4B-2609-NVFP4](https://huggingface.co/capyctl/FrogNano-4B-2609-NVFP4): NVFP4 with FP8 attention, for vLLM, SGLang and TensorFold.
## Status
0.1.2 is early software: one GPU per model, CLI only, and breaking changes are possible between minor versions while the version starts with 0. Models that span several GPUs or machines are coming; they are not in a release yet. Questions and bug reports go to [GitHub](https://github.com/edurdias/capyctl/blob/main/SUPPORT.md). Licensed under [Apache-2.0](https://github.com/edurdias/capyctl/blob/main/LICENSE).
