| --- |
| title: CapyCTL |
| emoji: ⏯️ |
| colorFrom: yellow |
| colorTo: red |
| sdk: static |
| pinned: false |
| short_description: Run more models than your GPU can hold. |
| --- |
| |
| <style> |
| .cy { font-size: 16px; line-height: 1.6; } |
| .cy h2, .cy p, .cy ul, .cy li, .cy pre { margin: 0; padding: 0; } |
| .cy ul { list-style: none; } |
| .cy-banner { border-radius: 14px; overflow: hidden; margin: 0 0 24px; } |
| .cy-banner img { display: block; width: 100%; aspect-ratio: 4 / 1; object-fit: cover; margin: 0; } |
| .cy h2.cy-lede { font-size: 28px; line-height: 1.2; font-weight: 700; letter-spacing: -0.015em; margin: 0 0 8px; } |
| .cy-sub { font-size: 17px; line-height: 1.55; max-width: 42em; } |
| .cy-row { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; margin-top: 18px; } |
| .cy a.cy-btn { display: inline-flex; align-items: center; height: 34px; padding: 0 14px; border-radius: 8px; font-size: 14px; font-weight: 600; line-height: 1; text-decoration: none; white-space: nowrap; color: inherit; } |
| .cy a.cy-primary { background: #c2410c; color: #ffffff; border: 1px solid #c2410c; } |
| .cy-meta { font-family: "IBM Plex Mono", ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12.5px; letter-spacing: 0.02em; margin-top: 14px; } |
| .cy-split { display: flex; flex-wrap: wrap; gap: 22px; margin-top: 28px; align-items: flex-start; } |
| .cy-shot { flex: 1 1 430px; min-width: 0; } |
| .cy-term { background: #1c1917; color: #e7e5e4; border: 1px solid #44403c; border-radius: 12px; overflow: hidden; min-width: 0; } |
| .cy-term-bar { display: flex; gap: 6px; padding: 10px 12px; border-bottom: 1px solid #292524; } |
| .cy-dot { width: 10px; height: 10px; border-radius: 50%; display: inline-block; } |
| .cy .cy-term pre { background: transparent; color: #e7e5e4; margin: 0; padding: 10px 14px 12px; border: 0; border-radius: 0; font-family: "IBM Plex Mono", ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px; line-height: 1.75; white-space: pre; overflow-x: auto; } |
| .cy-p { color: #fb923c; } |
| .cy-h { color: #a8a29e; } |
| .cy-ready { color: #86d07a; } |
| .cy-parked { color: #a8a29e; } |
| .cy-cap { font-size: 13px; line-height: 1.5; margin-top: 8px; } |
| .cy-why { flex: 1 1 280px; display: flex; flex-direction: column; gap: 14px; } |
| .cy-why li { font-size: 15px; line-height: 1.55; } |
| .cy-why b { display: block; font-size: 16px; } |
| </style> |
|
|
| <div class="cy"> |
| <div class="cy-banner dark:hidden" style="background: #fafaf9; border: 1px solid #e7e5e4;"><img src="https://raw.githubusercontent.com/edurdias/capyctl/main/docs/brand/horizontal-light.png" alt="CapyCTL, control what runs next"></div> |
| <div class="cy-banner hidden dark:block" style="background: #1c1917; border: 1px solid #292524;"><img src="https://raw.githubusercontent.com/edurdias/capyctl/main/docs/brand/horizontal-dark.png" alt="CapyCTL, control what runs next"></div> |
| <h2 class="cy-lede">Run more models than your GPU can hold.</h2> |
| <p class="cy-sub">CapyCTL deploys, parks, wakes and switches models on vLLM, SGLang and TensorFold, on GPU machines you own, behind one OpenAI-compatible endpoint.</p> |
| <div class="cy-row"> |
| <a class="cy-btn cy-primary" href="https://edurdias.github.io/capyctl/docs/install/">Get started</a> |
| <a class="cy-btn border border-gray-200 dark:border-gray-700" href="https://edurdias.github.io/capyctl/docs/">Docs</a> |
| <a class="cy-btn border border-gray-200 dark:border-gray-700" href="https://github.com/edurdias/capyctl">GitHub</a> |
| <a class="cy-btn border border-gray-200 dark:border-gray-700" href="https://github.com/edurdias/capyctl-recipes">Recipes</a> |
| </div> |
| <p class="cy-meta text-gray-500 dark:text-gray-400">v0.1.2 · Apache-2.0 · one binary, written in Rust · Linux x86-64 and ARM64 · NVIDIA GPUs</p> |
| <div class="cy-split"> |
| <div class="cy-shot"> |
| <div class="cy-term"> |
| <div class="cy-term-bar"><span class="cy-dot" style="background: #ef4444;"></span><span class="cy-dot" style="background: #eab308;"></span><span class="cy-dot" style="background: #22c55e;"></span></div> |
| <pre><span class="cy-p">$</span> capyctl list deployments |
| <span class="cy-h">NAME STATE READY REVISION HOSTS</span> |
| coding-small <span class="cy-ready">ready</span> 1/1 3 gpu-box |
| chat-large <span class="cy-ready">ready</span> 1/1 1 workstation |
| vision-small <span class="cy-parked">parked</span> 0/1 2 gpu-box</pre> |
| </div> |
| <p class="cy-cap text-gray-500 dark:text-gray-400">Example output. A parked model releases its GPU memory and wakes on the next request for it.</p> |
| </div> |
| <ul class="cy-why"> |
| <li><b>More models than memory</b>Most GPU machines hold one or two models at a time. CapyCTL parks the idle ones and wakes the one a request asks for, making room by parking or stopping an idle model.</li> |
| <li><b>One endpoint</b>OpenAI-compatible chat completions with streaming and tool calls, behind an API key. Your apps keep one base URL and pick a model by name.</li> |
| <li><b>Your engines, your weights</b>Register the vLLM, SGLang or TensorFold installation you already have. Models come from a directory or from the Hub, pinned to a commit when you deploy.</li> |
| </ul> |
| </div> |
| </div> |
|
|
| ## Quick start |
|
|
| On Linux with an NVIDIA GPU and a vLLM, SGLang or TensorFold environment: |
|
|
| ```bash |
| curl -fsSL https://edurdias.github.io/capyctl/install.sh | sh |
| capyctl engine add ~/venvs/vllm # a vLLM, SGLang or TensorFold environment |
| capyctl start standalone |
| ``` |
|
|
| In another shell, deploy a model from the Hub. Three fields are enough: |
|
|
| ```yaml |
| # qwen3-4b.yaml |
| name: qwen3-4b |
| engine: vllm |
| model: {hf: Qwen/Qwen3-4B-Instruct-2507} |
| ``` |
|
|
| ```bash |
| capyctl deploy model --file qwen3-4b.yaml --activate --wait |
| ``` |
|
|
| Any OpenAI client can now send `qwen3-4b` as the model name ([Make a request](https://edurdias.github.io/capyctl/docs/requests/)). To free its GPU memory, run `capyctl park deployment qwen3-4b`; the next request wakes it. One server can also drive several GPU machines ([Run on several machines](https://edurdias.github.io/capyctl/docs/several-machines/)). |
|
|
| ## Measured recipes |
|
|
| Deployment files run through CapyCTL, each sized so the model fits in 32 GiB once ready. Measured on a GB10 on 2026-10-07. |
|
|
| | Model and checkpoint | Engine | Decode, one stream | More streams | |
| |---|---|---|---| |
| | Qwen3.6-35B-A3B, MLX 4-bit, MTP drafts | [TensorFold 0.6.5](https://github.com/edurdias/capyctl-recipes/tree/main/tensorfold/qwen3.6-35b-a3b-mlx-4bit-32gib-gb10) | 164 tokens/s | 368 tokens/s at 8 streams | |
| | Qwen3.6-35B-A3B, NVFP4 | [vLLM 0.30.0](https://github.com/edurdias/capyctl-recipes/tree/main/vllm/qwen3.6-35b-a3b-nvfp4-32gib-gb10) | 76.7 tokens/s | 180 tokens/s at 4 streams | |
| | Qwen3.6-35B-A3B, NVFP4 | [SGLang 0.5.21](https://github.com/edurdias/capyctl-recipes/tree/main/sglang/qwen3.6-35b-a3b-nvfp4-32gib-gb10) | 83.5 tokens/s | 186 tokens/s at 4 streams | |
| | Nemotron 3.5 Lightning 30B-A3B, MLX 4-bit, MTP drafts | [TensorFold 0.6.5](https://github.com/edurdias/capyctl-recipes/tree/main/tensorfold/nemotron-3.5-lightning-30b-a3b-mlx-4bit-32gib-gb10) | 137 tokens/s | one request at a time | |
| | Nemotron 3.5 Lightning 30B-A3B, NVFP4 | [vLLM 0.30.0](https://github.com/edurdias/capyctl-recipes/tree/main/vllm/nemotron-3.5-lightning-30b-a3b-nvfp4-32gib-gb10) | 69.9 tokens/s | 164 tokens/s at 4 streams | |
| | Nemotron 3.5 Lightning 30B-A3B, NVFP4 | [SGLang 0.5.21](https://github.com/edurdias/capyctl-recipes/tree/main/sglang/nemotron-3.5-lightning-30b-a3b-nvfp4-32gib-gb10) | 69.9 tokens/s | 163 tokens/s at 4 streams | |
|
|
| The TensorFold and NVFP4 rows serve different checkpoints, so they are not a one-to-one engine comparison. Each recipe lists the model revision, the commands, ready times, memory and a capyctl-bench report; [the recipes repository](https://github.com/edurdias/capyctl-recipes) has gpt-oss-20b, Gemma 4, Qwen3.8-27B and more. |
|
|
| ## In this organization |
|
|
| Checkpoints converted so a recipe can serve them. Each card says what changed from the original and how it was made. |
|
|
| - [FrogNano-4B-2609-MLX-4bit](https://huggingface.co/capyctl/FrogNano-4B-2609-MLX-4bit): MLX 4-bit, text only, for TensorFold on NVIDIA GPUs. |
| - [FrogNano-4B-2609-NVFP4](https://huggingface.co/capyctl/FrogNano-4B-2609-NVFP4): NVFP4 with FP8 attention, for vLLM, SGLang and TensorFold. |
|
|
| ## Status |
|
|
| 0.1.2 is early software: one GPU per model, CLI only, and breaking changes are possible between minor versions while the version starts with 0. Models that span several GPUs or machines are coming; they are not in a release yet. Questions and bug reports go to [GitHub](https://github.com/edurdias/capyctl/blob/main/SUPPORT.md). Licensed under [Apache-2.0](https://github.com/edurdias/capyctl/blob/main/LICENSE). |
|
|
