I researched the practical GPU specs for Kimi-K3 — a few findings that surprised me more...
Well, I spent some time going through the Kimi K3 release, vLLM recipes, checkpoint sizing, speculative decoding setup, and the GPU configurations that are actually realistic for self-hosting it.
Kimi K3 is 2.8T parameters total with ~104B active per token. It has 896 routed experts, 16 active per token, native vision, and a 1M-token context window.
The interesting part is that it doesn't ship as a giant BF16 checkpoint. The released weights use a mixed MXFP4/BF16 representation and are only about 1.56 TB.
A few useful things I found:
8× B300 / GB300 is basically the minimum NVIDIA setup. With 288 GB per GPU, you get ~2.3 TB aggregate VRAM, leaving enough headroom beyond the ~1.56 TB checkpoint.
AMD is viable too: vLLM has a documented 8× MI355X TP8 path.
B200/GB200 needs more GPUs. vLLM says K3 requires at least 16× B200/GB200, since an 8-GPU B200 box doesn't have enough memory.
K3's 1M context doesn't mean you should always allocate 1M tokens. Long context still eats serving memory and concurrency, so --max-model-len should match the workload you're actually running.
The other big one is DSpark speculative decoding. vLLM's GB300 testing went from 118 tok/s/user → 370 tok/s/user, about a 3.14× improvement, using the open Kimi-K3 DSpark draft model.
---
Hardware picture:
Native MXFP4 / mixed checkpoint
~1.56 TB weights
8× B300 / GB300
8× MI355X
16× B200 / GB200
For production-scale traffic, the story quickly becomes multi-node with expert/data parallelism, fast NVLink/RDMA, prefix caching, and potentially prefill/decode disaggregation.
I collected the verified GPU layouts, vLLM commands, quants, DSpark config, and deployment notes here:
Full breakdown:
https://blog.gpus.market/deploying-kimi-k3-with-vllm-verified-gpu-pods-quants-and-serving-recipes