@shanjiaz can you explain all these vLLM flags/envs ? They are different than the official vLLM recipe
https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro?hardware=b300&strategy=single_node_tep&features=tool_calling%2Creasoning%2Cspec_decoding%2Cnvfp4_kv_cache&dcp=8
@g-a-b-y Please refer to the official vLLM recipe for the best performance! This is provided to reproduce the benchmarking 🫡
Thanks!
· Sign up or log in to comment