vLLM Support
#1
by benchislett - opened
I am reviewing vLLM support for EAGLE3 for Kimi K2, and I was not able to get this running with Kimi K2.5 + https://github.com/vllm-project/vllm/pull/36361. I got very low (< 1.5 AL) acceptance rates.
https://github.com/torchspec-project/TorchSpec/issues/38
That may be the reason, but I am still in the process of deploying the vLLM environment. Waiting for confirmation.
Thank you for finding this bug.
I've uploaded a fixed version. You can give it a try.
lightseek changed discussion status to closed