This model is fantastic, I got it running on my 3090 card by adding some other vllm arguments --max-num-seqs 2 --max-model-len 32067
--max-num-seqs 2 --max-model-len 32067
· Sign up or log in to comment
