Error Loading using vLLM

by suleimanelkhoury - opened 8 days ago

Hi, running the model using vLLM returns the following error:
AttributeError: CachedMistralCommonBackend has no attribute is_fast

vLLM is also detecting the architecture falsely: Resolved architecture: TransformersMultiModalForCausalLM

as if vLLM is programmed to only detect the source mistralai/Voxtral-Mini-4B-Realtime-2602 repository. Do you encounter the same issue?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment