Unbabel
/

TowerInstruct-Mistral-7B-v0.2

text-generation

text-generation-inference

Model card Files Files and versions

The reasoning speed is particularly slow, how to speed up the reasoning speed

#5

by LiPengtao12138 - opened Sep 26, 2024

The reasoning speed is particularly slow, how to speed up the reasoning speed

I noticed the same thing - very slow inference

We recommend using vllm.

jmprcp changed discussion status to closed Dec 26, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment