Hermes Bot commited on
fix: use Qwen/Qwen2.5-7B-Instruct as default model
Browse filesGemma 2 9B is not supported by the HF Inference router on this account.
Qwen 2.5 7B is supported, still well under 32B, and strong for storytelling.
shared/inference_client.py
CHANGED
|
@@ -31,7 +31,7 @@ log = logging.getLogger("inference")
|
|
| 31 |
# The HF model id used for text generation (VibeThinker 1.5B, Gemma 4 12B, etc.)
|
| 32 |
INFERENCE_MODEL = os.environ.get(
|
| 33 |
"INFERENCE_MODEL",
|
| 34 |
-
"
|
| 35 |
)
|
| 36 |
|
| 37 |
# Provider: "hf-inference" (free serverless), "together", "fal-ai", "replicate"
|
|
|
|
| 31 |
# The HF model id used for text generation (VibeThinker 1.5B, Gemma 4 12B, etc.)
|
| 32 |
INFERENCE_MODEL = os.environ.get(
|
| 33 |
"INFERENCE_MODEL",
|
| 34 |
+
"Qwen/Qwen2.5-7B-Instruct", # 7B, strong storytelling, HF Inference compatible
|
| 35 |
)
|
| 36 |
|
| 37 |
# Provider: "hf-inference" (free serverless), "together", "fal-ai", "replicate"
|