Hermes Bot commited on
Commit
e382b5e
·
unverified ·
1 Parent(s): 6f6cdba

fix: use Qwen/Qwen2.5-7B-Instruct as default model

Browse files

Gemma 2 9B is not supported by the HF Inference router on this account.
Qwen 2.5 7B is supported, still well under 32B, and strong for storytelling.

Files changed (1) hide show
  1. shared/inference_client.py +1 -1
shared/inference_client.py CHANGED
@@ -31,7 +31,7 @@ log = logging.getLogger("inference")
31
  # The HF model id used for text generation (VibeThinker 1.5B, Gemma 4 12B, etc.)
32
  INFERENCE_MODEL = os.environ.get(
33
  "INFERENCE_MODEL",
34
- "google/gemma-2-9b-it", # 9B, newer, better storytelling
35
  )
36
 
37
  # Provider: "hf-inference" (free serverless), "together", "fal-ai", "replicate"
 
31
  # The HF model id used for text generation (VibeThinker 1.5B, Gemma 4 12B, etc.)
32
  INFERENCE_MODEL = os.environ.get(
33
  "INFERENCE_MODEL",
34
+ "Qwen/Qwen2.5-7B-Instruct", # 7B, strong storytelling, HF Inference compatible
35
  )
36
 
37
  # Provider: "hf-inference" (free serverless), "together", "fal-ai", "replicate"