jayaspjacob Claude Opus 4.8 (1M context) commited on
Commit
ac82455
·
1 Parent(s): 979b98f

Switch script LLM to an available provider model

Browse files

HF Inference Providers no longer serve Qwen/Qwen2.5-14B-Instruct for this
account ("not supported by any provider you have enabled"), causing a 400
on every script generation. Default to Qwen/Qwen2.5-7B-Instruct — same
family/prompt behavior, still served, under the 32B cap. Override with
LLM_MODEL (e.g. google/gemma-3-27b-it) for higher quality.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. research/llm.py +6 -1
research/llm.py CHANGED
@@ -13,7 +13,12 @@ from typing import List, Dict
13
  from huggingface_hub import InferenceClient
14
 
15
  # Default kept under 32B params (project constraint). Override via the LLM_MODEL env var.
16
- DEFAULT_MODEL = os.environ.get("LLM_MODEL", "Qwen/Qwen2.5-14B-Instruct")
 
 
 
 
 
17
 
18
 
19
  @lru_cache(maxsize=1)
 
13
  from huggingface_hub import InferenceClient
14
 
15
  # Default kept under 32B params (project constraint). Override via the LLM_MODEL env var.
16
+ # Qwen2.5-14B-Instruct was dropped by HF Inference Providers ("not supported by any
17
+ # provider you have enabled"), so we use the 7B sibling — same family/prompt behavior,
18
+ # still served. For higher-quality scripts set LLM_MODEL=google/gemma-3-27b-it (also
19
+ # <32B and currently available). Avoid Qwen3 "thinking" variants: they emit <think>
20
+ # traces that break the strict `Speaker: text` script format.
21
+ DEFAULT_MODEL = os.environ.get("LLM_MODEL", "Qwen/Qwen2.5-7B-Instruct")
22
 
23
 
24
  @lru_cache(maxsize=1)