Spaces:
Running on Zero
Running on Zero
jayaspjacob Claude Opus 4.8 (1M context) commited on
Commit ·
ac82455
1
Parent(s): 979b98f
Switch script LLM to an available provider model
Browse filesHF Inference Providers no longer serve Qwen/Qwen2.5-14B-Instruct for this
account ("not supported by any provider you have enabled"), causing a 400
on every script generation. Default to Qwen/Qwen2.5-7B-Instruct — same
family/prompt behavior, still served, under the 32B cap. Override with
LLM_MODEL (e.g. google/gemma-3-27b-it) for higher quality.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- research/llm.py +6 -1
research/llm.py
CHANGED
|
@@ -13,7 +13,12 @@ from typing import List, Dict
|
|
| 13 |
from huggingface_hub import InferenceClient
|
| 14 |
|
| 15 |
# Default kept under 32B params (project constraint). Override via the LLM_MODEL env var.
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
|
| 19 |
@lru_cache(maxsize=1)
|
|
|
|
| 13 |
from huggingface_hub import InferenceClient
|
| 14 |
|
| 15 |
# Default kept under 32B params (project constraint). Override via the LLM_MODEL env var.
|
| 16 |
+
# Qwen2.5-14B-Instruct was dropped by HF Inference Providers ("not supported by any
|
| 17 |
+
# provider you have enabled"), so we use the 7B sibling — same family/prompt behavior,
|
| 18 |
+
# still served. For higher-quality scripts set LLM_MODEL=google/gemma-3-27b-it (also
|
| 19 |
+
# <32B and currently available). Avoid Qwen3 "thinking" variants: they emit <think>
|
| 20 |
+
# traces that break the strict `Speaker: text` script format.
|
| 21 |
+
DEFAULT_MODEL = os.environ.get("LLM_MODEL", "Qwen/Qwen2.5-7B-Instruct")
|
| 22 |
|
| 23 |
|
| 24 |
@lru_cache(maxsize=1)
|