Spaces:
Running
Running
Raise the default model output limit to 16000 tokens
Browse filesA full interview brief (RG-50.030.0002) needed about 6,000 output tokens,
so the old 6000 limit cut Claude's answer off and the run fell back to
the second model.
Claude-Session: https://claude.ai/code/session_01AteuhwysR4rHp4fqUteLAE
- curatorbot/config.py +1 -1
- docs/model-providers.md +1 -1
curatorbot/config.py
CHANGED
|
@@ -32,7 +32,7 @@ class Settings:
|
|
| 32 |
# For the 'compatible' provider: an institution's own gateway or a local vLLM/Ollama server.
|
| 33 |
llm_base_url: str = field(default_factory=lambda: os.getenv("CURATORBOT_LLM_BASE_URL", ""))
|
| 34 |
llm_json_mode: bool = field(default_factory=lambda: _flag("CURATORBOT_LLM_JSON_MODE", True))
|
| 35 |
-
llm_max_tokens: int = field(default_factory=lambda: int(os.getenv("CURATORBOT_LLM_MAX_TOKENS", "
|
| 36 |
ner_model: str = field(default_factory=lambda: os.getenv("CURATORBOT_NER_MODEL", "en_core_web_sm"))
|
| 37 |
data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_DATA_DIR", str(Path(__file__).resolve().parent.parent / "data" / "interviews"))))
|
| 38 |
# Private Hugging Face dataset holding real briefs (e.g. ignatius/curatorbot-data).
|
|
|
|
| 32 |
# For the 'compatible' provider: an institution's own gateway or a local vLLM/Ollama server.
|
| 33 |
llm_base_url: str = field(default_factory=lambda: os.getenv("CURATORBOT_LLM_BASE_URL", ""))
|
| 34 |
llm_json_mode: bool = field(default_factory=lambda: _flag("CURATORBOT_LLM_JSON_MODE", True))
|
| 35 |
+
llm_max_tokens: int = field(default_factory=lambda: int(os.getenv("CURATORBOT_LLM_MAX_TOKENS", "16000")))
|
| 36 |
ner_model: str = field(default_factory=lambda: os.getenv("CURATORBOT_NER_MODEL", "en_core_web_sm"))
|
| 37 |
data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_DATA_DIR", str(Path(__file__).resolve().parent.parent / "data" / "interviews"))))
|
| 38 |
# Private Hugging Face dataset holding real briefs (e.g. ignatius/curatorbot-data).
|
docs/model-providers.md
CHANGED
|
@@ -17,7 +17,7 @@ Other settings:
|
|
| 17 |
- `CURATORBOT_LLM_FALLBACK_MODEL`: tried once if the main model fails for a reason another model could fix (an outage, a rate limit, an unreadable answer). It is not tried for account problems such as a wrong key or no credit. Set it to an empty value to switch the fallback off. Briefs record which model actually answered.
|
| 18 |
- `CURATORBOT_LLM_BASE_URL`: the address of a `compatible` endpoint, for example `http://localhost:11434/v1` for Ollama.
|
| 19 |
- `CURATORBOT_LLM_JSON_MODE=0`: for local servers that reject the JSON-mode option.
|
| 20 |
-
- `CURATORBOT_LLM_MAX_TOKENS`:
|
| 21 |
|
| 22 |
`/api/health` shows the active provider, model and fallback.
|
| 23 |
|
|
|
|
| 17 |
- `CURATORBOT_LLM_FALLBACK_MODEL`: tried once if the main model fails for a reason another model could fix (an outage, a rate limit, an unreadable answer). It is not tried for account problems such as a wrong key or no credit. Set it to an empty value to switch the fallback off. Briefs record which model actually answered.
|
| 18 |
- `CURATORBOT_LLM_BASE_URL`: the address of a `compatible` endpoint, for example `http://localhost:11434/v1` for Ollama.
|
| 19 |
- `CURATORBOT_LLM_JSON_MODE=0`: for local servers that reject the JSON-mode option.
|
| 20 |
+
- `CURATORBOT_LLM_MAX_TOKENS` (default 16000): the longest answer the model may write. A full interview's brief needs about 6,000; if an answer is cut off, the run falls back to the second model, which the brief records.
|
| 21 |
|
| 22 |
`/api/health` shows the active provider, model and fallback.
|
| 23 |
|