ignatius commited on
Commit
de2ecdc
·
1 Parent(s): 153fd5b

Raise the default model output limit to 16000 tokens

Browse files

A full interview brief (RG-50.030.0002) needed about 6,000 output tokens,
so the old 6000 limit cut Claude's answer off and the run fell back to
the second model.

Claude-Session: https://claude.ai/code/session_01AteuhwysR4rHp4fqUteLAE

Files changed (2) hide show
  1. curatorbot/config.py +1 -1
  2. docs/model-providers.md +1 -1
curatorbot/config.py CHANGED
@@ -32,7 +32,7 @@ class Settings:
32
  # For the 'compatible' provider: an institution's own gateway or a local vLLM/Ollama server.
33
  llm_base_url: str = field(default_factory=lambda: os.getenv("CURATORBOT_LLM_BASE_URL", ""))
34
  llm_json_mode: bool = field(default_factory=lambda: _flag("CURATORBOT_LLM_JSON_MODE", True))
35
- llm_max_tokens: int = field(default_factory=lambda: int(os.getenv("CURATORBOT_LLM_MAX_TOKENS", "6000")))
36
  ner_model: str = field(default_factory=lambda: os.getenv("CURATORBOT_NER_MODEL", "en_core_web_sm"))
37
  data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_DATA_DIR", str(Path(__file__).resolve().parent.parent / "data" / "interviews"))))
38
  # Private Hugging Face dataset holding real briefs (e.g. ignatius/curatorbot-data).
 
32
  # For the 'compatible' provider: an institution's own gateway or a local vLLM/Ollama server.
33
  llm_base_url: str = field(default_factory=lambda: os.getenv("CURATORBOT_LLM_BASE_URL", ""))
34
  llm_json_mode: bool = field(default_factory=lambda: _flag("CURATORBOT_LLM_JSON_MODE", True))
35
+ llm_max_tokens: int = field(default_factory=lambda: int(os.getenv("CURATORBOT_LLM_MAX_TOKENS", "16000")))
36
  ner_model: str = field(default_factory=lambda: os.getenv("CURATORBOT_NER_MODEL", "en_core_web_sm"))
37
  data_dir: Path = field(default_factory=lambda: Path(os.getenv("CURATORBOT_DATA_DIR", str(Path(__file__).resolve().parent.parent / "data" / "interviews"))))
38
  # Private Hugging Face dataset holding real briefs (e.g. ignatius/curatorbot-data).
docs/model-providers.md CHANGED
@@ -17,7 +17,7 @@ Other settings:
17
  - `CURATORBOT_LLM_FALLBACK_MODEL`: tried once if the main model fails for a reason another model could fix (an outage, a rate limit, an unreadable answer). It is not tried for account problems such as a wrong key or no credit. Set it to an empty value to switch the fallback off. Briefs record which model actually answered.
18
  - `CURATORBOT_LLM_BASE_URL`: the address of a `compatible` endpoint, for example `http://localhost:11434/v1` for Ollama.
19
  - `CURATORBOT_LLM_JSON_MODE=0`: for local servers that reject the JSON-mode option.
20
- - `CURATORBOT_LLM_MAX_TOKENS`: raise it if long interviews return incomplete answers.
21
 
22
  `/api/health` shows the active provider, model and fallback.
23
 
 
17
  - `CURATORBOT_LLM_FALLBACK_MODEL`: tried once if the main model fails for a reason another model could fix (an outage, a rate limit, an unreadable answer). It is not tried for account problems such as a wrong key or no credit. Set it to an empty value to switch the fallback off. Briefs record which model actually answered.
18
  - `CURATORBOT_LLM_BASE_URL`: the address of a `compatible` endpoint, for example `http://localhost:11434/v1` for Ollama.
19
  - `CURATORBOT_LLM_JSON_MODE=0`: for local servers that reject the JSON-mode option.
20
+ - `CURATORBOT_LLM_MAX_TOKENS` (default 16000): the longest answer the model may write. A full interview's brief needs about 6,000; if an answer is cut off, the run falls back to the second model, which the brief records.
21
 
22
  `/api/health` shows the active provider, model and fallback.
23