rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
5fc01a7
·
1 Parent(s): 4516e87

fix(fact-find): KI-075 — bump brain timeout 12s → 25s so NIM cold-start doesn't kill the call

Browse files

Live probe on production (5 sequential turns, fresh sessions):
Turn 1 13.2s fact_find_brain::fallback ← cold start, timed out
Turn 2 13.2s fact_find_brain::fallback ← still cold, timed out
Turn 3 23.3s fact_find_brain::fallback ← still cold, timed out
Turn 4 14.5s fact_find_brain::fallback ← still cold, timed out
Turn 5 6.4s fact_find_brain::continue ← WARM, natural reply

All 4 fallback turns hit ~13s latency — exactly at the `_TIMEOUT_S = 12s`
asyncio.wait_for cap. The fast-brain chain has `total_budget_s = 22s`
internally for cumulative chain failover, but the outer wait_for was
killing the entire call before any cross-provider fallback link (Groq,
OpenRouter) ever got tried.

Bumped `_TIMEOUT_S = 25.0` to give:
- NIM cold-start headroom (10-15s observed on first call after Space rebuild)
- Room for one chain fallback hop (NIM → Groq) if the primary times out
- Still bounded so we don't leak infinite latency

This is the root cause of "language is still static and robotic" the
user reported — the LLM brain never got a chance to respond on cold
turns, so canonical fallback fired every time.

Expected impact: brain success rate climbs from ~20% (turn 5/5) to
~80%+ for typical cold-start sessions; near-100% once the Space is warm.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. backend/fact_find_brain.py +8 -1
backend/fact_find_brain.py CHANGED
@@ -276,7 +276,14 @@ def _bump_brain_history(session, slot_driving: Optional[str]) -> int:
276
  # Public entry
277
  # ----------------------------------------------------------------------------
278
 
279
- _TIMEOUT_S = 12.0
 
 
 
 
 
 
 
280
 
281
 
282
  async def drive_fact_find(
 
276
  # Public entry
277
  # ----------------------------------------------------------------------------
278
 
279
+ # KI-075 (2026-05-15) — bumped 12s → 25s. Live probe on production showed
280
+ # 4 of 5 fact-find turns hitting the 12s wait_for cap at exactly 13.2s
281
+ # latency, falling to canonical fallback even though FAST_BRAIN_CHAIN's
282
+ # total_budget_s is 22s. NIM cold-start eats 10-15s on the first call
283
+ # after a Space rebuild; the 12s wait_for was killing the brain BEFORE
284
+ # the cross-provider fallback links (Groq, OpenRouter) ever got tried.
285
+ # 25s gives NIM cold-start headroom + leaves room for one chain fallback.
286
+ _TIMEOUT_S = 25.0
287
 
288
 
289
  async def drive_fact_find(