rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
fd30ceb
·
1 Parent(s): 3dcbe9b

revert(upload-extract): roll back Gemini-primary switch — surfaced as live failures

Browse files

7ef3ca3 swapped the extraction LLM from NIM (get_brain_llm) to
Gemini 2.5-flash with native JSON-mode. Live probe just now showed
BOTH attempts (Gemini primary + NIM fallback) raising on the EXTRACT
prompt — error surfaces as "LLM returned no valid HealthPolicy
after primary + fallback retries". The Sarvah Param test that
worked at 9ec9eae (grade C, score 65) regressed to status=failed
on 3dcbe9b.

Reverted the primary back to NIM. Both attempts are NIM now, same
as the proven path. Gemini will be re-added in a follow-up after:
- tracing why response_format={"type":"json_object"} returned
something json_from_llm_text() couldn't parse against the
HealthPolicy schema
- confirming Gemini 2.5-flash's JSON-mode honors the EXTRACT_SYSTEM
schema_excerpt() instructions

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. backend/uploaded_docs.py +15 -24
backend/uploaded_docs.py CHANGED
@@ -853,39 +853,30 @@ async def extract_one_for_upload(
853
  ChatMessage(role="user", content=prompt),
854
  ]
855
 
856
- # Two-tier attempt: Gemini primary (native JSON mode, steady-state
857
- # brain) NIM chain fallback (older path, multi-candidate failover).
858
- try:
859
- llm_primary = GoogleGeminiLLM(timeout=180.0)
860
- primary_label = "gemini-2.5-flash"
861
- except Exception as e: # noqa: BLE001 should never happen at __init__
862
- _log.warning(
863
- "[upload-extract] Gemini init failed (%s) falling back to NIM",
864
- type(e).__name__,
865
- )
866
- llm_primary = get_brain_llm()
867
- primary_label = "nim-chain"
868
  llm_fallback = get_brain_llm()
869
 
870
  raw = ""
871
  policy: Optional[HealthPolicy] = None
872
  for attempt, (llm, label) in enumerate(
873
- [(llm_primary, primary_label), (llm_fallback, "nim-chain")]
874
  ):
875
  try:
876
  attempt_timeout = 180 if attempt == 0 else 120
877
- chat_kwargs = {
878
- "messages": messages,
879
- "temperature": 0.0,
880
- "max_tokens": 2048,
881
- }
882
- # Gemini supports native JSON mode via response_format —
883
- # forces the model to emit a JSON object the schema parser
884
- # can validate without prose-cleanup. Harmless on the NIM
885
- # path (the kwarg is absorbed by **kwargs).
886
- chat_kwargs["response_format"] = {"type": "json_object"}
887
  res = await asyncio.wait_for(
888
- llm.chat(**chat_kwargs),
 
 
 
 
889
  timeout=attempt_timeout,
890
  )
891
  raw = res.text
 
853
  ChatMessage(role="user", content=prompt),
854
  ]
855
 
856
+ # ROLLBACK (2026-05-27 late session) the Gemini-primary swap
857
+ # in commit 7ef3ca3 broke live extraction (both Gemini and the
858
+ # NIM-on-fallback raised, surfacing as "LLM returned no valid
859
+ # HealthPolicy after primary + fallback retries"). Reverting to
860
+ # NIM as primary (the proven-working path that yielded grade=C
861
+ # score=65 on Sarvah Param earlier today). Gemini will be
862
+ # re-attempted as a follow-up after proper debugging of its
863
+ # JSON-mode response shape against the EXTRACT prompt.
864
+ llm_primary = get_brain_llm()
 
 
 
865
  llm_fallback = get_brain_llm()
866
 
867
  raw = ""
868
  policy: Optional[HealthPolicy] = None
869
  for attempt, (llm, label) in enumerate(
870
+ [(llm_primary, "nim-primary"), (llm_fallback, "nim-fallback")]
871
  ):
872
  try:
873
  attempt_timeout = 180 if attempt == 0 else 120
 
 
 
 
 
 
 
 
 
 
874
  res = await asyncio.wait_for(
875
+ llm.chat(
876
+ messages=messages,
877
+ temperature=0.0,
878
+ max_tokens=2048,
879
+ ),
880
  timeout=attempt_timeout,
881
  )
882
  raw = res.text