rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
e8ccfa0
·
1 Parent(s): 835f745

fix(upload-extract): drop responseMimeType=application/json — was producing unparseable payloads

Browse files

Gemini's native JSON mode (responseMimeType=application/json) was
silently emitting JSON that json_from_llm_text / HealthPolicy schema
validation rejected — surfacing as "LLM returned no valid HealthPolicy"
on every fresh upload. Catalogued 148 extraction worked WITHOUT JSON
mode (rag/extract.py never sets it) because the EXTRACT_SYSTEM prompt
already mandates JSON-only output ("Single object. No whitespace
beyond what's needed.") AND json_from_llm_text already strips code
fences + <think> blocks + finds the first balanced { ... } block.

Removed the response_format kwarg from the Gemini chat call. Same
path as the catalogued 148 now — prompt-driven JSON, not transport-
driven JSON mode.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. backend/uploaded_docs.py +9 -8
backend/uploaded_docs.py CHANGED
@@ -876,14 +876,15 @@ async def extract_one_for_upload(
876
  "temperature": 0.0,
877
  "max_tokens": 8192,
878
  }
879
- # Native JSON mode on the Gemini path — forces a single
880
- # JSON object response that json_from_llm_text can parse
881
- # without prose-stripping. Absorbed by **kwargs on the
882
- # NIM provider (it doesn't honor response_format the same
883
- # way; rely on the EXTRACT_SYSTEM prompt's JSON-only
884
- # instruction there).
885
- if label.startswith("gemini"):
886
- chat_kwargs["response_format"] = {"type": "json_object"}
 
887
  res = await asyncio.wait_for(
888
  llm.chat(**chat_kwargs),
889
  timeout=attempt_timeout,
 
876
  "temperature": 0.0,
877
  "max_tokens": 8192,
878
  }
879
+ # NOTE we DON'T set response_format here. The
880
+ # EXTRACT_SYSTEM prompt already mandates JSON-only output
881
+ # ("Single object. No whitespace beyond what's needed.").
882
+ # Setting Gemini's responseMimeType=application/json was
883
+ # silently producing payloads that json_from_llm_text /
884
+ # HealthPolicy schema validation rejected. Letting the
885
+ # model emit JSON natively (and using json_from_llm_text's
886
+ # tolerant front-strip of fences + <think> blocks) is the
887
+ # path the catalogued 148 used successfully.
888
  res = await asyncio.wait_for(
889
  llm.chat(**chat_kwargs),
890
  timeout=attempt_timeout,