coreprinciple Claude Opus 4.8 commited on
Commit
5ef2ba5
·
1 Parent(s): 128e312

Call-1 reasoning budget 1024 + no-think A/B flag; narration name lockdown

Browse files

Live trace findings drove these:
- Call-1 reasoning ran past the 512-token budget without emitting JSON
(empty -> fallback). Bump think budget to 1024; no-think stays 256.
- Add DISCOVERROUTE_VIBE_THINKING A/B flag (config.VIBE_THINKING, default
on) and record the chosen mode on every vibe_extraction trace row so
think vs no-think can be compared head-to-head.
- Narration kept name-dropping real off-route landmarks (Pont des Arts,
Mona Lisa, Sorbonne, Sacre-Coeur) -> gate rejected -> template. Tighten
the prompt to a hard allowlist (stops + endpoints + gazetteer only;
scene-set without proper names) and fix the literal-"H3" artifact.
- Scale narration token budget with route length (600..880) so long
routes stop truncating mid-name.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

src/discoverroute/config.py CHANGED
@@ -149,6 +149,16 @@ EMBED_QUERY_INSTRUCTION = "Represent this sentence for searching relevant passag
149
  # LlamaForCausalLM architecture — no custom kernels.
150
  LLM_MODEL = "openbmb/MiniCPM5-1B"
151
 
 
 
 
 
 
 
 
 
 
 
152
  # --- Trace logging (Open Trace) ----------------------------------------------
153
  # Every inference call logs a row locally to logs/traces.jsonl; when a write
154
  # token is present, rows are ALSO pushed (async, non-blocking) to TRACE_REPO.
 
149
  # LlamaForCausalLM architecture — no custom kernels.
150
  LLM_MODEL = "openbmb/MiniCPM5-1B"
151
 
152
+ # A/B toggle for Call 1 (vibe→weights): run the model's REASONING pass
153
+ # (enable_thinking, MiniCPM5-1B is hybrid-reasoning) or the fast no-think path.
154
+ # Thinking may score a fuzzy vibe better but needs a far bigger token budget and
155
+ # more of the ZeroGPU slice. Flip the Space variable DISCOVERROUTE_VIBE_THINKING
156
+ # (1=think [default], 0=no-think); the chosen mode is recorded on every trace row
157
+ # so the two can be compared head-to-head. (Either way, degenerate/empty output
158
+ # falls through to the bge-small embed tier — the route is never left tasteless.)
159
+ VIBE_THINKING = os.environ.get(
160
+ "DISCOVERROUTE_VIBE_THINKING", "1").lower() in ("1", "true", "on")
161
+
162
  # --- Trace logging (Open Trace) ----------------------------------------------
163
  # Every inference call logs a row locally to logs/traces.jsonl; when a write
164
  # token is present, rows are ALSO pushed (async, non-blocking) to TRACE_REPO.
src/discoverroute/interpret/llm_vibe.py CHANGED
@@ -121,19 +121,22 @@ def extract(vibe: str) -> dict | None:
121
  {"role": "system", "content": SYSTEM_PROMPT},
122
  {"role": "user", "content": f"Vibe: {vibe}\nReturn the JSON weights now."},
123
  ]
 
 
 
124
  t0 = time.time()
125
  try:
126
  from discoverroute.narrate.llm import run_inference
127
- # Reasoning pass: MiniCPM5-1B is hybrid-reasoning, and scoring a fuzzy vibe
128
- # across 14 types is exactly the kind of short deliberation a 1B does better
129
- # with than off-the-cuff. enable_thinking=True lets it reason, then emit the
130
- # JSON; run_inference strips the <think> block and returns only the JSON. The
131
- # budget covers the reasoning + the ~120-token object (truncated reasoning
132
- # empty answer clean fallback).
133
- raw_text = run_inference(messages, max_new_tokens=512, enable_thinking=True)
134
  except Exception as exc: # noqa: BLE001 - never break interpretation
135
  latency = int((time.time() - t0) * 1000)
136
- trace.log_trace("vibe_extraction", {"vibe": vibe},
137
  {"error": f"{type(exc).__name__}: {exc}"},
138
  latency, used_fallback=True)
139
  return None
@@ -143,13 +146,13 @@ def extract(vibe: str) -> dict | None:
143
  # Reject unparseable OR degenerate (all-zero / all-equal) output: both leave the
144
  # router with no taste signal, so fall through to the embedding tier instead.
145
  if parsed is None or _is_degenerate(parsed):
146
- trace.log_trace("vibe_extraction", {"vibe": vibe},
147
  {"raw": raw_text, "degenerate": parsed is not None},
148
  latency, used_fallback=True)
149
  return None
150
 
151
  affinity = mapping.brief_scores_to_affinity(parsed)
152
  budget_mult = max(0.5, min(2.0, parsed["detour_budget_multiplier"]))
153
- trace.log_trace("vibe_extraction", {"vibe": vibe}, parsed,
154
  latency, used_fallback=False)
155
  return {"affinity": affinity, "budget_multiplier": budget_mult, "raw": parsed}
 
121
  {"role": "system", "content": SYSTEM_PROMPT},
122
  {"role": "user", "content": f"Vibe: {vibe}\nReturn the JSON weights now."},
123
  ]
124
+ from discoverroute import config
125
+ think = config.VIBE_THINKING
126
+ meta_in = {"vibe": vibe, "thinking": think} # mode recorded for the A/B
127
  t0 = time.time()
128
  try:
129
  from discoverroute.narrate.llm import run_inference
130
+ # A/B (config.VIBE_THINKING): with thinking ON, MiniCPM5-1B reasons before
131
+ # emitting JSON but the <think> block + the ~120-token object need real
132
+ # room, so give it 1024 (512 ran past the budget and returned empty in the
133
+ # first live test). No-think is lean and fast. run_inference strips the
134
+ # <think> block; a truncated/unclosed reasoning empty answer fallback.
135
+ budget = 1024 if think else 256
136
+ raw_text = run_inference(messages, max_new_tokens=budget, enable_thinking=think)
137
  except Exception as exc: # noqa: BLE001 - never break interpretation
138
  latency = int((time.time() - t0) * 1000)
139
+ trace.log_trace("vibe_extraction", meta_in,
140
  {"error": f"{type(exc).__name__}: {exc}"},
141
  latency, used_fallback=True)
142
  return None
 
146
  # Reject unparseable OR degenerate (all-zero / all-equal) output: both leave the
147
  # router with no taste signal, so fall through to the embedding tier instead.
148
  if parsed is None or _is_degenerate(parsed):
149
+ trace.log_trace("vibe_extraction", meta_in,
150
  {"raw": raw_text, "degenerate": parsed is not None},
151
  latency, used_fallback=True)
152
  return None
153
 
154
  affinity = mapping.brief_scores_to_affinity(parsed)
155
  budget_mult = max(0.5, min(2.0, parsed["detour_budget_multiplier"]))
156
+ trace.log_trace("vibe_extraction", meta_in, parsed,
157
  latency, used_fallback=False)
158
  return {"affinity": affinity, "budget_multiplier": budget_mult, "raw": parsed}
src/discoverroute/narrate/narrate.py CHANGED
@@ -254,24 +254,27 @@ def _llm_narration(plain, discovery, pois, vibe, mode, start_label, end_label,
254
  system = (
255
  f"You are a {guide}local — a sharp, warm city guide who actually walks these "
256
  "streets. Write a vivid, first-person walk for this route.\n"
257
- "FORMAT — this matters: the first line is EXACTLY ONE markdown H3 heading a "
258
- "short, evocative title (3–6 words) you invent for THIS specific walk, hinting "
259
- "at its mood and place. After that, write 2–4 flowing paragraphs "
260
- "of continuous prose. Do NOT number the stops. Do NOT give any stop its own "
261
- "heading or bullet. If there are many stops, weave several into each paragraph "
262
- "rather than a sentence each — keep it moving and finish before you run long. "
263
- "Carry the reader start→finish with sensory detail and natural transitions "
264
- "('a block on', 'just around the corner', 'as the street opens up'). Bold each "
265
- "real stop's name the first time it appears, and reference the user's vibe in "
266
- "your own words.\n"
267
  "Describe each place in your OWN words — say what it is and why it's worth it. "
268
  "Never write bare type labels like 'park garden', 'place of worship' or "
269
- "'monument historic'. Set the scene freely with the districts, river and "
270
- "landmarks under 'You may reference', and mention the time of day or the light.\n"
271
- "ONE hard rule: do not invent a *named venue to visit* every place you name "
272
- "as a stop must be one of the 'Ordered stops' (spelled close to the list) or "
273
- "the start/destination. You don't need a name in every sentence; describe "
274
- "freely, just never fabricate a café/shop/museum name that isn't on the list."
 
 
 
275
  )
276
  user = (
277
  f"Vibe: {vibe or 'open to anything'}\n"
@@ -285,10 +288,13 @@ def _llm_narration(plain, discovery, pois, vibe, mode, start_label, end_label,
285
  )
286
  messages = [{"role": "system", "content": system},
287
  {"role": "user", "content": user}]
288
- # 600 tokens lets flowing prose cover ~10-12 stops without cutting off mid-walk
289
- # ("### 9: End your trip" in the traces); a 1B model on A10G still finishes inside
290
- # the 45s ZeroGPU slice (see llm.GPU_DURATION_S).
291
- text = run_inference(messages, max_new_tokens=600)
 
 
 
292
  # Title guard: a 1B model sometimes parrots a stock example title or omits the
293
  # heading. Ensure the walk opens with a sensible H3 — swap in our own computed
294
  # route-title if the model echoed a known example or wrote no heading at all.
 
254
  system = (
255
  f"You are a {guide}local — a sharp, warm city guide who actually walks these "
256
  "streets. Write a vivid, first-person walk for this route.\n"
257
+ "FORMAT — this matters: the FIRST line is a single short title (3–6 words) you "
258
+ "invent for THIS walk, written as a markdown H3 it MUST begin with '### ' "
259
+ "(three hashes then a space). Never write the literal letters 'H3'. After the "
260
+ "title, write 2–4 flowing paragraphs of continuous prose. Do NOT number the "
261
+ "stops. Do NOT give any stop its own heading or bullet. If there are many "
262
+ "stops, weave several into each paragraph rather than a sentence each — keep it "
263
+ "moving and finish before you run long. Carry the reader start→finish with "
264
+ "sensory detail and natural transitions ('a block on', 'just around the "
265
+ "corner', 'as the street opens up'). Bold each real stop's name the first time "
266
+ "it appears, and reference the user's vibe in your own words.\n"
267
  "Describe each place in your OWN words — say what it is and why it's worth it. "
268
  "Never write bare type labels like 'park garden', 'place of worship' or "
269
+ "'monument historic'. Mention the time of day or the light.\n"
270
+ "ONE HARD RULE NAMES: the ONLY proper names you may write are the 'Ordered "
271
+ "stops', the start/destination, and the items under 'You may reference'. Do NOT "
272
+ "name ANY other place no landmark, museum, monument, bridge, street, square, "
273
+ "university, church or artwork, even a real famous one (for example do not name "
274
+ "the Mona Lisa, Sacré-Cœur, Pont des Arts, the Sorbonne, the Louvre) unless it "
275
+ "is in those lists. To set the scene, describe WITHOUT proper names "
276
+ "('a stone bridge', 'a grand old church'). A single name outside the lists "
277
+ "makes the entire itinerary get thrown away."
278
  )
279
  user = (
280
  f"Vibe: {vibe or 'open to anything'}\n"
 
288
  )
289
  messages = [{"role": "system", "content": system},
290
  {"role": "user", "content": user}]
291
+ # Scale the budget with route length so long walks finish instead of truncating
292
+ # mid-name (a live trace cut "Sacré-C[œur]" off at 600 the gate then rejected
293
+ # the fragment). Short routes stay lean; long ones get up to 880, still inside the
294
+ # 45s ZeroGPU slice (see llm.GPU_DURATION_S). No-think path (fast, direct prose).
295
+ n = len(pois)
296
+ max_tok = 600 if n <= 6 else min(880, 600 + 40 * (n - 6))
297
+ text = run_inference(messages, max_new_tokens=max_tok)
298
  # Title guard: a 1B model sometimes parrots a stock example title or omits the
299
  # heading. Ensure the walk opens with a sensible H3 — swap in our own computed
300
  # route-title if the model echoed a known example or wrote no heading at all.