Spaces:
Running on Zero
Call-1 weights: drop the parroted example, default to no-think (A/B verdict)
Browse filesLive A/B on "quiet green wander" was decisive:
- No-think: 9.6s, clean JSON — but byte-identical to the worked example
in the prompt (the 1B copied the example's numbers verbatim).
- Think: 26s (3x slower/quota), and corrupted the JSON (zeroed the
matching categories). Reasoning did not help this scoring task.
Two parrot traps now removed: the old `0.0-1.0` schema AND the worked
JSON example. New prompt gives NO copyable number-set — it states 0..1 in
words and teaches the mapping with short word->type cues, so the model
scores the actual vibe. Wrong/empty output still falls through to the
embed tier (_is_degenerate). Default flag flipped to no-think; thinking
stays toggleable via DISCOVERROUTE_VIBE_THINKING=1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@@ -151,13 +151,13 @@ LLM_MODEL = "openbmb/MiniCPM5-1B"
|
|
| 151 |
|
| 152 |
# A/B toggle for Call 1 (vibe→weights): run the model's REASONING pass
|
| 153 |
# (enable_thinking, MiniCPM5-1B is hybrid-reasoning) or the fast no-think path.
|
| 154 |
-
#
|
| 155 |
-
#
|
| 156 |
-
#
|
| 157 |
-
#
|
| 158 |
-
#
|
| 159 |
VIBE_THINKING = os.environ.get(
|
| 160 |
-
"DISCOVERROUTE_VIBE_THINKING", "
|
| 161 |
|
| 162 |
# --- Trace logging (Open Trace) ----------------------------------------------
|
| 163 |
# Every inference call logs a row locally to logs/traces.jsonl; when a write
|
|
|
|
| 151 |
|
| 152 |
# A/B toggle for Call 1 (vibe→weights): run the model's REASONING pass
|
| 153 |
# (enable_thinking, MiniCPM5-1B is hybrid-reasoning) or the fast no-think path.
|
| 154 |
+
# DEFAULT no-think: the live A/B was decisive — thinking ran ~26s (3× no-think's
|
| 155 |
+
# ~9s, 3× the ZeroGPU slice) and *corrupted* the JSON (zeroed the matching
|
| 156 |
+
# categories), while no-think returned clean output. Reasoning did not help this
|
| 157 |
+
# short scoring task. Flip DISCOVERROUTE_VIBE_THINKING=1 to re-run the comparison;
|
| 158 |
+
# the chosen mode is recorded on every trace row.
|
| 159 |
VIBE_THINKING = os.environ.get(
|
| 160 |
+
"DISCOVERROUTE_VIBE_THINKING", "0").lower() in ("1", "true", "on")
|
| 161 |
|
| 162 |
# --- Trace logging (Open Trace) ----------------------------------------------
|
| 163 |
# Every inference call logs a row locally to logs/traces.jsonl; when a write
|
|
@@ -15,28 +15,33 @@ import time
|
|
| 15 |
|
| 16 |
from discoverroute.interpret import mapping
|
| 17 |
|
| 18 |
-
# NOTE on prompt design
|
| 19 |
-
# `"cafe": 0.0-1.0`
|
| 20 |
-
#
|
| 21 |
-
#
|
| 22 |
-
#
|
| 23 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
SYSTEM_PROMPT = (
|
| 25 |
"You convert a walk/ride 'vibe' into place-type preference weights for a "
|
| 26 |
"routing engine.\n"
|
| 27 |
-
"For
|
| 28 |
-
"
|
| 29 |
-
"to
|
| 30 |
-
"
|
| 31 |
-
"
|
| 32 |
-
"
|
| 33 |
-
"
|
| 34 |
-
"
|
| 35 |
-
"
|
| 36 |
-
"
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
|
|
|
| 40 |
)
|
| 41 |
|
| 42 |
REQUIRED_KEYS = (
|
|
|
|
| 15 |
|
| 16 |
from discoverroute.interpret import mapping
|
| 17 |
|
| 18 |
+
# NOTE on prompt design (two parrot traps, both seen live in the traces):
|
| 19 |
+
# 1) the original schema showed each key as `"cafe": 0.0-1.0`; the 1B parroted
|
| 20 |
+
# the `0.0` as the value → all-zero weighting → tasteless route.
|
| 21 |
+
# 2) replacing it with ONE worked JSON example backfired worse: the model copied
|
| 22 |
+
# the example's numbers VERBATIM regardless of the vibe (a "quiet green wander"
|
| 23 |
+
# came back byte-identical to the "riverside picnic" example).
|
| 24 |
+
# Lesson: give the 1B no concrete number-set to copy. State the 0..1 meaning in
|
| 25 |
+
# words, teach the *mapping* with short word→type cues (not a full object), and
|
| 26 |
+
# forbid the degenerate answers. Wrong/empty output is still caught downstream
|
| 27 |
+
# (_is_degenerate → embed tier), so the route is never left tasteless.
|
| 28 |
SYSTEM_PROMPT = (
|
| 29 |
"You convert a walk/ride 'vibe' into place-type preference weights for a "
|
| 30 |
"routing engine.\n"
|
| 31 |
+
"For EACH place type listed below, output a number from 0 to 1 for how "
|
| 32 |
+
"strongly THIS vibe calls for it: 0 = irrelevant, ~0.5 = a little, 0.8-1.0 = "
|
| 33 |
+
"central to the vibe. A typical vibe strongly wants only two to four types — "
|
| 34 |
+
"score those high and keep the rest low.\n"
|
| 35 |
+
"Read the actual words of the vibe and score from them, for example: "
|
| 36 |
+
"'green' or 'park' or 'nature' → park and green high; 'quiet' or 'calm' → "
|
| 37 |
+
"quiet high; 'lively' or 'buzzing' → busy and bar high; 'coffee' or 'café' → "
|
| 38 |
+
"cafe high; 'books' → bookshop high; 'history' or 'old' → historic high. "
|
| 39 |
+
"Never make every value 0, and never make every value the same.\n"
|
| 40 |
+
"detour_budget_multiplier: 0.5 = keep it direct, up to 2.0 = big detours "
|
| 41 |
+
"welcome.\n"
|
| 42 |
+
"Output ONLY a JSON object (no prose, no markdown) with EXACTLY these keys, "
|
| 43 |
+
"in this order: cafe, park, bookshop, museum, bakery, restaurant, bar, "
|
| 44 |
+
"viewpoint, market, quiet, green, historic, busy, detour_budget_multiplier."
|
| 45 |
)
|
| 46 |
|
| 47 |
REQUIRED_KEYS = (
|