agharsallah commited on
Commit
c1656a8
·
1 Parent(s): c1a916b

feat: Enhance model routing and deployment flexibility

Browse files

- Introduced `vllm_version` to allow per-model inference stack pinning, enabling models to opt for nightly builds when necessary.
- Updated image building logic to handle model-specific vLLM versions, ensuring compatibility with various architectures.
- Added `model_endpoint` to `AgentManifest` for explicit model binding, allowing agents to route to specific models instead of relying solely on profiles.
- Enhanced `ModelRouter` to resolve specific catalogue models, improving routing accuracy and flexibility.
- Updated UI components in the Fishbowl Lab to support new model selection features, ensuring only deployable models are presented.
- Improved tests to validate new model selection and routing behaviors, ensuring robustness and adherence to expected functionality.

docs/adr/0022-per-agent-explicit-model-binding.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ADR-0022: Per-Agent Explicit Model Binding (`model_endpoint`)
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ ADR-0010 gave per-agent model selection through four **logical tiers**
10
+ (`tiny`/`fast`/`balanced`/`strong`), and `config/models.yaml` binds each tier to one
11
+ concrete catalogue model. That is the right default, but it has two limits:
12
+
13
+ 1. **A cast can express at most four distinct models** — one per tier. Two agents on
14
+ the same tier always share a model.
15
+ 2. **The unbound specialist models are unreachable.** Several catalogue entries have
16
+ `profile=None` (Nemotron Cascade 14B, Nemotron 30B, MiniCPM-o) precisely so they do
17
+ *not* displace a tier default — but then no manifest could ever cast them.
18
+
19
+ Both bite the hackathon strategy directly: the unfair advantage is running *different
20
+ sponsor models in one cast* (Judge → Nemotron, a worker → MiniCPM) to qualify for
21
+ multiple tracks from a single submission. And the Fishbowl Lab needed to let a user pick
22
+ concrete Modal-hosted models per cast member — with the pick actually driving the run
23
+ (the Lab's model controls were previously cosmetic: `on_summon` ignored
24
+ `collect_world_config` and always built the scenario's default cast).
25
+
26
+ ## Decision
27
+
28
+ Add an optional, additive per-agent override that names a **specific catalogue model**,
29
+ leaving the tier system as the default and fallback.
30
+
31
+ - **Manifest.** `AgentManifest` gains `model_endpoint: str | None = None` — a
32
+ `modal/catalogue.py` endpoint slug. `None` → route by `model_profile` (unchanged).
33
+ - **Routing.** `ManifestAgent` routes by a **route key** —
34
+ `self._route_key = model_endpoint or model_profile` — and calls
35
+ `router.for_profile(self._route_key)`. The `ModelRouter` already accepts any key;
36
+ `_spec_for` now resolves a non-tier key against the catalogue (`_catalogue_spec`:
37
+ `modal_catalogue.binding_for(key)` for the live model string / endpoint URL / api key,
38
+ with decoding inherited from the model's tier — an unbound specialist → `balanced`).
39
+ An unknown non-tier key degrades to the `fast` tier rather than crashing. Offline this
40
+ path is never reached: `_build` serves the deterministic stub for any key, with the key
41
+ folded into the stub's `variant` so a different pick still varies (reproducible) output.
42
+ - **Composed runs.** `Registry.from_world(world)` builds an in-memory registry from a
43
+ validated `WorldConfig`, so a UI- (or LLM-) composed run flows through the same
44
+ `build_scenario` / `build_router` / `governor_for` path as a config-file run.
45
+ - **Fishbowl Lab.** The cast section is a `@gr.render` over the scenario: one model
46
+ `gr.Dropdown` per non-judge player (the Judge picks in §04), its choices sourced *only*
47
+ from `modal_catalogue.entries()`. Picks accumulate in a `cast_models` state;
48
+ `collect_world_config` maps each onto the agent's `model_endpoint` (re-checking the key
49
+ against the catalogue), and `Summon` runs the composed world. Only catalogue-hosted
50
+ models are offerable, and the selection is load-bearing.
51
+
52
+ ## Consequences
53
+
54
+ - A cast can pin **any** catalogue model per agent, including the unbound specialists —
55
+ enabling genuine multi-sponsor-model casts from one engine, one submission.
56
+ - The tier abstraction (ADR-0010) is untouched: it remains the default, the decoding
57
+ source, the offline-variant tag, and the fallback. `model_endpoint` is purely additive,
58
+ so every existing manifest, scenario, and test is byte-identical (defaults to `None`).
59
+ - The Lab's model picker is now functional, not cosmetic: the model you choose is the
60
+ model that speaks (offline → the deterministic stub, demo still reproducible). A bad
61
+ compose degrades to the scenario's default cast, so Summon never breaks the demo.
62
+ - A run cannot point at an undeployed model: the UI offers only catalogue entries, and
63
+ `collect_world_config` re-validates the key, dropping anything out-of-band or stale.
64
+ - Offline determinism is preserved end-to-end (the route key, not just the tier, seeds
65
+ the stub).
66
+
67
+ ## Alternatives considered
68
+
69
+ - **Per-tier rebinding** (let the run choose which catalogue model backs each of the four
70
+ tiers): zero engine change, but still capped at four distinct models and still cannot
71
+ reach the unbound specialists. Rejected as too weak for the multi-sponsor goal.
72
+ - **Widening `ModelProfile` to an arbitrary `str`**: would dissolve the tier contract that
73
+ drives decoding defaults, the `MODEL_<TIER>` env overrides, and the offline variant.
74
+ Rejected in favour of a separate, additive field that keeps both concepts crisp.
75
+
76
+ ## Code
77
+
78
+ - `src/core/manifest.py` — `AgentManifest.model_endpoint`
79
+ - `src/agents/base.py` — `ManifestAgent._route_key`
80
+ - `src/models/router.py` — `ModelRouter._spec_for` / `_catalogue_spec`
81
+ - `src/core/registry.py` — `Registry.from_world`
82
+ - `src/ui/fishbowl/lab.py` — `model_choices`, `_cast_defaults`, `_judge_manifest`, the
83
+ cast `gr.render`, `collect_world_config`
84
+ - `src/ui/fishbowl/app.py` — `_compose_session`, the `Summon` wiring
85
+
86
+ See also: ADR-0010 (logical-profile routing), ADR-0011 (declarative validatable config),
87
+ ADR-0015 (LiteLLM gateway to Modal models), ADR-0019 (single model catalogue), ADR-0021
88
+ (Fishbowl Gradio presenter).
docs/architecture/fishbowl-ui.md CHANGED
@@ -47,14 +47,22 @@ play-head, so concurrent visitors never share a world.
47
  ### The Lab — compose a run
48
 
49
  `lab.py` (`build_lab`) is the full interactive composer (ADR-0021, decision 4): a
50
- scenario grid, premise / seed / world text fields, a narrator `gr.Dropdown`, an
51
- **editable cast `gr.Dataframe`** (per-agent model→profile map + temperature override),
52
- a judge `gr.Group`, a tools `gr.CheckboxGroup`, and budget `gr.Number`/`gr.Slider`
53
- controls. "Surprise me" rerolls a cast; "Summon" collects the inputs into a per-run
54
- `WorldConfig`, runs it through `validate_world()` (ADR-0011), builds the `Conductor`,
55
- and switches to The Show. The Lab's abstract model choices map onto the four engine
56
- profiles (`tiny`/`fast`/`balanced`/`strong`, ADR-0010); per-agent temperature is a
57
- per-run override, since temperature is otherwise per-*profile* in `config/models.yaml`.
 
 
 
 
 
 
 
 
58
 
59
  ### The Show — watch it unfold
60
 
 
47
  ### The Lab — compose a run
48
 
49
  `lab.py` (`build_lab`) is the full interactive composer (ADR-0021, decision 4): a
50
+ scenario grid, premise / seed / world text fields, a narrator `gr.Dropdown`, a
51
+ **per-cast model picker**, a judge `gr.Group`, a tools `gr.CheckboxGroup`, and budget
52
+ `gr.Number`/`gr.Slider` controls.
53
+
54
+ The cast picker is a `@gr.render` over the scenario: one row per player (name + a model
55
+ `gr.Dropdown`), re-rendered as the cast changes. Crucially, **every dropdown offers only
56
+ the models actually hosted on Modal** — its choices come from `modal_catalogue.entries()`
57
+ (the catalogue is the single source of truth and loads offline), so you can't cast a model
58
+ that isn't deployable. Each pick writes the chosen endpoint slug into a `cast_models`
59
+ `gr.State` (`{agent_name: endpoint_key}`); the Judge gets its own catalogue dropdown in
60
+ §04. "Surprise me" rerolls a cast; **"Summon" makes the choice real**: `collect_world_config`
61
+ maps each selection onto the agent's `model_endpoint` (ADR-0022), runs the per-run
62
+ `WorldConfig` through `validate_world()` (ADR-0011), and `Registry.from_world()` builds a
63
+ `Conductor` on the exact same engine path as a config-file run — so the model you pick is
64
+ the model that speaks (offline → the deterministic stub, demo still reproducible). A bad
65
+ compose degrades to the scenario's default cast, so Summon never breaks the demo.
66
 
67
  ### The Show — watch it unfold
68
 
docs/architecture/manifest-spec.md CHANGED
@@ -25,6 +25,7 @@ class AgentManifest(BaseModel):
25
 
26
  # Model
27
  model_profile: ModelProfile # tiny | fast | balanced | strong
 
28
 
29
  # Memory
30
  memory: MemoryConfig # window, use_salience, salience_top_k, reflection_threshold
@@ -98,6 +99,15 @@ Logical model tier. Resolved to a concrete model name at runtime:
98
 
99
  The pattern: workers use `fast` or `tiny`; the judge and reflector use `balanced` or `strong`.
100
 
 
 
 
 
 
 
 
 
 
101
  ### `memory.window`
102
  Number of recent visible events to include in every prompt (recency-window mode).
103
  Default: 8. Reduce to 4–5 for very small models.
 
25
 
26
  # Model
27
  model_profile: ModelProfile # tiny | fast | balanced | strong
28
+ model_endpoint: str | None # optional: pin ONE catalogue model, overriding the tier
29
 
30
  # Memory
31
  memory: MemoryConfig # window, use_salience, salience_top_k, reflection_threshold
 
99
 
100
  The pattern: workers use `fast` or `tiny`; the judge and reflector use `balanced` or `strong`.
101
 
102
+ ### `model_endpoint`
103
+ Optional escape hatch from tiers to a **specific served model**: a `modal/catalogue.py`
104
+ endpoint slug (e.g. `minicpm-4-1-8b`) the router binds this agent to, overriding
105
+ `model_profile`. `None` (default) → route by tier. Lets a cast mix concrete sponsor
106
+ models (one mind on MiniCPM, the Judge on Nemotron Cascade), including the *unbound
107
+ specialist* models no tier defaults to. Offline it folds into the deterministic stub like
108
+ any tier, so demos stay reproducible. This is what the Fishbowl Lab's per-cast model picker
109
+ writes. See [model-routing.md](model-routing.md) and ADR-0022.
110
+
111
  ### `memory.window`
112
  Number of recent visible events to include in every prompt (recency-window mode).
113
  Default: 8. Reduce to 4–5 for very small models.
docs/architecture/model-routing.md CHANGED
@@ -17,13 +17,38 @@ names a model.
17
  ## How a turn resolves a model
18
 
19
  ```
20
- manifest.model_profile ──► ModelRouter.for_profile(profile) ──► ModelProvider
21
- (e.g. "tiny") (cached per profile) (concrete model)
22
  ```
23
 
24
- `ManifestAgent._complete()` calls `router.for_profile(self.manifest.model_profile)`
25
- every turn and records the provider's `last_usage` so the conductor can meter
26
- tokens and, on the live path, real cost — into the Governor.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  ## Transport: the LiteLLM gateway (live path)
29
 
@@ -118,7 +143,9 @@ everything on the big model.
118
 
119
  ## Code
120
 
121
- - `src/models/router.py` — `ModelRouter`, `ProfileSpec`, `_PROFILE_DECODING`
 
 
122
  - `src/models/litellm_provider.py` — `LiteLLMProvider` (live transport, real cost)
123
  - `src/models/modal_catalogue.py` — engine view of the catalogue (key → binding)
124
  - `src/core/manifest.py` — `resolve_model()` (env → catalogue default)
 
17
  ## How a turn resolves a model
18
 
19
  ```
20
+ manifest.model_endpoint or model_profile ──► ModelRouter.for_profile(key) ──► ModelProvider
21
+ (the agent's "route key") (cached per key) (concrete model)
22
  ```
23
 
24
+ `ManifestAgent` computes a **route key** — `self._route_key`, the explicit
25
+ `model_endpoint` when set, else the `model_profile` tier and calls
26
+ `router.for_profile(self._route_key)` every turn, recording the provider's
27
+ `last_usage` so the conductor can meter tokens (and, live, real cost) into the
28
+ Governor. The router accepts either kind of key: a tier resolves to the profile
29
+ default, a catalogue endpoint slug to that specific model's binding.
30
+
31
+ ## Pinning a specific model (`model_endpoint`)
32
+
33
+ Tiers are the default, but a manifest can pin one mind to a **specific catalogue
34
+ model** by setting `model_endpoint` to an endpoint slug from `modal/catalogue.py`
35
+ (e.g. `minicpm-4-1-8b`). This overrides the tier and is how a cast mixes concrete
36
+ sponsor models — one worker on MiniCPM, the Judge on Nemotron Cascade — including the
37
+ *unbound specialist* models that no tier defaults to. See ADR-0022.
38
+
39
+ ```
40
+ ModelRouter._spec_for(key)
41
+ key in specs → that ProfileSpec (the four tiers from models.yaml)
42
+ key is a catalogue endpoint → _catalogue_spec(key): binding_for(key) + the model's
43
+ tier decoding (unbound specialist → balanced defaults)
44
+ unknown non-tier key → degrade to the fast tier (never crash)
45
+ ```
46
+
47
+ Offline this path is never reached — `_build` serves the deterministic stub for any
48
+ key, with the key folded into the stub's `variant`, so picking a different model still
49
+ varies the (reproducible) output. The **Fishbowl Lab** writes `model_endpoint` from
50
+ its per-cast model picker, so the model you choose in the UI is the model that runs
51
+ (see [fishbowl-ui.md](fishbowl-ui.md)).
52
 
53
  ## Transport: the LiteLLM gateway (live path)
54
 
 
143
 
144
  ## Code
145
 
146
+ - `src/models/router.py` — `ModelRouter`, `ProfileSpec`, `_PROFILE_DECODING`, `_catalogue_spec()` (endpoint key → binding)
147
+ - `src/agents/base.py` — `ManifestAgent._route_key` (endpoint-or-tier)
148
+ - `src/core/registry.py` — `Registry.from_world()` (a UI/LLM-composed run on the same path)
149
  - `src/models/litellm_provider.py` — `LiteLLMProvider` (live transport, real cost)
150
  - `src/models/modal_catalogue.py` — engine view of the catalogue (key → binding)
151
  - `src/core/manifest.py` — `resolve_model()` (env → catalogue default)
docs/schema/agent-manifest.md CHANGED
@@ -28,6 +28,8 @@ schedule:
28
 
29
  # Model (resolved to a concrete small model by the ModelRouter)
30
  model_profile: fast # tiny ≤4B | fast ≤7B | balanced ≤13B | strong ≤32B
 
 
31
 
32
  # Memory (a view over the ledger, not separate state)
33
  memory:
@@ -52,6 +54,12 @@ output_extra_fields: [] # extra payload fields the model is asked for, e.
52
  reactive, periodic, or both. Cadence is per-agent; scenarios don't schedule.
53
  - **`model_profile`** never names a model; the router (config/env) does. Mix
54
  tiers freely across a cast.
 
 
 
 
 
 
55
  - **`handler`** stays `null` for the common case (the generic `ManifestAgent`).
56
  Set it to a key registered via `@register_handler` for agents that call tools or
57
  need custom prompt logic; the YAML still supplies all declarative fields.
 
28
 
29
  # Model (resolved to a concrete small model by the ModelRouter)
30
  model_profile: fast # tiny ≤4B | fast ≤7B | balanced ≤13B | strong ≤32B
31
+ model_endpoint: null # optional: pin ONE specific catalogue model (modal/catalogue.py
32
+ # endpoint slug, e.g. minicpm-4-1-8b), overriding the tier above
33
 
34
  # Memory (a view over the ledger, not separate state)
35
  memory:
 
54
  reactive, periodic, or both. Cadence is per-agent; scenarios don't schedule.
55
  - **`model_profile`** never names a model; the router (config/env) does. Mix
56
  tiers freely across a cast.
57
+ - **`model_endpoint`** is the escape hatch from tiers to a *specific* served model:
58
+ a `modal/catalogue.py` endpoint slug the router resolves to that model's live
59
+ binding (overriding `model_profile`). `null` → route by tier. This is how a cast
60
+ pins concrete sponsor models — one mind on MiniCPM, the Judge on Nemotron — and what
61
+ the Fishbowl Lab's per-cast model picker writes. Offline it folds into the
62
+ deterministic stub like any tier, so demos stay reproducible. See ADR-0022.
63
  - **`handler`** stays `null` for the common case (the generic `ManifestAgent`).
64
  Set it to a key registered via `@register_handler` for agents that call tools or
65
  need custom prompt logic; the YAML still supplies all declarative fields.
modal/docs/deploying.md CHANGED
@@ -81,9 +81,17 @@ changes needed:
81
  | `reasoning_parser` / `tool_call_parser` / `enable_auto_tool_choice` | OpenAI tool/reasoning features. |
82
  | `multimodal` / `mm_limits` | Image/audio/video inputs and per-prompt caps. |
83
  | `trust_remote_code` | Required by MiniCPM / Nemotron custom modeling code. |
 
84
  | `extra_vllm_args` | Raw `vllm serve` flags appended verbatim (escape hatch). |
85
  | `extra_pip` / `env` | Extra image deps / container env (escape hatch). |
86
 
 
 
 
 
 
 
 
87
  ### Add a model
88
 
89
  Append one `ModelConfig` to the appropriate provider list in `catalogue.py` (tag
 
81
  | `reasoning_parser` / `tool_call_parser` / `enable_auto_tool_choice` | OpenAI tool/reasoning features. |
82
  | `multimodal` / `mm_limits` | Image/audio/video inputs and per-prompt caps. |
83
  | `trust_remote_code` | Required by MiniCPM / Nemotron custom modeling code. |
84
+ | `vllm_version` | Per-model inference-stack pin (escape hatch); `None` = the default `VLLM_VERSION`, `"nightly"` = latest nightly wheel, else a pinned version. |
85
  | `extra_vllm_args` | Raw `vllm serve` flags appended verbatim (escape hatch). |
86
  | `extra_pip` / `env` | Extra image deps / container env (escape hatch). |
87
 
88
+ > **Per-model vLLM version.** The image pins `VLLM_VERSION` (see `service.py`) for
89
+ > reproducible deploys. A single model can override it via `vllm_version` when the
90
+ > pinned release can't serve its architecture — this is scoped to that model's image,
91
+ > so one model's bump never touches another provider's app. The Gemma 4 entries set
92
+ > `vllm_version="nightly"` (plus `transformers>=5.10.2` and `VLLM_USE_FLASHINFER_SAMPLER=0`)
93
+ > because the `gemma4_unified` architecture is unservable on the pinned release.
94
+
95
  ### Add a model
96
 
97
  Append one `ModelConfig` to the appropriate provider list in `catalogue.py` (tag
modal/service.py CHANGED
@@ -86,12 +86,16 @@ _BASE_ENV = {
86
  def build_image(cfg: ModelConfig) -> modal.Image:
87
  """Build the container image for a model. Layers are cached and shared, so
88
  text models that only differ in env reuse the same base layers."""
89
- image = (
90
- modal.Image.from_registry(CUDA_IMAGE, add_python=PYTHON_VERSION)
91
- .entrypoint([]) # drop the CUDA image's default entrypoint
92
- .uv_pip_install(f"vllm=={VLLM_VERSION}")
93
- .env(_BASE_ENV)
94
- )
 
 
 
 
95
  if cfg.extra_pip:
96
  image = image.uv_pip_install(*cfg.extra_pip)
97
  if cfg.env:
 
86
  def build_image(cfg: ModelConfig) -> modal.Image:
87
  """Build the container image for a model. Layers are cached and shared, so
88
  text models that only differ in env reuse the same base layers."""
89
+ image = modal.Image.from_registry(CUDA_IMAGE, add_python=PYTHON_VERSION).entrypoint(
90
+ []
91
+ ) # drop the CUDA image's default entrypoint
92
+ # vLLM version is per-model (defaults to the pinned VLLM_VERSION). A model can
93
+ # opt into a nightly wheel when the pinned release can't serve its architecture.
94
+ if cfg.vllm_version == "nightly":
95
+ image = image.uv_pip_install("vllm", pre=True, extra_index_url="https://wheels.vllm.ai/nightly")
96
+ else:
97
+ image = image.uv_pip_install(f"vllm=={cfg.vllm_version or VLLM_VERSION}")
98
+ image = image.env(_BASE_ENV)
99
  if cfg.extra_pip:
100
  image = image.uv_pip_install(*cfg.extra_pip)
101
  if cfg.env:
src/agents/base.py CHANGED
@@ -16,6 +16,7 @@ Backward compatibility: Phase-0/1 agents extend Agent directly and are
16
  unaffected. The conductor checks ``getattr(agent, "manifest", None)`` to
17
  decide whether manifest-based routing applies.
18
  """
 
19
  from __future__ import annotations
20
 
21
  from abc import ABC, abstractmethod
@@ -42,6 +43,7 @@ _REFLECTION_KIND = "agent.reflected"
42
 
43
  # ── minimal interface ─────────────────────────────────────────────────────────
44
 
 
45
  class Agent(ABC):
46
  name: str
47
 
@@ -58,6 +60,7 @@ class Agent(ABC):
58
 
59
  # ── manifest-driven base ──────────────────────────────────────────────────────
60
 
 
61
  class ManifestAgent(Agent):
62
  """Base class for manifest-driven agents.
63
 
@@ -141,6 +144,16 @@ class ManifestAgent(Agent):
141
 
142
  # ── model routing ─────────────────────────────────────────────────────────
143
 
 
 
 
 
 
 
 
 
 
 
144
  def _resolve_payload(
145
  self,
146
  role: str,
@@ -157,7 +170,7 @@ class ManifestAgent(Agent):
157
  instruction and run the tolerant parser as before. Token/cost usage is
158
  recorded from the provider in both paths.
159
  """
160
- provider = self.router.for_profile(self.manifest.model_profile)
161
  if hasattr(provider, "complete_structured"):
162
  model = build_output_model(allowed, extra_fields)
163
  try:
@@ -176,7 +189,7 @@ class ManifestAgent(Agent):
176
 
177
  def _complete(self, role: str, prompt: str) -> str:
178
  """Route to the provider for this agent's profile and record token usage."""
179
- provider = self.router.for_profile(self.manifest.model_profile)
180
  raw = provider.complete(role, prompt)
181
  self.last_usage = dict(provider.last_usage)
182
  return raw
@@ -206,7 +219,7 @@ class ManifestAgent(Agent):
206
  f"RECENT MEMORY (events you witnessed)\n{memory}\n\n"
207
  "TASK\nSynthesise the above into ONE short, high-level belief about yourself or the "
208
  "world. It will replace raw memories in your future context.\n\n"
209
- 'OUTPUT FORMAT\nReply with a single JSON object and nothing else: '
210
  '{"kind": "agent.reflected", "text": "<one-sentence belief>"}'
211
  )
212
  raw = self._complete(self.manifest.name + "-reflect", prompt)
 
16
  unaffected. The conductor checks ``getattr(agent, "manifest", None)`` to
17
  decide whether manifest-based routing applies.
18
  """
19
+
20
  from __future__ import annotations
21
 
22
  from abc import ABC, abstractmethod
 
43
 
44
  # ── minimal interface ─────────────────────────────────────────────────────────
45
 
46
+
47
  class Agent(ABC):
48
  name: str
49
 
 
60
 
61
  # ── manifest-driven base ──────────────────────────────────────────────────────
62
 
63
+
64
  class ManifestAgent(Agent):
65
  """Base class for manifest-driven agents.
66
 
 
144
 
145
  # ── model routing ─────────────────────────────────────────────────────────
146
 
147
+ @property
148
+ def _route_key(self) -> str:
149
+ """Router key for this agent: the explicit ``model_endpoint`` catalogue key
150
+ when set (a specific served model), else the logical ``model_profile`` tier.
151
+
152
+ The router accepts either — a catalogue key resolves to that model's live
153
+ binding, a tier to the profile default — so an agent can be pinned to one
154
+ concrete Modal model without the engine naming a model anywhere (ADR-0022)."""
155
+ return self.manifest.model_endpoint or self.manifest.model_profile
156
+
157
  def _resolve_payload(
158
  self,
159
  role: str,
 
170
  instruction and run the tolerant parser as before. Token/cost usage is
171
  recorded from the provider in both paths.
172
  """
173
+ provider = self.router.for_profile(self._route_key)
174
  if hasattr(provider, "complete_structured"):
175
  model = build_output_model(allowed, extra_fields)
176
  try:
 
189
 
190
  def _complete(self, role: str, prompt: str) -> str:
191
  """Route to the provider for this agent's profile and record token usage."""
192
+ provider = self.router.for_profile(self._route_key)
193
  raw = provider.complete(role, prompt)
194
  self.last_usage = dict(provider.last_usage)
195
  return raw
 
219
  f"RECENT MEMORY (events you witnessed)\n{memory}\n\n"
220
  "TASK\nSynthesise the above into ONE short, high-level belief about yourself or the "
221
  "world. It will replace raw memories in your future context.\n\n"
222
+ "OUTPUT FORMAT\nReply with a single JSON object and nothing else: "
223
  '{"kind": "agent.reflected", "text": "<one-sentence belief>"}'
224
  )
225
  raw = self._complete(self.manifest.name + "-reflect", prompt)
src/core/manifest.py CHANGED
@@ -104,6 +104,16 @@ class AgentManifest(BaseModel):
104
  """Logical profile: resolved to a concrete model name by the provider.
105
  tiny=<=4B, fast=<=7B, balanced=<=13B, strong=<=32B."""
106
 
 
 
 
 
 
 
 
 
 
 
107
  # Memory
108
  memory: MemoryConfig = Field(default_factory=MemoryConfig)
109
 
 
104
  """Logical profile: resolved to a concrete model name by the provider.
105
  tiny=<=4B, fast=<=7B, balanced=<=13B, strong=<=32B."""
106
 
107
+ model_endpoint: str | None = None
108
+ """Optional catalogue key (``modal/catalogue.py`` endpoint slug, e.g.
109
+ ``"minicpm-4-1-8b"``) binding this agent to one *specific* served model,
110
+ overriding the tier default in ``model_profile``. None → route by profile.
111
+
112
+ This is how a cast mixes concrete sponsor models — one mind on MiniCPM, the
113
+ Judge on Nemotron — while ``model_profile`` stays the decoding/offline fallback.
114
+ The router resolves the key against the live catalogue (ADR-0022); offline it
115
+ folds into the deterministic stub like any profile, so demos stay reproducible."""
116
+
117
  # Memory
118
  memory: MemoryConfig = Field(default_factory=MemoryConfig)
119
 
src/core/registry.py CHANGED
@@ -22,7 +22,14 @@ from pathlib import Path
22
  import yaml
23
 
24
  from src.agents.base import Agent, ManifestAgent
25
- from src.core.config import GovernorConfig, ModelsConfig, ScenarioConfig, validate_agent, validate_scenario
 
 
 
 
 
 
 
26
  from src.core.governor import Governor
27
  from src.core.manifest import AgentManifest
28
  from src.models.router import ModelRouter, ProfileSpec
@@ -156,6 +163,21 @@ class Registry:
156
 
157
  return cls(agents=agents, scenarios=scenarios, models=models)
158
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
159
  # ── building ───────────────────────────────────────────────────────────────
160
 
161
  def build_router(self) -> ModelRouter:
 
22
  import yaml
23
 
24
  from src.agents.base import Agent, ManifestAgent
25
+ from src.core.config import (
26
+ GovernorConfig,
27
+ ModelsConfig,
28
+ ScenarioConfig,
29
+ WorldConfig,
30
+ validate_agent,
31
+ validate_scenario,
32
+ )
33
  from src.core.governor import Governor
34
  from src.core.manifest import AgentManifest
35
  from src.models.router import ModelRouter, ProfileSpec
 
163
 
164
  return cls(agents=agents, scenarios=scenarios, models=models)
165
 
166
+ @classmethod
167
+ def from_world(cls, world: WorldConfig) -> "Registry":
168
+ """Build an in-memory registry from a composed, validated :class:`WorldConfig`.
169
+
170
+ The in-memory mirror of :meth:`from_dir`: agents, scenarios, and model
171
+ bindings come straight off the world object instead of ``config/``. So a
172
+ run composed by the Lab (or an LLM) flows through the exact same
173
+ ``build_scenario`` / ``build_router`` / ``governor_for`` path as a
174
+ config-file run — emit a world, validate it, run it. See ADR-0011 / ADR-0022."""
175
+ return cls(
176
+ agents={a.name: a for a in world.agents},
177
+ scenarios={s.name: s for s in world.scenarios},
178
+ models=world.models,
179
+ )
180
+
181
  # ── building ───────────────────────────────────────────────────────────────
182
 
183
  def build_router(self) -> ModelRouter:
src/models/router.py CHANGED
@@ -101,13 +101,49 @@ class ModelRouter:
101
  def _spec_for(self, profile: str) -> ProfileSpec:
102
  if profile in self.specs:
103
  return self.specs[profile]
104
- decoding = _PROFILE_DECODING.get(profile, _PROFILE_DECODING["fast"])
 
 
 
 
 
 
 
 
 
 
 
105
  return ProfileSpec(
106
  model=resolve_model(profile), # type: ignore[arg-type]
107
  temperature=float(decoding["temperature"]),
108
  max_tokens=int(decoding["max_tokens"]),
109
  )
110
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
  # ── factory ─────────────────────────────────────────────────────────────
112
 
113
  @classmethod
 
101
  def _spec_for(self, profile: str) -> ProfileSpec:
102
  if profile in self.specs:
103
  return self.specs[profile]
104
+ # A key that is not one of the four tiers names a *specific* catalogue model
105
+ # (an agent's ``model_endpoint``): resolve it to that model's live binding so
106
+ # a cast can pin concrete Modal models (ADR-0022). Only reached on the live
107
+ # path — offline, ``_build`` serves the stub before this runs.
108
+ if profile not in _PROFILE_DECODING:
109
+ spec = self._catalogue_spec(profile)
110
+ if spec is not None:
111
+ return spec
112
+ # Unknown non-tier key with no catalogue match → degrade to the fast tier
113
+ # rather than crash ``resolve_model`` on a key it does not recognise.
114
+ profile = "fast"
115
+ decoding = _PROFILE_DECODING[profile]
116
  return ProfileSpec(
117
  model=resolve_model(profile), # type: ignore[arg-type]
118
  temperature=float(decoding["temperature"]),
119
  max_tokens=int(decoding["max_tokens"]),
120
  )
121
 
122
+ def _catalogue_spec(self, key: str) -> ProfileSpec | None:
123
+ """Build a :class:`ProfileSpec` from a catalogue endpoint *key*, or None.
124
+
125
+ The model string / endpoint URL / api key come from the catalogue + env
126
+ (``modal_catalogue.binding_for``); decoding inherits the model's tier default
127
+ (an unbound specialist model → the balanced tier). Returns None when the key
128
+ is not in the catalogue, so the caller can fall back gracefully."""
129
+ try:
130
+ from src.models import modal_catalogue
131
+
132
+ entry = modal_catalogue.entry_by_key(key)
133
+ if entry is None:
134
+ return None
135
+ binding = modal_catalogue.binding_for(key)
136
+ except Exception: # pragma: no cover - defensive: catalogue unavailable
137
+ return None
138
+ decoding = _PROFILE_DECODING.get(entry.get("profile") or "balanced", _PROFILE_DECODING["balanced"])
139
+ return ProfileSpec(
140
+ model=binding["model"],
141
+ base_url=binding["base_url"] or None,
142
+ api_key=binding["api_key"] or None,
143
+ temperature=float(decoding["temperature"]),
144
+ max_tokens=int(decoding["max_tokens"]),
145
+ )
146
+
147
  # ── factory ─────────────────────────────────────────────────────────────
148
 
149
  @classmethod
src/ui/fishbowl/app.py CHANGED
@@ -527,6 +527,16 @@ def _wire(
527
  summon_btn = _h(lab_handles, "summon", "summon_btn", "launch", "start")
528
  scenario_in = _h(lab_handles, "scenario", "scenario_select", "scenario_dropdown")
529
  seed_in = _h(lab_handles, "seed", "seed_in", "world_seed")
 
 
 
 
 
 
 
 
 
 
530
 
531
  def _scenario_title(value) -> str:
532
  """Resolve a Lab scenario selection (title or internal name) to a title key."""
@@ -537,11 +547,67 @@ def _wire(
537
  return title
538
  return _DEFAULT_TITLE
539
 
540
- # ── Summon: build a fresh session, reset, jump to the Show, render head ──────
541
- def on_summon(scenario_value, seed_value, layout, mind_reader):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
542
  title = _scenario_title(scenario_value)
543
  name = SCENARIOS.get(title, "")
544
- session = _new_session(name) if name else None
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
545
  if session is not None:
546
  session.reset((seed_value or "").strip())
547
  k = session.head
@@ -555,6 +621,14 @@ def _wire(
555
  summon_inputs = [
556
  scenario_in if scenario_in is not None else scenario_state,
557
  seed_in if seed_in is not None else blank_state, # empty seed when no widget
 
 
 
 
 
 
 
 
558
  layout_state,
559
  mind_reader_state,
560
  ]
@@ -568,30 +642,33 @@ def _wire(
568
  # Without this, switching to a new world (e.g. the spy game) leaves the premise,
569
  # seed, cast table, and narrator showing the previous scenario — and Summon would
570
  # genesis with the wrong seed. Refreshing them makes "compose a spy game" one click.
 
 
571
  _scenario_fields = [
572
  (_h(lab_handles, "premise"), "premise"),
573
  (seed_in, "seed"),
574
  (_h(lab_handles, "world"), "world"),
575
- (_h(lab_handles, "cast"), "cast"),
576
  (_h(lab_handles, "narrator"), "narrator"),
 
577
  ]
578
  _present_fields = [(handle, key) for handle, key in _scenario_fields if handle is not None]
579
 
580
  if scenario_in is not None and _present_fields:
581
 
582
  def on_scenario_change(scenario_value):
583
- from src.ui.fishbowl.lab import _cast_rows_for
584
 
585
  cfg = _registry.scenarios.get(SCENARIOS.get(_scenario_title(scenario_value), ""))
586
  if cfg is None:
587
  return tuple(gr.update() for _ in _present_fields)
588
  seeds = list(cfg.example_seeds) or [cfg.default_seed]
 
589
  updates = {
590
  "premise": gr.update(value=cfg.goal),
591
  "seed": gr.update(choices=seeds, value=cfg.default_seed),
592
  "world": gr.update(value=cfg.genesis_text or ""),
593
- "cast": gr.update(value=_cast_rows_for(cfg)),
594
  "narrator": gr.update(value=scenario_voice(cfg.name)),
 
595
  }
596
  return tuple(updates[key] for _handle, key in _present_fields)
597
 
 
527
  summon_btn = _h(lab_handles, "summon", "summon_btn", "launch", "start")
528
  scenario_in = _h(lab_handles, "scenario", "scenario_select", "scenario_dropdown")
529
  seed_in = _h(lab_handles, "seed", "seed_in", "world_seed")
530
+ # Composer inputs — the per-cast Modal model picks (cast_models) and the run knobs;
531
+ # all looked up defensively so the shell still runs if the Lab omits a widget.
532
+ premise_in = _h(lab_handles, "premise")
533
+ cast_models_in = _h(lab_handles, "cast_models")
534
+ judge_model_in = _h(lab_handles, "judge_model")
535
+ judge_policy_in = _h(lab_handles, "judge_policy")
536
+ judge_strictness_in = _h(lab_handles, "judge_strictness")
537
+ tools_in = _h(lab_handles, "tools")
538
+ tokens_in = _h(lab_handles, "tokens")
539
+ max_rounds_in = _h(lab_handles, "max_rounds")
540
 
541
  def _scenario_title(value) -> str:
542
  """Resolve a Lab scenario selection (title or internal name) to a title key."""
 
547
  return title
548
  return _DEFAULT_TITLE
549
 
550
+ def _compose_session(name, **knobs):
551
+ """Build a session for a Lab-composed run: the selected Modal models drive the
552
+ cast (ADR-0022). The composed WorldConfig flows through ``Registry.from_world``
553
+ onto the exact same engine path as a config-file run. Any compose/validate error
554
+ degrades to the scenario's default cast so Summon always yields a runnable show
555
+ (and with no credentials the deterministic stub drives it, demo reproducible)."""
556
+ from src.core.registry import Registry
557
+ from src.ui.fishbowl.lab import collect_world_config
558
+
559
+ try:
560
+ world = collect_world_config(
561
+ scenario=name,
562
+ premise=knobs.get("premise") or "",
563
+ seed=knobs.get("seed") or "",
564
+ cast_models=knobs.get("cast_models") if isinstance(knobs.get("cast_models"), dict) else {},
565
+ judge_policy=knobs.get("judge_policy") or "Majority Vote",
566
+ judge_model=knobs.get("judge_model") or "",
567
+ judge_strictness=knobs.get("judge_strictness")
568
+ if isinstance(knobs.get("judge_strictness"), (int, float))
569
+ else 50,
570
+ tools=knobs.get("tools") if isinstance(knobs.get("tools"), list) else [],
571
+ tokens=knobs.get("tokens") if isinstance(knobs.get("tokens"), (int, float)) else None,
572
+ max_rounds=knobs.get("max_rounds") if isinstance(knobs.get("max_rounds"), (int, float)) else None,
573
+ )
574
+ return FishbowlSession(name, registry=Registry.from_world(world), tools=_tools)
575
+ except Exception:
576
+ return _new_session(name) # bad compose → default cast; Summon never breaks
577
+
578
+ # ── Summon: build a fresh session from the composed run, reset, jump to the Show ──
579
+ def on_summon(
580
+ scenario_value,
581
+ seed_value,
582
+ premise,
583
+ cast_models,
584
+ judge_model,
585
+ judge_policy,
586
+ judge_strictness,
587
+ tools,
588
+ tokens,
589
+ max_rounds,
590
+ layout,
591
+ mind_reader,
592
+ ):
593
  title = _scenario_title(scenario_value)
594
  name = SCENARIOS.get(title, "")
595
+ session = (
596
+ _compose_session(
597
+ name,
598
+ premise=premise,
599
+ seed=seed_value,
600
+ cast_models=cast_models,
601
+ judge_model=judge_model,
602
+ judge_policy=judge_policy,
603
+ judge_strictness=judge_strictness,
604
+ tools=tools,
605
+ tokens=tokens,
606
+ max_rounds=max_rounds,
607
+ )
608
+ if name
609
+ else None
610
+ )
611
  if session is not None:
612
  session.reset((seed_value or "").strip())
613
  k = session.head
 
621
  summon_inputs = [
622
  scenario_in if scenario_in is not None else scenario_state,
623
  seed_in if seed_in is not None else blank_state, # empty seed when no widget
624
+ premise_in if premise_in is not None else blank_state,
625
+ cast_models_in if cast_models_in is not None else blank_state,
626
+ judge_model_in if judge_model_in is not None else blank_state,
627
+ judge_policy_in if judge_policy_in is not None else blank_state,
628
+ judge_strictness_in if judge_strictness_in is not None else blank_state,
629
+ tools_in if tools_in is not None else blank_state,
630
+ tokens_in if tokens_in is not None else blank_state,
631
+ max_rounds_in if max_rounds_in is not None else blank_state,
632
  layout_state,
633
  mind_reader_state,
634
  ]
 
642
  # Without this, switching to a new world (e.g. the spy game) leaves the premise,
643
  # seed, cast table, and narrator showing the previous scenario — and Summon would
644
  # genesis with the wrong seed. Refreshing them makes "compose a spy game" one click.
645
+ # The cast picker re-seeds itself (it is a gr.render over the scenario), so it is not
646
+ # in this list; we re-seed the static fields plus the Judge's default Modal model.
647
  _scenario_fields = [
648
  (_h(lab_handles, "premise"), "premise"),
649
  (seed_in, "seed"),
650
  (_h(lab_handles, "world"), "world"),
 
651
  (_h(lab_handles, "narrator"), "narrator"),
652
+ (_h(lab_handles, "judge_model"), "judge_model"),
653
  ]
654
  _present_fields = [(handle, key) for handle, key in _scenario_fields if handle is not None]
655
 
656
  if scenario_in is not None and _present_fields:
657
 
658
  def on_scenario_change(scenario_value):
659
+ from src.ui.fishbowl.lab import _default_model_key, _judge_manifest
660
 
661
  cfg = _registry.scenarios.get(SCENARIOS.get(_scenario_title(scenario_value), ""))
662
  if cfg is None:
663
  return tuple(gr.update() for _ in _present_fields)
664
  seeds = list(cfg.example_seeds) or [cfg.default_seed]
665
+ judge = _judge_manifest(cfg)
666
  updates = {
667
  "premise": gr.update(value=cfg.goal),
668
  "seed": gr.update(choices=seeds, value=cfg.default_seed),
669
  "world": gr.update(value=cfg.genesis_text or ""),
 
670
  "narrator": gr.update(value=scenario_voice(cfg.name)),
671
+ "judge_model": gr.update(value=_default_model_key(judge) if judge else None),
672
  }
673
  return tuple(updates[key] for _handle, key in _present_fields)
674
 
src/ui/fishbowl/lab.py CHANGED
@@ -23,7 +23,9 @@ from __future__ import annotations
23
  import gradio as gr
24
 
25
  from src.core.config import ScenarioConfig, validate_scenario, validate_world
 
26
  from src.core.registry import default_registry
 
27
  from src.ui.fishbowl.adapter import VOICES, scenario_voice
28
 
29
  # ── design vocabulary (mirrors ui/raw/lab.jsx) ──────────────────────────────────
@@ -37,9 +39,6 @@ JUDGE_POLICIES: list[str] = [
37
  "Judge's Whim",
38
  ]
39
 
40
- # Logical model profiles the engine understands (manifest.ModelProfile).
41
- MODEL_PROFILES: list[str] = ["tiny", "fast", "balanced", "strong"]
42
-
43
  # MCP tool grants the cast may reach for (friendly label, stored id). Mirrors
44
  # ui/raw/lab.jsx:MCP_TOOLS; the value is what we store on the run.
45
  TOOL_CHOICES: list[tuple[str, str]] = [
@@ -51,9 +50,6 @@ TOOL_CHOICES: list[tuple[str, str]] = [
51
  ("tts.speak · give it a voice", "tts.speak"),
52
  ]
53
 
54
- # Cast Dataframe columns (editable in the Lab).
55
- CAST_COLUMNS: list[str] = ["name", "archetype", "model_profile", "temp"]
56
-
57
  # The scenario we lead with — the hackathon's north-star world.
58
  _PREFERRED_SCENARIO = "thousand-token-wood"
59
 
@@ -77,25 +73,62 @@ def _scenario_by_title(title: str) -> ScenarioConfig | None:
77
  return None
78
 
79
 
80
- def _cast_rows_for(scenario: ScenarioConfig) -> list[list]:
81
- """Seed editable cast rows from a scenario's cast manifests.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
 
83
- Each row is ``[name, archetype, model_profile, temp]``. The archetype and
84
- model_profile come straight from the agent manifest; temp defaults to 0.8 (the
85
- model default) so the column is meaningful but editable.
86
- """
 
87
  registry = default_registry()
88
- rows: list[list] = []
89
  for agent_name in scenario.cast:
90
  manifest = registry.agents.get(agent_name)
91
- if manifest is None:
92
- # A scenario referencing an unknown agent shouldn't happen (validate_world
93
- # guards it) but degrade gracefully rather than crash the form.
94
- rows.append([agent_name, f"the {agent_name}", "fast", 0.8])
95
  continue
96
- archetype = manifest.archetype or f"the {manifest.role}"
97
- rows.append([manifest.name, archetype, manifest.model_profile, 0.8])
98
- return rows
 
99
 
100
 
101
  def _voice_choices() -> list[tuple[str, str]]:
@@ -113,8 +146,9 @@ def build_lab() -> dict[str, gr.components.Component]:
113
  Wires no callbacks and imports no sibling render/show module — the app shell
114
  (Unit 9) binds ``summon_btn`` to the session. Returns a dict of every handle a
115
  caller needs to read the composed run (keys: ``scenario, premise, seed, world,
116
- narrator, cast, judge_policy, judge_model, judge_strictness, tools, tokens,
117
- max_rounds, seed_num, cadence, summon_btn, surprise_btn``).
 
118
  """
119
  scenarios = _ordered_scenarios()
120
  first = scenarios[0]
@@ -160,25 +194,64 @@ def build_lab() -> dict[str, gr.components.Component]:
160
  lines=2,
161
  )
162
 
163
- # 03 — The Cast (editable Dataframe)
 
 
 
 
 
 
164
  with gr.Group():
165
  gr.Markdown(
166
- "**03 · The Cast** — bind any mind to any model "
167
- "(profile: tiny / fast / balanced / strong); watch the expensive one play"
168
- )
169
- handles["cast"] = gr.Dataframe(
170
- value=_cast_rows_for(first),
171
- headers=CAST_COLUMNS,
172
- datatype=["str", "str", "str", "number"],
173
- column_count=len(CAST_COLUMNS),
174
- # The grid re-seeds (name/archetype/model) when the scenario changes; it shows
175
- # the player roster — a scenario's judge/host is configured under §04, mirroring
176
- # the prototype's 4-player spy cast (ui/raw/data.js).
177
- row_count=len(first.cast),
178
- interactive=True,
179
- label="Cast",
180
  )
181
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
182
  # 04 — The Judge + 05 — Tools (side by side)
183
  with gr.Row():
184
  with gr.Group():
@@ -189,9 +262,10 @@ def build_lab() -> dict[str, gr.components.Component]:
189
  label="Policy preset",
190
  )
191
  handles["judge_model"] = gr.Dropdown(
192
- choices=MODEL_PROFILES,
193
- value="strong",
194
- label="Bound model profile",
 
195
  )
196
  handles["judge_strictness"] = gr.Slider(
197
  minimum=0,
@@ -244,7 +318,7 @@ def collect_world_config(
244
  scenario: str,
245
  premise: str,
246
  seed: str,
247
- cast_rows: list,
248
  judge_policy: str,
249
  judge_model: str,
250
  judge_strictness: float,
@@ -255,25 +329,29 @@ def collect_world_config(
255
  """Assemble + validate a per-run world from the Lab's form values.
256
 
257
  Returns the validated :class:`WorldConfig` (raising ``pydantic.ValidationError``
258
- on an incoherent run). This is the bridge Unit 9 uses to build a Conductor from a
259
- composed run.
260
-
261
- The base scenario (selected by its display *title*) supplies the cast roster and
262
- the agent manifests; the edited ``cast_rows`` (``[name, archetype, model_profile,
263
- temp]``) override each agent's ``model_profile`` and ``archetype`` non-destructively
264
- via :meth:`pydantic.BaseModel.model_copy`. The premise overrides the scenario goal,
265
- ``seed`` becomes its ``default_seed``, and the budget knobs feed the governor.
266
-
267
- The judge knobs (``judge_policy`` / ``judge_model`` / ``judge_strictness``) and the
268
- ``tools`` grant are accepted and shape-checked, but the deeper synthesis (turning a
269
- policy preset into a judge AgentManifest, wiring tool grants onto worker manifests) is
270
- deferred to the engine and TODO'd below — every dict this builds is still passed
271
- through ``validate_scenario`` / ``validate_world`` before return, so the contract
272
- ("emit, validate, run") holds. See ADR-0011.
273
-
274
- TODO(unit-9): map ``judge_policy`` / ``judge_model`` to a concrete judge AgentManifest
275
- and wire the selected ``tools`` onto each worker's manifest ``tools`` grant once the
276
- judge/tool contracts land in the live path.
 
 
 
 
277
  """
278
  registry = default_registry()
279
  base = _scenario_by_title(scenario)
@@ -283,28 +361,27 @@ def collect_world_config(
283
  if base is None:
284
  raise ValueError(f"unknown scenario {scenario!r} (have: {sorted(registry.scenarios)})")
285
 
286
- # Edits per cast row manifest overrides, keyed by name. Non-destructive: we
287
- # model_copy each manifest rather than mutating the cached registry instance.
288
- edits: dict[str, dict] = {}
289
- for row in cast_rows or []:
290
- if not row or row[0] in (None, ""):
291
- continue
292
- name = str(row[0]).strip()
293
- patch: dict = {}
294
- if len(row) > 1 and row[1] not in (None, ""):
295
- patch["archetype"] = str(row[1])
296
- if len(row) > 2 and str(row[2]).strip() in MODEL_PROFILES:
297
- patch["model_profile"] = str(row[2]).strip()
298
- edits[name] = patch
299
 
300
- # Build the per-run cast: every manifest the scenario references, with overrides.
 
 
301
  agents = []
302
  cast_names: list[str] = []
303
  for agent_name in base.cast:
304
  manifest = registry.agents.get(agent_name)
305
  if manifest is None:
306
  raise ValueError(f"scenario {base.name!r} references undefined agent {agent_name!r}")
307
- manifest = manifest.model_copy(update=edits.get(agent_name, {}))
 
 
 
 
 
308
  agents.append(manifest.model_dump(mode="python"))
309
  cast_names.append(manifest.name)
310
 
 
23
  import gradio as gr
24
 
25
  from src.core.config import ScenarioConfig, validate_scenario, validate_world
26
+ from src.core.manifest import AgentManifest
27
  from src.core.registry import default_registry
28
+ from src.models import modal_catalogue
29
  from src.ui.fishbowl.adapter import VOICES, scenario_voice
30
 
31
  # ── design vocabulary (mirrors ui/raw/lab.jsx) ──────────────────────────────────
 
39
  "Judge's Whim",
40
  ]
41
 
 
 
 
42
  # MCP tool grants the cast may reach for (friendly label, stored id). Mirrors
43
  # ui/raw/lab.jsx:MCP_TOOLS; the value is what we store on the run.
44
  TOOL_CHOICES: list[tuple[str, str]] = [
 
50
  ("tts.speak · give it a voice", "tts.speak"),
51
  ]
52
 
 
 
 
53
  # The scenario we lead with — the hackathon's north-star world.
54
  _PREFERRED_SCENARIO = "thousand-token-wood"
55
 
 
73
  return None
74
 
75
 
76
+ def model_choices() -> list[tuple[str, str]]:
77
+ """Dropdown choices for the Modal-hosted catalogue: ``(friendly label, endpoint key)``.
78
+
79
+ The single source of truth is ``modal/catalogue.py`` (read through the engine's
80
+ ``modal_catalogue`` view), so the Lab can *only* offer models that are actually
81
+ deployable — and the list loads offline (the catalogue is a plain stdlib file), so
82
+ the picker is populated even with no API key. Empty list → a stripped deployment
83
+ with no catalogue, in which case the cast falls back to the deterministic stub."""
84
+ choices: list[tuple[str, str]] = []
85
+ for entry in modal_catalogue.entries():
86
+ served = entry["served_model_id"].split("/")[-1]
87
+ params = f"{entry['params_b']:g}B" if entry.get("params_b") else "?"
88
+ tier = entry["profile"] or "specialist"
89
+ provider = entry["provider"].title()
90
+ choices.append((f"{served} · {params} · {tier} · {provider}", entry["key"]))
91
+ return choices
92
+
93
+
94
+ def _default_model_key(manifest: AgentManifest) -> str | None:
95
+ """Catalogue key a cast row defaults to: the manifest's explicit ``model_endpoint``,
96
+ else the catalogue's default model for its tier, else the first catalogue model (or
97
+ None when the catalogue is empty)."""
98
+ if manifest.model_endpoint:
99
+ return manifest.model_endpoint
100
+ tiered = modal_catalogue.default_key_for_profile(manifest.model_profile)
101
+ if tiered:
102
+ return tiered
103
+ entries = modal_catalogue.entries()
104
+ return entries[0]["key"] if entries else None
105
+
106
+
107
+ def _judge_manifest(scenario: ScenarioConfig) -> AgentManifest | None:
108
+ """The scenario's judge agent (first ``role == "judge"`` in the cast), or None."""
109
+ registry = default_registry()
110
+ for agent_name in scenario.cast:
111
+ manifest = registry.agents.get(agent_name)
112
+ if manifest is not None and manifest.role == "judge":
113
+ return manifest
114
+ return None
115
+
116
 
117
+ def _cast_defaults(scenario: ScenarioConfig) -> dict[str, str]:
118
+ """Default model selection for a scenario's *non-judge* cast (name endpoint key).
119
+
120
+ The Judge is bound under §04, so it is excluded here. Used to seed (and re-seed on
121
+ scenario change) the ``cast_models`` state the picker writes into."""
122
  registry = default_registry()
123
+ defaults: dict[str, str] = {}
124
  for agent_name in scenario.cast:
125
  manifest = registry.agents.get(agent_name)
126
+ if manifest is None or manifest.role == "judge":
 
 
 
127
  continue
128
+ key = _default_model_key(manifest)
129
+ if key:
130
+ defaults[agent_name] = key
131
+ return defaults
132
 
133
 
134
  def _voice_choices() -> list[tuple[str, str]]:
 
146
  Wires no callbacks and imports no sibling render/show module — the app shell
147
  (Unit 9) binds ``summon_btn`` to the session. Returns a dict of every handle a
148
  caller needs to read the composed run (keys: ``scenario, premise, seed, world,
149
+ narrator, cast_models, judge_policy, judge_model, judge_strictness, tools, tokens,
150
+ max_rounds, seed_num, cadence, summon_btn, surprise_btn``). ``cast_models`` is a
151
+ ``gr.State`` holding ``{agent_name: catalogue_endpoint_key}`` for the non-judge cast.
152
  """
153
  scenarios = _ordered_scenarios()
154
  first = scenarios[0]
 
194
  lines=2,
195
  )
196
 
197
+ # 03 — The Cast: one Modal-hosted model per mind. The picker offers ONLY models in
198
+ # the catalogue (modal/catalogue.py), and the choice drives the run (ADR-0022). A
199
+ # gr.render keeps one row per player as the scenario (and its cast size) changes; each
200
+ # row's dropdown writes the chosen endpoint key into the cast_models state below.
201
+ cast_models = gr.State(_cast_defaults(first))
202
+ handles["cast_models"] = cast_models
203
+ catalogue = model_choices()
204
  with gr.Group():
205
  gr.Markdown(
206
+ "**03 · The Cast** — bind each mind to a model **hosted on Modal** "
207
+ "(the only models you can pick); the Judge is set in §04"
 
 
 
 
 
 
 
 
 
 
 
 
208
  )
209
 
210
+ @gr.render(inputs=[handles["scenario"]])
211
+ def _render_cast(scenario_value, _choices=catalogue):
212
+ scenario = _scenario_by_title(scenario_value) or default_registry().scenarios.get(scenario_value)
213
+ if scenario is None:
214
+ gr.Markdown("_No scenario selected._")
215
+ return
216
+ registry = default_registry()
217
+ shown = 0
218
+ for agent_name in scenario.cast:
219
+ manifest = registry.agents.get(agent_name)
220
+ if manifest is None or manifest.role == "judge":
221
+ continue # the Judge is configured under §04
222
+ shown += 1
223
+ with gr.Row():
224
+ gr.Markdown(
225
+ f"**{manifest.name}**<br/>"
226
+ f"<span style='opacity:.7'>{manifest.archetype or f'the {manifest.role}'}</span>"
227
+ )
228
+ picker = gr.Dropdown(
229
+ choices=_choices,
230
+ value=_default_model_key(manifest),
231
+ label="model · Modal",
232
+ interactive=bool(_choices),
233
+ scale=2,
234
+ )
235
+
236
+ # Capture the agent name per row; the dropdown writes its key into the
237
+ # shared cast_models dict the Summon handler reads.
238
+ def _set_model(key, state, _name=manifest.name):
239
+ return {**(state or {}), _name: key}
240
+
241
+ picker.change(_set_model, inputs=[picker, cast_models], outputs=[cast_models])
242
+ if not shown:
243
+ gr.Markdown("_This scenario has no selectable players._")
244
+ elif not _choices:
245
+ gr.Markdown("_No Modal models in the catalogue — the cast runs the deterministic stub._")
246
+
247
+ # Switching scenarios re-seeds the model picks to the new cast's defaults so a stale
248
+ # override from the previous world never leaks into the run.
249
+ def _reset_cast_models(scenario_value):
250
+ scn = _scenario_by_title(scenario_value) or default_registry().scenarios.get(scenario_value)
251
+ return _cast_defaults(scn) if scn else {}
252
+
253
+ handles["scenario"].change(_reset_cast_models, inputs=[handles["scenario"]], outputs=[cast_models])
254
+
255
  # 04 — The Judge + 05 — Tools (side by side)
256
  with gr.Row():
257
  with gr.Group():
 
262
  label="Policy preset",
263
  )
264
  handles["judge_model"] = gr.Dropdown(
265
+ choices=catalogue,
266
+ value=_default_model_key(_judge_manifest(first)) if _judge_manifest(first) else None,
267
+ label="Judge model · Modal",
268
+ interactive=bool(catalogue),
269
  )
270
  handles["judge_strictness"] = gr.Slider(
271
  minimum=0,
 
318
  scenario: str,
319
  premise: str,
320
  seed: str,
321
+ cast_models: dict[str, str] | None,
322
  judge_policy: str,
323
  judge_model: str,
324
  judge_strictness: float,
 
329
  """Assemble + validate a per-run world from the Lab's form values.
330
 
331
  Returns the validated :class:`WorldConfig` (raising ``pydantic.ValidationError``
332
+ on an incoherent run). This is the bridge the app shell uses to build a Conductor
333
+ from a composed run via :meth:`Registry.from_world`.
334
+
335
+ The base scenario (selected by its display *title* or internal name) supplies the
336
+ cast roster and agent manifests. Model selection binds each mind to a *specific*
337
+ Modal-hosted model: ``cast_models`` maps ``{agent_name: catalogue_endpoint_key}`` for
338
+ the players and ``judge_model`` is the Judge's endpoint key (§04). Each becomes that
339
+ agent's ``model_endpoint`` (ADR-0022) non-destructively via ``model_copy``, so the
340
+ shared registry is untouched. Only keys that exist in ``modal_catalogue`` are honoured
341
+ (the picker offers nothing else; we re-check so a stale key can't reach the run); an
342
+ agent with no/blank/unknown selection keeps its manifest tier. The premise overrides
343
+ the scenario goal, ``seed`` becomes its ``default_seed``, and the budget knobs feed the
344
+ governor.
345
+
346
+ The judge knobs (``judge_policy`` / ``judge_strictness``) and the ``tools`` grant are
347
+ accepted and shape-checked, but the deeper synthesis (policy preset → judge behaviour,
348
+ tool grants onto worker manifests) is deferred and TODO'd below every dict this builds
349
+ still passes through ``validate_scenario`` / ``validate_world`` before return, so the
350
+ contract ("emit, validate, run") holds. See ADR-0011 / ADR-0022.
351
+
352
+ TODO: map ``judge_policy`` / ``judge_strictness`` to concrete judge behaviour and wire
353
+ the selected ``tools`` onto each worker's manifest ``tools`` grant once those contracts
354
+ land in the live path.
355
  """
356
  registry = default_registry()
357
  base = _scenario_by_title(scenario)
 
361
  if base is None:
362
  raise ValueError(f"unknown scenario {scenario!r} (have: {sorted(registry.scenarios)})")
363
 
364
+ # Only catalogue-hosted models may be cast: the picker offers nothing else, and we
365
+ # re-check here so an out-of-band or stale key can never reach the run.
366
+ valid_keys = {e["key"] for e in modal_catalogue.entries()}
367
+ selections = dict(cast_models or {})
368
+ judge_key = (judge_model or "").strip()
 
 
 
 
 
 
 
 
369
 
370
+ # Build the per-run cast: every manifest the scenario references, with its chosen
371
+ # Modal model pinned via model_endpoint. Non-destructive: model_copy, never mutate
372
+ # the cached registry instance.
373
  agents = []
374
  cast_names: list[str] = []
375
  for agent_name in base.cast:
376
  manifest = registry.agents.get(agent_name)
377
  if manifest is None:
378
  raise ValueError(f"scenario {base.name!r} references undefined agent {agent_name!r}")
379
+ # The Judge's model comes from §04; every other mind from the cast picker.
380
+ chosen = judge_key if manifest.role == "judge" else selections.get(agent_name)
381
+ patch: dict = {}
382
+ if chosen and chosen in valid_keys:
383
+ patch["model_endpoint"] = chosen
384
+ manifest = manifest.model_copy(update=patch)
385
  agents.append(manifest.model_dump(mode="python"))
386
  cast_names.append(manifest.name)
387
 
tests/test_fishbowl_lab.py CHANGED
@@ -3,7 +3,8 @@
3
  Cover both surfaces: ``build_lab`` returns the expected handles inside a Blocks, and
4
  ``collect_world_config`` assembles a real scenario's data into a validated WorldConfig
5
  (round-tripping through ``validate_world`` / ``validate_scenario``) without mutating the
6
- shared registry.
 
7
  """
8
 
9
  from __future__ import annotations
@@ -13,6 +14,7 @@ import pytest
13
 
14
  from src.core.config import WorldConfig
15
  from src.core.registry import default_registry
 
16
  from src.ui.fishbowl import lab
17
 
18
  EXPECTED_HANDLE_KEYS = {
@@ -21,7 +23,7 @@ EXPECTED_HANDLE_KEYS = {
21
  "seed",
22
  "world",
23
  "narrator",
24
- "cast",
25
  "judge_policy",
26
  "judge_model",
27
  "judge_strictness",
@@ -34,6 +36,9 @@ EXPECTED_HANDLE_KEYS = {
34
  "surprise_btn",
35
  }
36
 
 
 
 
37
 
38
  def test_build_lab_returns_expected_handles():
39
  with gr.Blocks():
@@ -45,44 +50,55 @@ def test_build_lab_returns_expected_handles():
45
  assert isinstance(handles["seed"], gr.Dropdown)
46
  assert handles["seed"].allow_custom_value is True
47
  assert isinstance(handles["narrator"], gr.Dropdown)
48
- assert isinstance(handles["cast"], gr.Dataframe)
49
- assert handles["cast"].interactive is True
50
  assert isinstance(handles["tools"], gr.CheckboxGroup)
51
  assert isinstance(handles["judge_strictness"], gr.Slider)
52
  assert isinstance(handles["summon_btn"], gr.Button)
53
  assert isinstance(handles["surprise_btn"], gr.Button)
54
 
55
 
56
- def test_build_lab_radio_lists_real_scenarios():
57
- registry = default_registry()
58
- real_titles = {s.title or s.name for s in registry.scenarios.values()}
59
  with gr.Blocks():
60
  handles = lab.build_lab()
61
- radio_choices = {c[0] for c in handles["scenario"].choices}
62
- assert radio_choices == real_titles
 
 
63
 
64
 
65
- def test_build_lab_cast_seeded_from_scenario():
66
- with gr.Blocks():
67
- handles = lab.build_lab()
68
- rows = handles["cast"].value["data"]
69
- assert rows, "cast should be seeded with at least one row"
70
- profiles = {row[2] for row in rows}
71
- assert profiles <= set(lab.MODEL_PROFILES)
 
 
 
 
 
 
 
 
 
72
 
73
 
74
- def test_collect_world_config_validates_real_scenario():
75
  registry = default_registry()
76
  scenario = registry.scenarios["thousand-token-wood"]
77
- cast_rows = lab._cast_rows_for(scenario)
 
 
78
 
79
  world = lab.collect_world_config(
80
  scenario=scenario.title,
81
  premise="A new whimsical premise for the wood.",
82
  seed=scenario.default_seed,
83
- cast_rows=cast_rows,
84
  judge_policy="Majority Vote",
85
- judge_model="strong",
86
  judge_strictness=60,
87
  tools=["dice.roll", "vote.tally"],
88
  tokens=120_000,
@@ -90,46 +106,42 @@ def test_collect_world_config_validates_real_scenario():
90
  )
91
 
92
  assert isinstance(world, WorldConfig)
93
- assert len(world.scenarios) == 1
 
 
 
 
 
 
94
  out = world.scenarios[0]
95
  assert out.name == scenario.name
96
  assert out.goal == "A new whimsical premise for the wood."
97
  assert out.cast == list(scenario.cast)
98
- # cross-reference check: every cast name resolves to a defined agent
99
- assert {a.name for a in world.agents} >= set(out.cast)
100
  assert out.governor is not None
101
  assert out.governor.max_turns == 25
102
  assert out.governor.max_total_tokens == 120_000
103
 
104
 
105
- def test_collect_world_config_applies_cast_edits_nondestructively():
106
  registry = default_registry()
107
  scenario = registry.scenarios["thousand-token-wood"]
108
- first_agent = scenario.cast[0]
109
- original_profile = registry.agents[first_agent].model_profile
110
-
111
- rows = lab._cast_rows_for(scenario)
112
- # flip the first agent's profile in the edited rows
113
- new_profile = "strong" if original_profile != "strong" else "tiny"
114
- rows[0][2] = new_profile
115
 
116
  world = lab.collect_world_config(
117
  scenario=scenario.name, # also accept name, not just title
118
  premise="",
119
  seed="",
120
- cast_rows=rows,
121
  judge_policy="Judge's Whim",
122
- judge_model="balanced",
123
  judge_strictness=10,
124
  tools=[],
125
  tokens=None,
126
  max_rounds=None,
127
  )
128
 
129
- edited = next(a for a in world.agents if a.name == first_agent)
130
- assert edited.model_profile == new_profile
131
- # registry untouched
132
- assert registry.agents[first_agent].model_profile == original_profile
133
  # blank premise/seed fall back to the scenario's own
134
  assert world.scenarios[0].goal == scenario.goal
135
  assert world.scenarios[0].default_seed == scenario.default_seed
@@ -141,9 +153,9 @@ def test_collect_world_config_unknown_scenario_raises():
141
  scenario="not-a-real-world",
142
  premise="",
143
  seed="",
144
- cast_rows=[],
145
  judge_policy="Majority Vote",
146
- judge_model="fast",
147
  judge_strictness=50,
148
  tools=[],
149
  tokens=None,
 
3
  Cover both surfaces: ``build_lab`` returns the expected handles inside a Blocks, and
4
  ``collect_world_config`` assembles a real scenario's data into a validated WorldConfig
5
  (round-tripping through ``validate_world`` / ``validate_scenario``) without mutating the
6
+ shared registry. Model selection is constrained to the Modal catalogue and pins each
7
+ agent's ``model_endpoint`` (ADR-0022).
8
  """
9
 
10
  from __future__ import annotations
 
14
 
15
  from src.core.config import WorldConfig
16
  from src.core.registry import default_registry
17
+ from src.models import modal_catalogue
18
  from src.ui.fishbowl import lab
19
 
20
  EXPECTED_HANDLE_KEYS = {
 
23
  "seed",
24
  "world",
25
  "narrator",
26
+ "cast_models",
27
  "judge_policy",
28
  "judge_model",
29
  "judge_strictness",
 
36
  "surprise_btn",
37
  }
38
 
39
+ # A couple of real catalogue keys to cast (the catalogue loads offline — a plain file).
40
+ _CATALOGUE_KEYS = [e["key"] for e in modal_catalogue.entries()]
41
+
42
 
43
  def test_build_lab_returns_expected_handles():
44
  with gr.Blocks():
 
50
  assert isinstance(handles["seed"], gr.Dropdown)
51
  assert handles["seed"].allow_custom_value is True
52
  assert isinstance(handles["narrator"], gr.Dropdown)
53
+ # The cast picker is a gr.render writing into this state (one dropdown per player).
54
+ assert isinstance(handles["cast_models"], gr.State)
55
  assert isinstance(handles["tools"], gr.CheckboxGroup)
56
  assert isinstance(handles["judge_strictness"], gr.Slider)
57
  assert isinstance(handles["summon_btn"], gr.Button)
58
  assert isinstance(handles["surprise_btn"], gr.Button)
59
 
60
 
61
+ def test_judge_model_dropdown_offers_only_catalogue_models():
 
 
62
  with gr.Blocks():
63
  handles = lab.build_lab()
64
+ assert isinstance(handles["judge_model"], gr.Dropdown)
65
+ values = {c[1] for c in handles["judge_model"].choices}
66
+ assert values <= set(_CATALOGUE_KEYS)
67
+ assert values, "judge model dropdown should list the catalogue"
68
 
69
 
70
+ def test_model_choices_are_all_catalogue_keys():
71
+ choices = lab.model_choices()
72
+ # Every selectable value is a real catalogue endpoint key — nothing else is offerable.
73
+ assert {key for _label, key in choices} == set(_CATALOGUE_KEYS)
74
+ # Labels are human-readable and name the served model.
75
+ assert all(" · " in label for label, _ in choices)
76
+
77
+
78
+ def test_cast_defaults_cover_non_judge_cast_with_catalogue_keys():
79
+ registry = default_registry()
80
+ scenario = registry.scenarios["thousand-token-wood"]
81
+ defaults = lab._cast_defaults(scenario)
82
+ judge_names = {n for n in scenario.cast if (registry.agents.get(n) and registry.agents[n].role == "judge")}
83
+ non_judge = [n for n in scenario.cast if n not in judge_names]
84
+ assert set(defaults) == set(non_judge) # judge excluded (set under §04)
85
+ assert all(v in _CATALOGUE_KEYS for v in defaults.values())
86
 
87
 
88
+ def test_collect_world_config_pins_selected_models_as_endpoints():
89
  registry = default_registry()
90
  scenario = registry.scenarios["thousand-token-wood"]
91
+ worker = next(n for n in scenario.cast if registry.agents[n].role != "judge")
92
+ judge = next(n for n in scenario.cast if registry.agents[n].role == "judge")
93
+ worker_key, judge_key = _CATALOGUE_KEYS[0], _CATALOGUE_KEYS[-1]
94
 
95
  world = lab.collect_world_config(
96
  scenario=scenario.title,
97
  premise="A new whimsical premise for the wood.",
98
  seed=scenario.default_seed,
99
+ cast_models={worker: worker_key},
100
  judge_policy="Majority Vote",
101
+ judge_model=judge_key,
102
  judge_strictness=60,
103
  tools=["dice.roll", "vote.tally"],
104
  tokens=120_000,
 
106
  )
107
 
108
  assert isinstance(world, WorldConfig)
109
+ by_name = {a.name: a for a in world.agents}
110
+ assert by_name[worker].model_endpoint == worker_key
111
+ assert by_name[judge].model_endpoint == judge_key # §04 binds the judge
112
+ # registry untouched (non-destructive model_copy)
113
+ assert registry.agents[worker].model_endpoint is None
114
+ assert registry.agents[judge].model_endpoint is None
115
+
116
  out = world.scenarios[0]
117
  assert out.name == scenario.name
118
  assert out.goal == "A new whimsical premise for the wood."
119
  assert out.cast == list(scenario.cast)
 
 
120
  assert out.governor is not None
121
  assert out.governor.max_turns == 25
122
  assert out.governor.max_total_tokens == 120_000
123
 
124
 
125
+ def test_collect_world_config_ignores_unknown_or_blank_model_keys():
126
  registry = default_registry()
127
  scenario = registry.scenarios["thousand-token-wood"]
128
+ worker = next(n for n in scenario.cast if registry.agents[n].role != "judge")
 
 
 
 
 
 
129
 
130
  world = lab.collect_world_config(
131
  scenario=scenario.name, # also accept name, not just title
132
  premise="",
133
  seed="",
134
+ cast_models={worker: "not-a-real-endpoint"}, # bogus key → ignored
135
  judge_policy="Judge's Whim",
136
+ judge_model="", # blank → judge keeps its tier
137
  judge_strictness=10,
138
  tools=[],
139
  tokens=None,
140
  max_rounds=None,
141
  )
142
 
143
+ by_name = {a.name: a for a in world.agents}
144
+ assert by_name[worker].model_endpoint is None # stale/unknown key dropped
 
 
145
  # blank premise/seed fall back to the scenario's own
146
  assert world.scenarios[0].goal == scenario.goal
147
  assert world.scenarios[0].default_seed == scenario.default_seed
 
153
  scenario="not-a-real-world",
154
  premise="",
155
  seed="",
156
+ cast_models={},
157
  judge_policy="Majority Vote",
158
+ judge_model="",
159
  judge_strictness=50,
160
  tools=[],
161
  tokens=None,
tests/test_manifest_agent.py CHANGED
@@ -71,6 +71,25 @@ class TestManifestAgentEmits:
71
  agent.act("r", 1, StageProjection(), ())
72
  assert router.for_profile("tiny").variant == "stub:tiny"
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
 
75
  class TestManifestAgentSalience:
76
  def test_salience_path_runs(self):
 
71
  agent.act("r", 1, StageProjection(), ())
72
  assert router.for_profile("tiny").variant == "stub:tiny"
73
 
74
+ def test_model_endpoint_overrides_profile(self):
75
+ # An explicit catalogue endpoint pins a specific model: _route_key prefers it
76
+ # over the tier, and the agent routes there (ADR-0022).
77
+ class _Pinned(ManifestAgent):
78
+ manifest = AgentManifest(
79
+ name="scene-whisperer",
80
+ role="worker",
81
+ persona="You speak on a specific model.",
82
+ may_emit=["world.observed"],
83
+ model_profile="tiny",
84
+ model_endpoint="minicpm-4-1-8b",
85
+ )
86
+
87
+ agent = _Pinned(router := _router())
88
+ assert agent._route_key == "minicpm-4-1-8b"
89
+ agent.act("r", 1, StageProjection(), ())
90
+ # the routed (cached) provider is the endpoint's stub, not the tiny tier's
91
+ assert router.for_profile("minicpm-4-1-8b").variant == "stub:minicpm-4-1-8b"
92
+
93
 
94
  class TestManifestAgentSalience:
95
  def test_salience_path_runs(self):
tests/test_registry.py CHANGED
@@ -56,6 +56,43 @@ class TestDefaultRegistry:
56
  reg.build_scenario("no-such-scenario")
57
 
58
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  class TestHandlerBinding:
60
  def test_handler_class_used_when_named(self):
61
  @register_handler("test-handler")
 
56
  reg.build_scenario("no-such-scenario")
57
 
58
 
59
+ class TestFromWorld:
60
+ """A composed WorldConfig (e.g. from the Lab) builds a registry on the same path."""
61
+
62
+ def _world(self):
63
+ from src.core.config import validate_world
64
+
65
+ reg = default_registry()
66
+ base = reg.scenarios["thousand-token-wood"]
67
+ # Pin one agent to a specific catalogue model, the rest by tier.
68
+ agents = []
69
+ for name in base.cast:
70
+ m = reg.agents[name]
71
+ if name == "pocket-actor":
72
+ m = m.model_copy(update={"model_endpoint": "minicpm-4-1-8b"})
73
+ agents.append(m.model_dump(mode="python"))
74
+ return validate_world(
75
+ {
76
+ "agents": agents,
77
+ "scenarios": [base.model_dump(mode="python")],
78
+ }
79
+ )
80
+
81
+ def test_builds_scenario_and_router_from_world(self):
82
+ reg = Registry.from_world(self._world())
83
+ assert set(reg.agents) >= {"scene-whisperer", "pocket-actor", "echo"}
84
+ sc = reg.build_scenario("thousand-token-wood")
85
+ assert len(sc.agents) == 4
86
+ pocket = next(a for a in sc.agents if a.name == "pocket-actor")
87
+ assert pocket.manifest.model_endpoint == "minicpm-4-1-8b"
88
+ assert pocket._route_key == "minicpm-4-1-8b"
89
+
90
+ def test_governor_threaded_from_world(self):
91
+ reg = Registry.from_world(self._world())
92
+ gov = reg.governor_for("thousand-token-wood")
93
+ assert gov.max_turns == 60 # carried from the scenario's governor
94
+
95
+
96
  class TestHandlerBinding:
97
  def test_handler_class_used_when_named(self):
98
  @register_handler("test-handler")
tests/test_router.py CHANGED
@@ -71,6 +71,45 @@ class TestModelRouterOnline:
71
  assert provider.max_tokens == 320 # _PROFILE_DECODING["balanced"]
72
 
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  class TestFromEnv:
75
  def test_offline_without_binding(self, monkeypatch):
76
  # No Modal binding (and no stray cloud key) → deterministic offline stub.
 
71
  assert provider.max_tokens == 320 # _PROFILE_DECODING["balanced"]
72
 
73
 
74
+ class TestModelRouterCatalogueEndpoint:
75
+ """A non-tier router key names a specific catalogue model (manifest.model_endpoint)."""
76
+
77
+ def test_offline_endpoint_key_serves_distinct_stub(self):
78
+ # A concrete catalogue key routes like any profile offline: the deterministic
79
+ # stub, with the key folded into its variant so the choice still varies output.
80
+ router = ModelRouter(offline=True)
81
+ provider = router.for_profile("minicpm-4-1-8b")
82
+ assert isinstance(provider, DeterministicTinyModel)
83
+ assert "minicpm-4-1-8b" in provider.variant
84
+ assert provider.variant != router.for_profile("gemma-4-12b").variant
85
+
86
+ def test_online_endpoint_key_resolves_to_catalogue_binding(self, monkeypatch):
87
+ monkeypatch.setenv("MODAL_WORKSPACE", "demo-ws")
88
+ monkeypatch.setenv("MODAL_LLM_KEY", "EMPTY")
89
+ monkeypatch.delenv("MODEL_BALANCED", raising=False)
90
+ router = ModelRouter(offline=False)
91
+ provider = router.for_profile("gemma-4-12b")
92
+ assert isinstance(provider, LiteLLMProvider)
93
+ assert provider.model == "openai/google/gemma-4-12B"
94
+ assert "gemma-4-12b" in provider.api_base
95
+ assert provider.max_tokens == 320 # balanced tier decoding
96
+
97
+ def test_unbound_specialist_uses_balanced_decoding(self, monkeypatch):
98
+ # nemotron-cascade-14b-thinking has profile=None → balanced decoding defaults.
99
+ monkeypatch.setenv("MODAL_WORKSPACE", "demo-ws")
100
+ router = ModelRouter(offline=False)
101
+ provider = router.for_profile("nemotron-cascade-14b-thinking")
102
+ assert isinstance(provider, LiteLLMProvider)
103
+ assert provider.max_tokens == 320
104
+
105
+ def test_unknown_key_degrades_to_fast_tier(self, monkeypatch):
106
+ monkeypatch.setenv("MODEL_FAST", "fallback-model")
107
+ router = ModelRouter(offline=False)
108
+ provider = router.for_profile("not-a-real-endpoint")
109
+ assert provider.model == "fallback-model"
110
+ assert provider.max_tokens == 220 # fast tier decoding
111
+
112
+
113
  class TestFromEnv:
114
  def test_offline_without_binding(self, monkeypatch):
115
  # No Modal binding (and no stray cloud key) → deterministic offline stub.