MODAL_ARCHITECTURE.md

# Modal Architecture

Glass-Box Agent is structured so the UI is a thin client and Modal owns the durable/compute-heavy pieces.

## Flow

```mermaid
flowchart LR
  UI["Gradio UI"] --> Model["Modal model service"]
  UI --> Trace["Modal trace store"]
  Trace --> Dataset["RL feedback dataset"]
  Dataset --> Tune["Modal RL tuning job"]
  Tune --> Checkpoint["Modal checkpoint"]
  Checkpoint --> Model
```

## 1. Model served by Modal

`modal_service.py` exposes `reason_next`, a Modal web function that accepts:

```json
{
  "task": "User task",
  "step": 3,
  "memory": ["recent trace observations"],
  "model_id": "checkpoint-abc123 or base model id"
}
```

The Gradio app uses this when **Reasoner** is set to `Modal served model`.

Configuration options:

```bash
export GLASS_BOX_MODAL_MODEL_URL="https://...--reason-next.modal.run"
export GLASS_BOX_MODAL_TRACE_URL="https://...--store-trace.modal.run"
export GLASS_BOX_MODAL_TUNE_URL="https://...--start-rl-tune.modal.run"
```

If URLs are not configured, the app can call deployed Modal functions by name when the Modal SDK is authenticated:

```bash
export GLASS_BOX_MODAL_APP="glass-box-agent-modal"
export GLASS_BOX_MODAL_MODEL_FUNCTION="reason_next"
export GLASS_BOX_MODAL_TRACE_FUNCTION="store_trace"
export GLASS_BOX_MODAL_TUNE_FUNCTION="start_rl_tune"
export GLASS_BOX_MODAL_TRACE_ENABLED=1
```

## 2. Traces stored by Modal

Every server-side trace mutation calls `store_trace` when Modal trace storage is configured:

- run start and streamed trace updates
- retry branch
- replay downstream
- mark/clear labels
- fork selected node
- export trace
- export RL data
- preference feedback

Client-side visual edits also post to `GLASS_BOX_MODAL_TRACE_URL` when that endpoint is configured.

Traces are written to the Modal Volume:

```text
/data/traces/<trace_id>/<timestamp>-<artifact>-<event>.json
/data/traces/<trace_id>/latest.json
```

Modal Volumes require explicit `volume.commit()` to persist writes; `modal_service.py` commits after every write.

## 3. RL feedback and tuning

The **Training signal** panel records branch preferences:

```json
{
  "dpo": {
    "prompt": "Task and upstream trace context",
    "chosen": "Better branch continuation",
    "rejected": "Weaker branch continuation"
  }
}
```

`Export RL data` writes a local JSON artifact and stores the trace snapshot through Modal.

`RL tune on Modal` calls `start_rl_tune`, which stores:

```text
/data/rl-feedback/<trace_id>/<checkpoint_id>/dataset.json
/data/checkpoints/<checkpoint_id>/manifest.json
```

The current implementation creates a checkpoint manifest scaffold. The intended next step is to replace the body of `start_rl_tune` with TRL/LoRA training and write adapter weights beside `manifest.json`.

## 4. Using checkpoints

After a tuning run returns a checkpoint such as:

```text
checkpoint-a1b2c3d4e5
```

put that value in the **Small model** field and keep **Reasoner** set to `Modal served model`. `reason_next` will look for:

```text
/data/checkpoints/checkpoint-a1b2c3d4e5/manifest.json
```

and can load adapter weights from that directory once the tuning scaffold is replaced with real training.

## Deploy

```bash
modal deploy modal_service.py
```

For local development with live URLs:

```bash
modal serve modal_service.py
```

Modal web endpoints use `@modal.fastapi_endpoint(method="POST")`, and durable trace/checkpoint files use `modal.Volume`.