docs/offline-training.md

Offline training workflow

This workflow avoids live SSH-dependent demos while still producing real supervised training data for a remote GPU run.

What you will produce

1. A recorded trajectory directory under runs/trajectories/
2. A replayable live demo using the same actions
3. An SFT-ready dataset in JSONL format under runs/datasets/

Scripts

• scripts/record_trajectory.py
• scripts/replay_trajectory.py
• scripts/export_sft_dataset.py
• tasks/examples/aki-demo-actions.json

Step-by-step

1. Start the app and env locally

npm run dev

This gives:

• EHR UI on http://127.0.0.1:3000
• Env server on http://127.0.0.1:8000

2. Record a deterministic trajectory

python scripts/record_trajectory.py tasks/examples/aki-demo-actions.json --stop-on-done

This creates a directory like:

• runs/trajectories/20260307-120000-aki-chart-review-demo/

with:

• manifest.json
• steps.jsonl
• screenshots/

3. Replay the trajectory for a reliable demo

python scripts/replay_trajectory.py runs/trajectories/<YOUR_TRAJECTORY_DIR>

4. Export SFT training data

python scripts/export_sft_dataset.py runs/trajectories --output runs/datasets/ehrgym_sft.jsonl

The output is JSONL with one next-action training example per step.

5. Copy the dataset to the remote GPU box

Example with scp:

scp runs/datasets/ehrgym_sft.jsonl <user>@<gpu-host>:/workspace/data/

Or upload the whole directory:

scp -r runs/datasets <user>@<gpu-host>:/workspace/

Record more than one demo

After you create more action bundles, rerun the recorder and then rerun the exporter over the whole runs/trajectories/ directory.

Notes

• The exporter uses the observation from step t as the input and the action at step t+1 as the supervised target.
• Screenshot paths are stored as absolute paths by default.
• For text-only experiments, export without image paths:

python scripts/export_sft_dataset.py runs/trajectories --image-mode none --output runs/datasets/ehrgym_sft_textonly.jsonl