File size: 1,209 Bytes
e2bfccc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # Experiment Workflow
Before running a remote benchmark:
1. Create `experiments/runs/<run_id>/`.
2. Save the exact RepoBridge config as `repobridge.config.json`.
3. Add a `manifest.yaml` with the goal, commits, dataset, tokenizer, and expected comparison.
4. Run RepoBridge with that config.
5. Copy only the final compact CSV/JSON metrics into the run folder.
6. Write `summary.md`.
7. Update `experiments/index.csv`.
8. Update the top-level `README.md` only when the current best model or conclusion changes.
RepoBridge should remain a tool repo. New experiment configs should live here, not in the RepoBridge root.
For cross-repo orientation, see `docs/DIRECTORY_NAVIGATION.md`.
Preferred run command:
```powershell
python -m repobridge.cli sync --config C:\path\to\Taotern_LLM_Experiments\experiments\runs\<run_id>\repobridge.config.json --use-stored-password
python -m repobridge.cli run --config C:\path\to\Taotern_LLM_Experiments\experiments\runs\<run_id>\repobridge.config.json --use-stored-password
```
For downloads, prefer targeted downloads of the specific run folder. Avoid recursively downloading the whole remote output base if it contains many historical runs.
|