code/Taotern_LLM_Experiments/docs/WORKFLOW.md

Experiment Workflow

Before running a remote benchmark:

1. Create experiments/runs/<run_id>/.
2. Save the exact RepoBridge config as repobridge.config.json.
3. Add a manifest.yaml with the goal, commits, dataset, tokenizer, and expected comparison.
4. Run RepoBridge with that config.
5. Copy only the final compact CSV/JSON metrics into the run folder.
6. Write summary.md.
7. Update experiments/index.csv.
8. Update the top-level README.md only when the current best model or conclusion changes.

RepoBridge should remain a tool repo. New experiment configs should live here, not in the RepoBridge root.

For cross-repo orientation, see docs/DIRECTORY_NAVIGATION.md.

Preferred run command:

python -m repobridge.cli sync --config C:\path\to\Taotern_LLM_Experiments\experiments\runs\<run_id>\repobridge.config.json --use-stored-password
python -m repobridge.cli run --config C:\path\to\Taotern_LLM_Experiments\experiments\runs\<run_id>\repobridge.config.json --use-stored-password

For downloads, prefer targeted downloads of the specific run folder. Avoid recursively downloading the whole remote output base if it contains many historical runs.