File size: 2,559 Bytes
a2ffd07 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | # training_method — bathroom-toilet v2 training (ported from multilayer-sae)
Self-contained copy of the code needed to run `run_bathroom_toilet_v2.sh`
(originally `/data/caotue/multilayer-sae/run_bathroom_toilet_v2.sh`), placed in
the hallucination repo. Files that already exist **identically** in hallucination
are NOT copied — they're imported from the existing `experiment.*` / `sae.*`
packages. Only files that are new or differ from hallucination were added here.
## Added here (copied — differ from / missing in hallucination)
- `sequence_probe.py` — DIFF (adds per-token `token_logits` / `forward_token_logits`)
- `finetune_adv.py` — DIFF (adds `scene_only` probe-label mode)
- `train_probe_gen.py` — DIFF (stage-1 entry: probe pretrain)
- `finetune_adv_gen.py` — identical to hallucination, but copied because it imports
the DIFF `finetune_adv` (intra-package import rewired)
- `finetune_adv_gen_resume.py` — stage-2 entry; same reason (imports DIFF deps)
- `adv_config_bathroom_toilet_seqprobe_adv16.json` — missing in hallucination
- `run_bathroom_toilet_v2.sh`, `run_finetune_adv_gen_refined.sh` — drivers (module
paths repointed to `training_method.*`, REPO → hallucination)
Cross-imports between the 5 copied `.py` were rewired `experiment.training.X`
→ `training_method.X`. All other imports are left as `experiment.*` / `sae.*`
and resolve to hallucination's existing (byte-identical) modules.
## Imported from hallucination (NOT copied — identical)
`sae.Training_Utils`, `experiment.config.{train_config,relation_config}`,
`experiment.data.{datasets,hf_loader}`, `experiment.evaluation.metrics`,
`experiment.training.{gen_features,preference}`.
## Run
```bash
source /data/caotue/multilayer-sae/.venv/bin/activate # torch/transformers env
cd /data/caotue/hallucination
bash training_method/run_bathroom_toilet_v2.sh
```
`PYTHONPATH` is set to the hallucination root by the driver, so both
`training_method.*` and `experiment.*`/`sae.*` import correctly.
## Caveats
- Outputs land under `hallucination/probe_outputs_seq` and
`hallucination/adv_gen_outputs` (driver `REPO` = hallucination).
- The config's `output_dir` still points at `/data/caotue/multilayer-sae/...`
but is overridden by the driver's `--output_dir`. `general_image_dir`
(`/data/caotue/CC3M-Dataset/cc3m_images/train`) and `dataset_id` are shared paths.
- GPUs auto-selected at runtime (>28 GiB free). Stage-1 DDP probe pretrain →
stage-2 adversarial LoRA via the runner.
|