| # training_method — bathroom-toilet v2 training (ported from multilayer-sae) |
| |
| Self-contained copy of the code needed to run `run_bathroom_toilet_v2.sh` |
| (originally `/data/caotue/multilayer-sae/run_bathroom_toilet_v2.sh`), placed in |
| the hallucination repo. Files that already exist **identically** in hallucination |
| are NOT copied — they're imported from the existing `experiment.*` / `sae.*` |
| packages. Only files that are new or differ from hallucination were added here. |
|
|
| ## Added here (copied — differ from / missing in hallucination) |
| - `sequence_probe.py` — DIFF (adds per-token `token_logits` / `forward_token_logits`) |
| - `finetune_adv.py` — DIFF (adds `scene_only` probe-label mode) |
| - `train_probe_gen.py` — DIFF (stage-1 entry: probe pretrain) |
| - `finetune_adv_gen.py` — identical to hallucination, but copied because it imports |
| the DIFF `finetune_adv` (intra-package import rewired) |
| - `finetune_adv_gen_resume.py` — stage-2 entry; same reason (imports DIFF deps) |
| - `adv_config_bathroom_toilet_seqprobe_adv16.json` — missing in hallucination |
| - `run_bathroom_toilet_v2.sh`, `run_finetune_adv_gen_refined.sh` — drivers (module |
| paths repointed to `training_method.*`, REPO → hallucination) |
| |
| Cross-imports between the 5 copied `.py` were rewired `experiment.training.X` |
| → `training_method.X`. All other imports are left as `experiment.*` / `sae.*` |
| and resolve to hallucination's existing (byte-identical) modules. |
|
|
| ## Imported from hallucination (NOT copied — identical) |
| `sae.Training_Utils`, `experiment.config.{train_config,relation_config}`, |
| `experiment.data.{datasets,hf_loader}`, `experiment.evaluation.metrics`, |
| `experiment.training.{gen_features,preference}`. |
|
|
| ## Run |
| ```bash |
| source /data/caotue/multilayer-sae/.venv/bin/activate # torch/transformers env |
| cd /data/caotue/hallucination |
| bash training_method/run_bathroom_toilet_v2.sh |
| ``` |
| `PYTHONPATH` is set to the hallucination root by the driver, so both |
| `training_method.*` and `experiment.*`/`sae.*` import correctly. |
|
|
| ## Caveats |
| - Outputs land under `hallucination/probe_outputs_seq` and |
| `hallucination/adv_gen_outputs` (driver `REPO` = hallucination). |
| - The config's `output_dir` still points at `/data/caotue/multilayer-sae/...` |
| but is overridden by the driver's `--output_dir`. `general_image_dir` |
| (`/data/caotue/CC3M-Dataset/cc3m_images/train`) and `dataset_id` are shared paths. |
| - GPUs auto-selected at runtime (>28 GiB free). Stage-1 DDP probe pretrain → |
| stage-2 adversarial LoRA via the runner. |
|
|
