Hugging Face's logo

ToiTenBao
/
hallucination

Model card Files
xet
 Community
hallucination / training_method /README.md
ToiTenBao's picture
ToiTenBao
Upload hallucination folder
a2ffd07 verified
preview code
|
Raw
History Blame Contribute Delete
2.56 kB

training_method — bathroom-toilet v2 training (ported from multilayer-sae)

Self-contained copy of the code needed to run run_bathroom_toilet_v2.sh (originally /data/caotue/multilayer-sae/run_bathroom_toilet_v2.sh), placed in the hallucination repo. Files that already exist identically in hallucination are NOT copied — they're imported from the existing experiment.* / sae.* packages. Only files that are new or differ from hallucination were added here.

Added here (copied — differ from / missing in hallucination)

  • sequence_probe.py — DIFF (adds per-token token_logits / forward_token_logits)
  • finetune_adv.py — DIFF (adds scene_only probe-label mode)
  • train_probe_gen.py — DIFF (stage-1 entry: probe pretrain)
  • finetune_adv_gen.py — identical to hallucination, but copied because it imports the DIFF finetune_adv (intra-package import rewired)
  • finetune_adv_gen_resume.py — stage-2 entry; same reason (imports DIFF deps)
  • adv_config_bathroom_toilet_seqprobe_adv16.json — missing in hallucination
  • run_bathroom_toilet_v2.sh, run_finetune_adv_gen_refined.sh — drivers (module paths repointed to training_method.*, REPO → hallucination)

Cross-imports between the 5 copied .py were rewired experiment.training.X → training_method.X. All other imports are left as experiment.* / sae.* and resolve to hallucination's existing (byte-identical) modules.

Imported from hallucination (NOT copied — identical)

sae.Training_Utils, experiment.config.{train_config,relation_config}, experiment.data.{datasets,hf_loader}, experiment.evaluation.metrics, experiment.training.{gen_features,preference}.

Run 

source /data/caotue/multilayer-sae/.venv/bin/activate   # torch/transformers env
cd /data/caotue/hallucination
bash training_method/run_bathroom_toilet_v2.sh

PYTHONPATH is set to the hallucination root by the driver, so both training_method.* and experiment.*/sae.* import correctly.

Caveats

  • Outputs land under hallucination/probe_outputs_seq and hallucination/adv_gen_outputs (driver REPO = hallucination).
  • The config's output_dir still points at /data/caotue/multilayer-sae/... but is overridden by the driver's --output_dir. general_image_dir (/data/caotue/CC3M-Dataset/cc3m_images/train) and dataset_id are shared paths.
  • GPUs auto-selected at runtime (>28 GiB free). Stage-1 DDP probe pretrain → stage-2 adversarial LoRA via the runner.