Upload folder using huggingface_hub

README.md

@@ -0,0 +1,96 @@
+---
+license: mit
+base_model: gpt2-medium
+tags:
+  - controllable-text-generation
+  - concept-bottleneck
+  - compositional-generalization
+  - lora
+library_name: peft
+---
+
+# gpt2-medium-cbctg-prior
+
+Concept-bottleneck controllable-generation modules for **GPT-2 Medium (355M)**, from
+the paper "The Illusion of Control: Why Bare Classifier Inversion Silently Fails in Concept-Bottleneck Text Generation" (EMNLP 2026).
+
+Matched-backbone comparison against the CompMCTG baselines (Section 7.1), all of which use GPT-2-Medium. Reported over three training seeds.
+
+This repository contains **only our own trained parameters**: the per-axis
+concept encoder MLPs, the per-axis classifier heads, the concept injector,
+the LoRA adapter, and the post-hoc label prior. No base-model weights are
+included; download **[gpt2-medium](https://huggingface.co/gpt2-medium)** separately.
+
+## Contents
+
+Each split directory holds one trained checkpoint.
+
+| Path | Description |
+|---|---|
+| `hold-out/` | Fyelp Hold-Out idx=-0 split (39 seen / 1 unseen configuration) |
+| `acd/` | Fyelp ACD split (half of all configurations held out) |
+
+Within each split directory:
+
+| File | Description |
+|---|---|
+| `config.json` | Architecture and training configuration |
+| `full_model_best.pt` | Concept encoder MLPs, classifier heads, injector, internal prior |
+| `best_lora_adapter.pt` | LoRA adapter weights (rank 8, alpha 16); optimiser state stripped |
+| `label_prior.pt` | Post-hoc label-conditioned prior $g_\gamma$ (the paper's recommended z-source) |
+| `retrieval_prior.pt` | Nearest-seen-configuration retrieval baseline (Appendix L), where available |
+| `label_prior_flow.pt` | Conditional normalising-flow baseline (Appendix K), where available |
+| `label_prior_factorized.pt` | Factorised per-axis prior (Appendix N), where available |
+| `clsinv_reg_*.pt` | Manifold-regulariser statistics for the inversion sweeps (Appendix I), where available |
+
+## Architecture
+
+- **Concept code**: 4 axes (cuisine, gender, sentiment, tense), 32 dimensions per axis
+- **Injector**: AdaLN-zero, injected at every block
+- **Generator**: GPT-2 Medium (355M) with LoRA (rank 8, alpha 16) on the attention projections
+- **Prior** $g_\gamma$: one hidden layer, 128 GELU units, fitted post-hoc in under 30 seconds
+
+## Usage
+
+Clone the code repository and point the generation script at a downloaded
+checkpoint directory:
+
+```bash
+git clone https://github.com/BiancaBing/cbctg-illusion-of-control
+cd cbctg-illusion-of-control
+
+huggingface-cli download kabing/gpt2-medium-cbctg-prior --local-dir ckpt/gpt2-medium-cbctg-prior
+
+python evaluation/generate_v3.py \
+    --checkpoint ckpt/gpt2-medium-cbctg-prior/hold-out/full_model_best.pt \
+    --data_dir   data/fyelp_hold-out_idx0_4ax \
+    --output_dir out/ \
+    --model_name gpt2-medium-cbctg-prior_prior \
+    --z_source prior \
+    --prior_path ckpt/gpt2-medium-cbctg-prior/hold-out/label_prior.pt
+```
+
+Swap `--z_source prior` for `oracle` (reference-text encoding) or `mode_b`
+(classifier inversion) to reproduce the other two protocols compared in the
+paper. Full reproduction instructions are in the code repository.
+
+## Licence
+
+GPT-2 Medium is released under the MIT License.
+
+Our released parameters (encoder MLPs, classifier heads, injector, LoRA
+adapter, label prior) are made available for research use under the same
+terms as the corresponding base model.
+
+## Citation
+
+```bibtex
+@inproceedings{bing2026illusion,
+  title     = {The Illusion of Control: Why Bare Classifier Inversion
+               Silently Fails in Concept-Bottleneck Text Generation},
+  author    = {Bing, Qi and Shao, Xiaowei},
+  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in
+               Natural Language Processing (EMNLP)},
+  year      = {2026}
+}
+```

acd/best_lora_adapter.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb0661db0a1fcf6a095d6728c416117e8cbff974617a86017eff834a6ca86577
+size 1590414

acd/config.json

@@ -0,0 +1,59 @@
+{
+  "backbone": "gpt2-medium",
+  "dataset": "fyelp_acd_idx0_4ax_full",
+  "data_dir": "E:\\research\\cbm\\project\\data\\fyelp_acd_idx0_4ax_full",
+  "axis_names": [
+    "cuisine",
+    "gender",
+    "sentiment",
+    "tense"
+  ],
+  "num_axes": 4,
+  "num_classes_per_axis": 5,
+  "concept_dim": 32,
+  "epochs": 25,
+  "batch_size": 16,
+  "lr": 5e-05,
+  "lora_lr": 2e-05,
+  "lora_rank": 8,
+  "lora_alpha": 16,
+  "noise_sigma": 0.01,
+  "gen_warmup_epochs": 10,
+  "scheduler": "cosine",
+  "grad_clip": 1.0,
+  "per_axis_injection": false,
+  "injection_scale": 1.0,
+  "inject_every_n": 1,
+  "gate_init": 0.0,
+  "injector_type": "adaln",
+  "quantize": false,
+  "bf16": true,
+  "fp32_small_modules": true,
+  "max_seq_len": 128,
+  "seed": 42,
+  "device": "cuda:0",
+  "run_timestamp": "20260515_171809",
+  "run_name": "gpt2-medium__fyelp_acd_idx0_4ax_full__adaln-e1__4axis__e25_s42_20260515_171809",
+  "label_maps": {
+    "cuisine": {
+      "american": 0,
+      "asian": 1,
+      "bar": 2,
+      "dessert": 3,
+      "mexican": 4
+    },
+    "gender": {
+      "female": 0,
+      "male": 1
+    },
+    "sentiment": {
+      "negative": 0,
+      "positive": 1
+    },
+    "tense": {
+      "past": 0,
+      "present": 1
+    }
+  },
+  "lambda3_final": 0.5
+}

acd/full_model_best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a99899d3995ef5260720f16e5347a8718b9bdcc63432666373fc84065189d206
+size 229275784

acd/label_prior.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51a1717c5f7d5cb68076888405ae6fc4a8ab51330d13d7ac187165377d18d7ee
+size 75056

acd/label_prior_factorized.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e98bb08db1262a58248d24b3c5d21acc2ce27b1592604172a3b19db87c769c47
+size 32726

hold-out/best_lora_adapter.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18a02db8f1167e6df129710421eccf6a5d2e3507c36a9d629ed463ff5342f079
+size 1590414

hold-out/config.json

@@ -0,0 +1,59 @@
+{
+  "backbone": "gpt2-medium",
+  "dataset": "fyelp_hold-out_idx0_4ax_full",
+  "data_dir": "E:\\research\\cbm\\project\\data\\fyelp_hold-out_idx0_4ax_full",
+  "axis_names": [
+    "cuisine",
+    "gender",
+    "sentiment",
+    "tense"
+  ],
+  "num_axes": 4,
+  "num_classes_per_axis": 5,
+  "concept_dim": 32,
+  "epochs": 25,
+  "batch_size": 16,
+  "lr": 5e-05,
+  "lora_lr": 2e-05,
+  "lora_rank": 8,
+  "lora_alpha": 16,
+  "noise_sigma": 0.01,
+  "gen_warmup_epochs": 10,
+  "scheduler": "cosine",
+  "grad_clip": 1.0,
+  "per_axis_injection": false,
+  "injection_scale": 1.0,
+  "inject_every_n": 1,
+  "gate_init": 0.0,
+  "injector_type": "adaln",
+  "quantize": false,
+  "bf16": true,
+  "fp32_small_modules": true,
+  "max_seq_len": 128,
+  "seed": 42,
+  "device": "cuda:0",
+  "run_timestamp": "20260515_214922",
+  "run_name": "gpt2-medium__fyelp_hold-out_idx0_4ax_full__adaln-e1__4axis__e25_s42_20260515_214922",
+  "label_maps": {
+    "cuisine": {
+      "american": 0,
+      "asian": 1,
+      "bar": 2,
+      "dessert": 3,
+      "mexican": 4
+    },
+    "gender": {
+      "female": 0,
+      "male": 1
+    },
+    "sentiment": {
+      "negative": 0,
+      "positive": 1
+    },
+    "tense": {
+      "past": 0,
+      "present": 1
+    }
+  },
+  "lambda3_final": 0.5
+}

hold-out/full_model_best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5dc5c3a1bc774532f2395d3e5632b539034fac1688e6fdfac071223c241b7c2
+size 229275784

hold-out/label_prior.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54a1b8fa8c01474eee2785f960f8d471e2301ee9095430663165fe028a794e42
+size 75120