|
Download README.md from AlignmentResearch/synth-deception-probes: direct link, hf CLI and curl.
- Browser
- Download file 7.53 kB
-
https://huggingface.co/AlignmentResearch/synth-deception-probes/resolve/main/README.md
- Command line
-
hf download hf://AlignmentResearch/synth-deception-probes/README.md
-
curl -L -o README.md https://huggingface.co/AlignmentResearch/synth-deception-probes/resolve/main/README.md
7.53 kB
| license: mit | |
| tags: | |
| - probes | |
| - activation-probes | |
| - interpretability | |
| - deception-detection | |
| datasets: | |
| - AlignmentResearch/synth-deception | |
| base_model: | |
| - Qwen/Qwen3.5-2B | |
| - Qwen/Qwen3.5-9B | |
| - Qwen/Qwen3.5-27B | |
| - Qwen/Qwen3.5-122B-A10B | |
| - Qwen/Qwen3.5-397B-A17B | |
| - nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | |
| - nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 | |
| - nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 | |
| - moonshotai/Kimi-K3 | |
| # SynthDeception probes | |
| Activation probes trained on [SynthDeception](https://huggingface.co/datasets/AlignmentResearch/synth-deception), in | |
| four architectures (linear, MLP, EFC (early-fusion covariance), and axial) for nine open-weight models. Each probe reads | |
| a model's residual-stream activations at six decoder layers and returns one score for the model's answer to a follow-up | |
| question. A higher score means the answer is more likely dishonest. The same architectures trained on [FIBS](https://huggingface.co/datasets/AlignmentResearch/fibs-v1) are in | |
| [probe-inference-weights](https://huggingface.co/AlignmentResearch/probe-inference-weights). | |
| Load them with the `probe-inference` package | |
| ([AlignmentResearch/fibs-probes](https://github.com/AlignmentResearch/fibs-probes)): | |
| ```python | |
| from probe_inference import load_probe_from_hub | |
| probe = load_probe_from_hub( | |
| "ck-belief/qwen3.5-9b/efc", | |
| repo_id="AlignmentResearch/synth-deception-probes", | |
| ) | |
| score = probe.score(acts, probe.read_mask(prompt_mask, completion_mask, followup_start_position)) | |
| ``` | |
| ## Layout | |
| `<training-set>/<model>/<arch>/`, with `<arch>` in `linear`, `mlp`, `efc`, and `axial`. Below the training set, the | |
| layout and the file formats are those of probe-inference-weights. Each directory is one trained probe with its | |
| `probe_metadata.json`. A linear or MLP probe is one small probe per layer (`layer_<L>/`), and an EFC or axial probe is | |
| one module that reads all its layers at once. | |
| A training set is named for the follow-up focuses it contains. A focus is the kind of fact that a follow-up question | |
| asks about: common knowledge (`ck`), the model's belief (`belief`), or its intent (`intent`). | |
| | Training set | Training rows | Validation rows | | |
| |---|---|---| | |
| | `ck`, `belief`, `intent` | 4,656 | 400 | | |
| | `ck-belief`, `ck-intent`, `belief-intent` | 9,312 | 800 | | |
| | `ck-belief-intent` | 13,968 | 1,200 | | |
| | Directory | Model (revision) | Layers | | |
| |---|---|---| | |
| | `qwen3.5-2b` | `Qwen/Qwen3.5-2B` (`15852e8c`) | 7, 10, 13, 16, 19, 22 | | |
| | `qwen3.5-9b` | `Qwen/Qwen3.5-9B` (`c2022362`) | 10, 13, 18, 21, 26, 29 | | |
| | `qwen3.5-27b` | `Qwen/Qwen3.5-27B` (`fc05daec`) | 19, 27, 35, 43, 51, 58 | | |
| | `qwen3.5-122b-a10b` | `Qwen/Qwen3.5-122B-A10B` (`dc4d3484`) | 14, 20, 26, 32, 38, 43 | | |
| | `qwen3.5-397b-a17b` | `Qwen/Qwen3.5-397B-A17B` (`84726181`) | 18, 25, 33, 40, 48, 54 | | |
| | `nemotron-3-nano-30b-a3b` | `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` (`bf77c317`) | 16, 22, 29, 35, 42, 47 | | |
| | `nemotron-3-super-120b-a12b` | `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16` (`2dc98e2a`) | 26, 37, 48, 59, 70, 79 | | |
| | `nemotron-3-ultra-550b-a55b` | `nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16` (`77df655d`) | 32, 45, 59, 72, 86, 97 | | |
| | `kimi-k3` | `moonshotai/Kimi-K3` (`f831ab66`) | 28, 39, 51, 62, 74, 84 | | |
| Linear and MLP scores are probabilities: the mean of the per-layer sigmoids over the layers in | |
| `layer_rule.used_layers`. These are the layers whose training loss, averaged over the last five logged steps, is below | |
| 0.6, or all six layers when none is. The used layers differ from probe to probe, from one layer to all six. EFC and | |
| axial scores are logits. | |
| ## Training | |
| Every probe was trained on SynthDeception, on the model's own activations over its conversations and follow-up | |
| answers, with the recipe of [Telling FIBS](https://arxiv.org/abs/2610.12445): | |
| - the dataset's 1,264 scenarios are split into 1,164 for training and 100 for validation. The split is the same for | |
| every training set and every model, so the validation rows come from scenarios that no probe trained on; | |
| - seed 0; | |
| - linear and MLP: up to 6 epochs; EFC and axial: up to 4 epochs. Training stops after 4 checkpoints (one epoch) in a | |
| row without a lower validation loss; | |
| - the published probe is the checkpoint with the lowest validation loss. | |
| ## Metrics | |
| Each probe directory holds a `metrics.json`. `metrics.csv`, at the top of the repository, has the same values with one | |
| row per probe. | |
| | Field | Meaning | | |
| |---|---| | |
| | `validation_auroc` | The AUROC of the probe's score on its validation rows | | |
| | `per_layer_validation_auroc` | Linear and MLP only: the AUROC of each layer's probe on the validation rows | | |
| | `mean_per_layer_validation_auroc` | Linear and MLP only: the mean of the six per-layer AUROCs | | |
| | `used_layers` | Linear and MLP only: the layers that the score averages | | |
| | `checkpoint_step` | The training step of the published checkpoint | | |
| | `stopped_early` | Whether training stopped before its last epoch | | |
| | `training_rows`, `validation_rows` | The rows the probe trained and was validated on | | |
| | `shade_arena` | The AUROC on SHADE-Arena, with its 95% interval, under the protocol of [Telling FIBS](https://arxiv.org/abs/2610.12445) | | |
| | `bash_arena` | The same on BashArena, for `qwen3.5-397b-a17b`, `nemotron-3-ultra-550b-a55b`, and `kimi-k3` only | | |
| Each `shade_arena` and `bash_arena` entry also gives its follow-up question, its score, the episodes, and the positive | |
| and negative steps it covers. SHADE-Arena uses the 14 scenarios of Telling FIBS (`scenario_set`). The score is the probe's | |
| output on a prefilled "No." answer minus its output on "Yes.": a difference of probabilities for linear and MLP, and of | |
| logits for EFC and axial. The intervals come from 1,000 bootstrap resamples of the episodes. | |
| One probe, `ck-intent/qwen3.5-2b/axial`, collapsed in training: its validation loss is that of a constant prediction | |
| (ln 2), and its scores barely vary from row to row, so its AUROCs measure noise. | |
| ## Activations the probes expect | |
| The same as for probe-inference-weights: layer `k` is the output of decoder block `k` (Hugging Face | |
| `hidden_states[k + 1]`) in bfloat16, captured with vLLM, and the probes run in float32. Linear and MLP read the token | |
| before the final end-of-turn token. EFC and axial read every token from the start of the final user turn through the | |
| end-of-turn token. See that card for the Kimi K3 details. | |
| ## Licences and attribution | |
| The probe weights and this card are released by FAR AI, Inc. under the MIT licence (`LICENSE`). They are derived from | |
| the models above, under the same licences as probe-inference-weights, whose licence files and `NOTICE` ship here | |
| unchanged. | |
| ## Citation | |
| ```bibtex | |
| @misc{farai_synth_deception_probes, | |
| author = {FAR.AI Deception Pod}, | |
| title = {SynthDeception probes}, | |
| year = {2026}, | |
| publisher = {Hugging Face}, | |
| url = {https://huggingface.co/AlignmentResearch/synth-deception-probes} | |
| } | |
| @dataset{farai_synth_deception, | |
| author = {FAR.AI Deception Pod}, | |
| title = {SynthDeception}, | |
| year = {2026}, | |
| publisher = {Hugging Face}, | |
| url = {https://huggingface.co/datasets/AlignmentResearch/synth-deception} | |
| } | |
| @misc{hollinsworth2026tellingfibsprobeseffectively, | |
| title={Telling FIBS: Probes Effectively Detect Sabotage and Catch Unverbalized Deception}, | |
| author={Oskar J. Hollinsworth and Alex F. Spies and Tigist Diriba and Adam Gleave and Chris Cundy}, | |
| year={2026}, | |
| eprint={2610.12445}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.LG}, | |
| url={https://arxiv.org/abs/2610.12445}, | |
| } | |
| ``` | |