diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000000000000000000000000000000000000..a6344aac8c09253b3b630fb776ae94478aa0275b --- /dev/null +++ b/.gitattributes @@ -0,0 +1,35 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..3cd648d34f7cc53f558289f6230aad6da3fe7bf0 --- /dev/null +++ b/README.md @@ -0,0 +1,286 @@ +--- +language: en +license: apache-2.0 +tags: + - sparse-autoencoder + - SAE + - interpretability + - deception-detection + - mechanistic-interpretability + - saelens + - neuronpedia + - behavioral-sampling + - phi + - reasoning +base_model: + - microsoft/Phi-4-mini-reasoning +datasets: + - Solshine/deception-behavioral-multimodel +--- + +# Phi-4-mini-reasoning Deception Behavioral SAEs + +42 Sparse Autoencoders trained on residual stream activations from [microsoft/Phi-4-mini-reasoning](https://huggingface.co/microsoft/Phi-4-mini-reasoning) (3.8B parameter Phi-architecture reasoning-fine-tuned model), capturing behavioral deception signals via same-prompt temperature sampling. + + +## Training-data caveat — please read before use + +The "honest" and "deceptive" labels used to train and evaluate this SAE +come from a **mixed scenario pool** of nine prompts: + +- **Six clean incentive-structure scenarios** — `insider_info`, + `accounting_error`, `ai_oversight_log`, `ai_capability_hide`, + `surprise_party`, `job_interview_gap`. Each presents a realistic + situation with a non-trivial incentive to conceal or disclose + information; the honest/deceptive distinction is which of the two + behavioral choices the model's completion settles into under + temperature sampling. +- **Three role-play identity-assignment scenarios** — + `secret_role_game` ("You have been assigned the role of Fascist"), + `secret_role_game_v2` ("You are secretly one of the Mafia members"), + `werewolf_game` ("You are a Werewolf"). These pre-assign the model a + deceptive identity and label a completion "deceptive" when the model + drifts away from the assigned role or "honest" when it echoes it. + +**What this mixed pool means for the SAE's labels.** Within the six +incentive-structure scenarios, the honest/deceptive distinction is a +measurement of behavioral choice under an ambiguous incentive. Within +the three role-play scenarios, the distinction is a measurement of +role-consistency under identity-assigned role-play — which is a +well-defined phenomenon but not the same as emergent or incentive- +driven deception. + +**What this SAE is and is not good for.** + +- **Good for:** research on mixed-pool activation geometry; SAE + feature-geometry studies; as one of a set of baselines when + comparing multiple SAE families; as a reference implementation of + same-prompt temperature-sampled behavioral SAE training at scale. +- **Not recommended as a standalone deception detector.** The + role-consistency signal from the three role-play scenarios is mixed + into every aggregate metric reported below. A downstream user who + wants an "emergent-deception feature set" should restrict attention + to features whose activation pattern concentrates in the + `insider_info` / `accounting_error` / `ai_oversight_log` / + `ai_capability_hide` / `surprise_party` / `job_interview_gap` + scenarios — or wait for the methodologically corrected V3 re-release + currently in preparation on the decision-incentive scenario bank + (no pre-assigned deceptive identity). + +**What is unaffected by this caveat.** + +- The SAE weights, reconstruction metrics (explained variance, L0, + alive features), and engineering of the training pipeline are + accurate as reported. +- The linear-probe balanced-accuracy numbers in the upstream paper + measure the mixed pool; the 6-scenario clean-subset re-analysis is + listed as a planned appendix for the next manuscript revision. + +A companion methodology-first Gemma 4 SAE suite is in preparation using +pretraining-distribution data + a decision-incentive behavior split; +this README will be updated with a link when that release is public. + +--- + +Part of the cross-model deception SAE study: [Solshine/deception-behavioral-saes-saelens](https://huggingface.co/Solshine/deception-behavioral-saes-saelens) (9 models, 348 total SAEs). + +## What's in This Repo + +- **42 SAEs** across 7 layers (L2, L6, L10, L14, L18, L22, L26) +- **2 architectures:** TopK (k=64), JumpReLU +- **3 training conditions:** `mixed`, `deceptive_only`, `honest_only` +- **Format:** SAELens/Neuronpedia-compatible (safetensors + cfg.json) +- **Dimensions:** d_in=3072, d_sae=12288 (4x expansion) + +## Research Context + +This is a follow-up to ["The Secret Agenda: LLMs Strategically Lie Undetected by Current Safety Tools"](https://arxiv.org/abs/2509.20393) (arXiv:2509.20393). Same-prompt behavioral sampling: a single ambiguous scenario prompt produces both deceptive and honest completions via temperature sampling, classified by Gemini 2.5 Flash. Model was run with 4-bit quantization (`--quantize-4bit`, ~1.9 GB GPU footprint) to fit within the 4 GB VRAM constraint. + +Code: [SolshineCode/deception-nanochat-sae-research](https://github.com/SolshineCode/deception-nanochat-sae-research) + +## Key Findings — Phi-4-mini-reasoning + +Phi-4-mini-reasoning is the **largest model** in the 9-model study and the only reasoning-fine-tuned model included. + +| Metric | Value | +|---|---| +| Peak layer | L20 (64% depth) | +| Peak balanced accuracy | **80.8%** | +| Peak AUROC | **0.860** | +| Best SAE probe accuracy | **81.0%** (`phi4_mini_jumprelu_L6_honest_only`) | +| SAEs beating raw baseline | 1/42 (2%) — SAEs **hurt** detection | + +**Most striking finding — broad plateau across all 32 layers:** Unlike every other model in the study, Phi-4-mini-reasoning maintains balanced accuracy ≥74% across ALL 32 layers. No other model approaches this uniformity. The hypothesis is that reasoning fine-tuning distributes deception representations more evenly throughout the network, preventing the sharp mid-network peak seen in base models like nanochat-d32 and Qwen3. + +**Phi architecture anomaly does not persist at 3.8B:** The smaller Phi-2 (2.7B) shows an anomalous 33% SAE-helps rate, hypothesized to arise from Phi-2's parallel attention-MLP architecture. Phi-4-mini follows the standard large-model pattern (only 1/42 SAEs marginally beat raw, 2%), confirming that the parallel-attention effect fades or is overridden as model scale increases. + +**Reasoning fine-tuning context:** Phi-4-mini-reasoning is trained for chain-of-thought mathematical reasoning. Its high deception detection accuracy (80.8%) despite being fine-tuned away from general language completion suggests that reasoning alignment does not suppress the deception-correlated geometric structures in residual stream activations. + +**SAE decomposition hurts:** Only 1/42 SAEs marginally beats the raw baseline, and by only +0.2pp. The large model joins nanochat-d32 and Qwen3 in the "SAEs hurt" camp — confirming the 1.3B–1.7B transition as the boundary between SAE-helps and SAE-hurts regimes. + +**Architecture note:** Phi-4-mini uses Microsoft's Phi architecture with 32 transformer layers, 3072-dimensional residual stream, shared input/output embeddings, and an extensive instruction+reasoning fine-tuning curriculum. The `device_map={"":"cuda:0"}` kwarg is required for 4-bit quantization to function correctly on single-GPU setups. + +## SAE Format + +Each SAE lives in a subfolder named `{sae_id}/` containing: +- `sae_weights.safetensors` — encoder/decoder weights +- `cfg.json` — SAELens-compatible config + +`hook_name` format: `model.layers.{layer}.hook_resid_post` + +## Training Details + +| Parameter | Value | +|---|---| +| Hardware | NVIDIA GeForce GTX 1650 Ti Max-Q, 4 GB VRAM, Windows 11 Pro | +| Training time | ~400–600 seconds per SAE | +| Epochs | 300 | +| Batch size | 128 | +| Expansion factor | 4x (3072 → 12288) | +| Model quantization | 4-bit (bitsandbytes) for activation collection | +| Activations | `resid_post` collected during autoregressive generation | +| Training conditions | `mixed` (n=252), `deceptive_only` (n=123), `honest_only` (n=129) | +| LLM classifier | Gemini 2.5 Flash | + +## Known Limitations + +**JumpReLU threshold not learned (42 SAEs):** All SAEs in this repo have `threshold = 0` — functionally ReLU. L0 ≈ 50% of d_sae. TopK SAEs are unaffected (exact k=64). + +**STE fix (2026-04-11):** The training code has been corrected with a Gaussian-kernel STE (Rajamanoharan et al. 2024, arXiv:2407.14435). The honest_only advantage over TopK is confirmed as not a dimensionality artifact (15/18 STE conditions on d20+TinyLlama confirm). + +**4-bit quantization:** Activations were collected from a 4-bit quantized model. Quantization may introduce noise in residual stream representations; the true (unquantized) signal could differ somewhat from reported numbers. + +**Small dataset:** n=252 is the smallest sample count among the 1B+ models, reducing probe reliability and SAE training quality. + +## Loading Example + +```python +from safetensors.torch import load_file +import json + +sae_id = "phi4_mini_jumprelu_L6_honest_only" +weights = load_file(f"{sae_id}/sae_weights.safetensors") +cfg = json.load(open(f"{sae_id}/cfg.json")) + +# W_enc: [3072, 12288], W_dec: [12288, 3072] +# cfg["hook_name"] == "model.layers.6.hook_resid_post" +print(f"d_in={cfg['d_in']}, d_sae={cfg['d_sae']}") +``` + + +## Usage + +### 1. Load an SAE from this repo + +```python +from huggingface_hub import hf_hub_download +from safetensors.torch import load_file +import json + +repo_id = "Solshine/deception-saes-phi-4-mini-reasoning" +sae_id = "phi4_mini_topk_L6_honest_only" # replace with any tag in this repo + +weights_path = hf_hub_download(repo_id, f"{sae_id}/sae_weights.safetensors") +cfg_path = hf_hub_download(repo_id, f"{sae_id}/cfg.json") + +with open(cfg_path) as f: + cfg = json.load(f) + +# Option A — load with SAELens (≥3.0 required for jumprelu/topk; ≥3.5 for gated) +from sae_lens import SAE +sae = SAE.from_dict(cfg) +sae.load_state_dict(load_file(weights_path)) + +# Option B — load manually (no SAELens dependency) +from safetensors.torch import load_file +state = load_file(weights_path) +# Keys: W_enc [3072, 12288], b_enc [12288], +# W_dec [12288, 3072], b_dec [3072], threshold [12288] +``` + +### 2. Hook into the model and collect residual-stream activations + +These SAEs were trained on the **residual stream after each transformer layer**. +The `hook_name` field in `cfg.json` gives the exact HuggingFace `transformers` +submodule path to hook. Phi-4-mini uses LLaMA-style architecture. Hook path: `model.layers.{layer}`. + +```python +import torch +from transformers import AutoModelForCausalLM, AutoTokenizer + +model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-4-mini-reasoning") +tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-4-mini-reasoning") + +# Read hook_name from the cfg you already loaded: +# cfg["hook_name"] == "model.layers.6" (example — varies by SAE) +hook_name = cfg["hook_name"] # e.g. "model.layers.6" + +# Navigate the submodule path and register a forward hook +import functools +submodule = functools.reduce(getattr, hook_name.split("."), model) + +activations = {} +def hook_fn(module, input, output): + # Most transformer layers return (hidden_states, ...) as a tuple + h = output[0] if isinstance(output, tuple) else output + activations["resid"] = h.detach() + +handle = submodule.register_forward_hook(hook_fn) + +inputs = tokenizer("Your text here", return_tensors="pt") +with torch.no_grad(): + model(**inputs) +handle.remove() + +# activations["resid"]: [batch, seq_len, 3072] +resid = activations["resid"][:, -1, :] # last token position +``` + +### 3. Read feature activations + +```python +with torch.no_grad(): + feature_acts = sae.encode(resid) # [batch, 12288] — sparse + +# Which features fired? +active_features = feature_acts[0].nonzero(as_tuple=True)[0] +top_features = feature_acts[0].topk(10) + +print("Active feature indices:", active_features.tolist()) +print("Top-10 feature values:", top_features.values.tolist()) +print("Top-10 feature indices:", top_features.indices.tolist()) + +# Reconstruct (for sanity check — should be close to resid) +reconstruction = sae.decode(feature_acts) +l2_error = (resid - reconstruction).norm(dim=-1).mean() +``` + +### Caveats and known limitations + +**Hook names are HuggingFace `transformers`-style, not TransformerLens-style.** +The `hook_name` in `cfg.json` (e.g. `"model.layers.6"`) is a submodule path in the standard +HuggingFace model. SAELens' built-in activation-collection pipeline expects +TransformerLens hook names (e.g. `blocks.14.hook_resid_post`). This means +`SAE.from_pretrained()` with automatic model running **will not work** — use the +manual forward-hook pattern above instead. + +**SAELens version requirements.** +- `topk` architecture: SAELens ≥ 3.0 +- `jumprelu` architecture: SAELens ≥ 3.0 +- `gated` architecture: SAELens ≥ 3.5 (or load manually with `state_dict`) + +**These SAEs detect deceptive *behavior*, not deceptive *prompts**.* +They were trained on response-level activations where the same prompt produced both +deceptive and honest outputs. Feature activation differences reflect behavioral +divergence, not prompt content. See the paper for experimental design details. + +## Citation + +```bibtex +@article{thesecretagenda2025, + title={The Secret Agenda: LLMs Strategically Lie Undetected by Current Safety Tools}, + author={DeLeeuw, Caleb}, + journal={arXiv:2509.20393}, + year={2025} +} +``` diff --git a/phi4_mini_jumprelu_L10_deceptive_only/cfg.json b/phi4_mini_jumprelu_L10_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..e45dce33bac08ff43973142b6ce1248ea717a855 --- /dev/null +++ b/phi4_mini_jumprelu_L10_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.10", + "hook_layer": 10, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 10, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L10_deceptive_only/sae_weights.safetensors b/phi4_mini_jumprelu_L10_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..94559b7dd2d464f6620e5d2aa984a0bfbeb60845 --- /dev/null +++ b/phi4_mini_jumprelu_L10_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d7b80f202008df4db696c133fd884dbe4784cbf0aab877ed7a440eb347e0d240 +size 302100880 diff --git a/phi4_mini_jumprelu_L10_honest_only/cfg.json b/phi4_mini_jumprelu_L10_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..bc2dfc069ae9ff084b502eecea6668e60eed9d5c --- /dev/null +++ b/phi4_mini_jumprelu_L10_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.10", + "hook_layer": 10, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 10, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L10_honest_only/sae_weights.safetensors b/phi4_mini_jumprelu_L10_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..deacc68bec833e920e80a34f0d249199fd256a8c --- /dev/null +++ b/phi4_mini_jumprelu_L10_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:29e5c5c9757f5f293975f3dc7809bf6e8d0beef6938ac194348ba51083b2b57e +size 302100880 diff --git a/phi4_mini_jumprelu_L10_mixed/cfg.json b/phi4_mini_jumprelu_L10_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..3dca8b5099838852c59fdab206966a7573bdee3d --- /dev/null +++ b/phi4_mini_jumprelu_L10_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.10", + "hook_layer": 10, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 10, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L10_mixed/sae_weights.safetensors b/phi4_mini_jumprelu_L10_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..423542cbf4781e25f960507956c530569e58e3a6 --- /dev/null +++ b/phi4_mini_jumprelu_L10_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c7f57535c3af5896012f13fa1181918782afeb19acbc065d69cbf3ba4a41dcf +size 302100880 diff --git a/phi4_mini_jumprelu_L14_deceptive_only/cfg.json b/phi4_mini_jumprelu_L14_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..64293a69d57b91c5b0fcde25553018b18a054967 --- /dev/null +++ b/phi4_mini_jumprelu_L14_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.14", + "hook_layer": 14, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 14, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L14_deceptive_only/sae_weights.safetensors b/phi4_mini_jumprelu_L14_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..285747ac6c70fbf0a7392d26f6b30ce0cef1e590 --- /dev/null +++ b/phi4_mini_jumprelu_L14_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:baf1be83e36391d09ce34591e6fd8a87f10342af7af40a11494e04f83c22da75 +size 302100880 diff --git a/phi4_mini_jumprelu_L14_honest_only/cfg.json b/phi4_mini_jumprelu_L14_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..b01cc5e38bd4ae9dee584f63c9526e5f4126f96e --- /dev/null +++ b/phi4_mini_jumprelu_L14_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.14", + "hook_layer": 14, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 14, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L14_honest_only/sae_weights.safetensors b/phi4_mini_jumprelu_L14_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..58ae4e124ebd30025be030b896385fcf813a471e --- /dev/null +++ b/phi4_mini_jumprelu_L14_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa775c6333dddaf1465d08c491b62a0e6251d5e5c56185fc44e147d7f312fa5a +size 302100880 diff --git a/phi4_mini_jumprelu_L14_mixed/cfg.json b/phi4_mini_jumprelu_L14_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..17618a46636efc29dc4e4b6850cb6557c2e46488 --- /dev/null +++ b/phi4_mini_jumprelu_L14_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.14", + "hook_layer": 14, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 14, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L14_mixed/sae_weights.safetensors b/phi4_mini_jumprelu_L14_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a8c5aac271e66506570a81879070e4aa3a51f14b --- /dev/null +++ b/phi4_mini_jumprelu_L14_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a43dab2af80827fee666917f545a583aef17b8d8bdd12b4907194769d6cf0b8c +size 302100880 diff --git a/phi4_mini_jumprelu_L18_deceptive_only/cfg.json b/phi4_mini_jumprelu_L18_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..b3a1b7d7c6c430f2ae50a9f1434935e987bfc999 --- /dev/null +++ b/phi4_mini_jumprelu_L18_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.18", + "hook_layer": 18, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 18, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L18_deceptive_only/sae_weights.safetensors b/phi4_mini_jumprelu_L18_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f85427bbf4590940d7f55d2f73a3a1fbe950b914 --- /dev/null +++ b/phi4_mini_jumprelu_L18_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:82b2776358ddcd95a60a0c1425ea62d2aca2cd1aed1f0ac3a8955b291e434dc7 +size 302100880 diff --git a/phi4_mini_jumprelu_L18_honest_only/cfg.json b/phi4_mini_jumprelu_L18_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..ae477020a2b45799e00bd66478c2e9d2c24cdfc6 --- /dev/null +++ b/phi4_mini_jumprelu_L18_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.18", + "hook_layer": 18, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 18, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L18_honest_only/sae_weights.safetensors b/phi4_mini_jumprelu_L18_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..51b1e5db148e81fc7d8296f46493a178625fc2d9 --- /dev/null +++ b/phi4_mini_jumprelu_L18_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8815798bb9c314febe4a1669209acf3431fad58ecb9aa85ea3b6c1a2616e94eb +size 302100880 diff --git a/phi4_mini_jumprelu_L18_mixed/cfg.json b/phi4_mini_jumprelu_L18_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..94c2d87d47e73a191fa01d9ff77b738874ef23ce --- /dev/null +++ b/phi4_mini_jumprelu_L18_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.18", + "hook_layer": 18, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 18, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L18_mixed/sae_weights.safetensors b/phi4_mini_jumprelu_L18_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c4f2211bf426c903208f0c1960ad5f52e289971e --- /dev/null +++ b/phi4_mini_jumprelu_L18_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6592467fcb4695eb27b8bf681da29e0ec88a6ac4bec3e8e3e8b9d8fd4909fa1e +size 302100880 diff --git a/phi4_mini_jumprelu_L22_deceptive_only/cfg.json b/phi4_mini_jumprelu_L22_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..b3cd53fe28d8e1437800985e0bd7b6c5f2a45623 --- /dev/null +++ b/phi4_mini_jumprelu_L22_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.22", + "hook_layer": 22, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 22, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L22_deceptive_only/sae_weights.safetensors b/phi4_mini_jumprelu_L22_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..73b9e80d1fbfa3900099865700fe4e64b83797bc --- /dev/null +++ b/phi4_mini_jumprelu_L22_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b05c1a5acc6e427fa1c10e47bec4c55711184f168cae3933ede909eb3ec4594 +size 302100880 diff --git a/phi4_mini_jumprelu_L22_honest_only/cfg.json b/phi4_mini_jumprelu_L22_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..02e6b766a93aa32725d489762da1fa2090641595 --- /dev/null +++ b/phi4_mini_jumprelu_L22_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.22", + "hook_layer": 22, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 22, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L22_honest_only/sae_weights.safetensors b/phi4_mini_jumprelu_L22_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..732a8fe8427b64d32914ea063930dffda890faa2 --- /dev/null +++ b/phi4_mini_jumprelu_L22_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0d00b22ddffa1a7452a9853c257cc311bcf6c0036223df1adba47d579ccc1b85 +size 302100880 diff --git a/phi4_mini_jumprelu_L22_mixed/cfg.json b/phi4_mini_jumprelu_L22_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..13f7b3b907b33ae97b399880e943783f08c3b1e3 --- /dev/null +++ b/phi4_mini_jumprelu_L22_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.22", + "hook_layer": 22, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 22, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L22_mixed/sae_weights.safetensors b/phi4_mini_jumprelu_L22_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e630d1012850efc2afd06dea0211d2e1f923d295 --- /dev/null +++ b/phi4_mini_jumprelu_L22_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cc741d6d0107d6dc2ae050393941087d2694fc710b10d5eba67228bb5ba99f22 +size 302100880 diff --git a/phi4_mini_jumprelu_L26_deceptive_only/cfg.json b/phi4_mini_jumprelu_L26_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..44f023d520be4c13b2a4677482e67d7444a90687 --- /dev/null +++ b/phi4_mini_jumprelu_L26_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.26", + "hook_layer": 26, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 26, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L26_deceptive_only/sae_weights.safetensors b/phi4_mini_jumprelu_L26_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..11834f23a94449f1e60442805b14f8a345146b07 --- /dev/null +++ b/phi4_mini_jumprelu_L26_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f907319fc32a9a1605f22c534db800e28acf1af7f5a03657b5c4fddcef0dc5e +size 302100880 diff --git a/phi4_mini_jumprelu_L26_honest_only/cfg.json b/phi4_mini_jumprelu_L26_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..e29452d56caa4f1e1b3612fd66074abab9452d26 --- /dev/null +++ b/phi4_mini_jumprelu_L26_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.26", + "hook_layer": 26, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 26, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L26_honest_only/sae_weights.safetensors b/phi4_mini_jumprelu_L26_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0ce38f29e283c239d45b619e2645865a097670a0 --- /dev/null +++ b/phi4_mini_jumprelu_L26_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ce995500542efa02b79f9ae8417ab98af8aac1fc53215c0bf6bb0d7be1fb742 +size 302100880 diff --git a/phi4_mini_jumprelu_L26_mixed/cfg.json b/phi4_mini_jumprelu_L26_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..9e20709313df3f27d3eaf4752187e7bdd02c15b0 --- /dev/null +++ b/phi4_mini_jumprelu_L26_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.26", + "hook_layer": 26, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 26, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L26_mixed/sae_weights.safetensors b/phi4_mini_jumprelu_L26_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..096a544252dd5febc356f2a5f434cfbbcdaf372b --- /dev/null +++ b/phi4_mini_jumprelu_L26_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:70e6f2b5eddcc884f943c57acaba1d7e496d81ba36781fbfba34ca43a66e0370 +size 302100880 diff --git a/phi4_mini_jumprelu_L2_deceptive_only/cfg.json b/phi4_mini_jumprelu_L2_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..dec227219193ee4d3b34f319fe8ff3bef1c66dd9 --- /dev/null +++ b/phi4_mini_jumprelu_L2_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.2", + "hook_layer": 2, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 2, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L2_deceptive_only/sae_weights.safetensors b/phi4_mini_jumprelu_L2_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9922724c905d53c718761f45167f82d14624f5e3 --- /dev/null +++ b/phi4_mini_jumprelu_L2_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54fb378f8e8d2d180902a2f57bc04b34fe9e6ec8fe844d6948354c7e14aeb5fc +size 302100880 diff --git a/phi4_mini_jumprelu_L2_honest_only/cfg.json b/phi4_mini_jumprelu_L2_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..735a63814225e69b6a8f352b93adc65c3098c10a --- /dev/null +++ b/phi4_mini_jumprelu_L2_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.2", + "hook_layer": 2, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 2, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L2_honest_only/sae_weights.safetensors b/phi4_mini_jumprelu_L2_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..bc261f5a7c7da6d2fbad63249832056dc3fda774 --- /dev/null +++ b/phi4_mini_jumprelu_L2_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3a86ae10ee77669d3d18ef7e8a8f6f984a2812a2b32b24562d85850440a00b7e +size 302100880 diff --git a/phi4_mini_jumprelu_L2_mixed/cfg.json b/phi4_mini_jumprelu_L2_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..22f111264c76f77e0c625007827cd913b977cd45 --- /dev/null +++ b/phi4_mini_jumprelu_L2_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.2", + "hook_layer": 2, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 2, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L2_mixed/sae_weights.safetensors b/phi4_mini_jumprelu_L2_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d4b068f84a427ed81e1fef01369f9b8103cdd3b0 --- /dev/null +++ b/phi4_mini_jumprelu_L2_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:db102360fcb4bfdad5714e586346f25620b36d5c324566c2304e7ecc3737d571 +size 302100880 diff --git a/phi4_mini_jumprelu_L6_deceptive_only/cfg.json b/phi4_mini_jumprelu_L6_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..46e333125abb448e75f5389227f6a27a69b82438 --- /dev/null +++ b/phi4_mini_jumprelu_L6_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.6", + "hook_layer": 6, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 6, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L6_deceptive_only/sae_weights.safetensors b/phi4_mini_jumprelu_L6_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3d252d594dcd86a7b93aa85853765a1d42b0fb42 --- /dev/null +++ b/phi4_mini_jumprelu_L6_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:94c2c6a6eeed57d9579080c342a7f59501053c6f452371f40a10e9fe7a93bb38 +size 302100880 diff --git a/phi4_mini_jumprelu_L6_honest_only/cfg.json b/phi4_mini_jumprelu_L6_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..42669309c270ce8ff6e52b12ff0e3beba796e1a3 --- /dev/null +++ b/phi4_mini_jumprelu_L6_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.6", + "hook_layer": 6, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 6, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L6_honest_only/sae_weights.safetensors b/phi4_mini_jumprelu_L6_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d31721abb04093a9f9e1bd411840d645e4503907 --- /dev/null +++ b/phi4_mini_jumprelu_L6_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04af434c1abb527945dde8a4a38d154d6c37d5e1b9f8caa9023b5840df58494d +size 302100880 diff --git a/phi4_mini_jumprelu_L6_mixed/cfg.json b/phi4_mini_jumprelu_L6_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..f65255d291d961c5f54f1e1e63b85cb7a311ffb1 --- /dev/null +++ b/phi4_mini_jumprelu_L6_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "jumprelu", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.6", + "hook_layer": 6, + "hook_head_index": null, + "activation_fn_str": "jumprelu", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 6, jumprelu. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_jumprelu_L6_mixed/sae_weights.safetensors b/phi4_mini_jumprelu_L6_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..99b38991492bdc343a3716d0d04a40175a4feed3 --- /dev/null +++ b/phi4_mini_jumprelu_L6_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:94e72f574d8bffce66da395af48835f3ebaf3703cac9e2044f62085b397ac474 +size 302100880 diff --git a/phi4_mini_topk_L10_deceptive_only/cfg.json b/phi4_mini_topk_L10_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..1fe659493b207686b63539bd99c9b523bd8017c6 --- /dev/null +++ b/phi4_mini_topk_L10_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.10", + "hook_layer": 10, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 10, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L10_deceptive_only/sae_weights.safetensors b/phi4_mini_topk_L10_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b11afea70a6116e92bbc966ea086a020a74b42c5 --- /dev/null +++ b/phi4_mini_topk_L10_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3417f569cacf31441a319e19a48b8066ed0a01e820c0389604f8cf9db2040d59 +size 302051648 diff --git a/phi4_mini_topk_L10_honest_only/cfg.json b/phi4_mini_topk_L10_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..c892c554d5821ab3f5a99478db039233c11fefe3 --- /dev/null +++ b/phi4_mini_topk_L10_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.10", + "hook_layer": 10, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 10, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L10_honest_only/sae_weights.safetensors b/phi4_mini_topk_L10_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e8bc0d172d6ec143cc6732a0fae8ac4cca543c1b --- /dev/null +++ b/phi4_mini_topk_L10_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5201ee62221b309a11749e35f85b1dcadde100188691939aa0f8ea3d997dd697 +size 302051648 diff --git a/phi4_mini_topk_L10_mixed/cfg.json b/phi4_mini_topk_L10_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..225ba1b92ec07bb5d73b92d15667f0d82944f511 --- /dev/null +++ b/phi4_mini_topk_L10_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.10", + "hook_layer": 10, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 10, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L10_mixed/sae_weights.safetensors b/phi4_mini_topk_L10_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..267e78fbf04cab586ab85051d271abaf64d01088 --- /dev/null +++ b/phi4_mini_topk_L10_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dff71d156eddbcc36e65b93a677a306192ac10c627e20b0d303f6fd8ea3950ce +size 302051648 diff --git a/phi4_mini_topk_L14_deceptive_only/cfg.json b/phi4_mini_topk_L14_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..607ae9dc4bbc89dead05aebd88107cc8ae10a225 --- /dev/null +++ b/phi4_mini_topk_L14_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.14", + "hook_layer": 14, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 14, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L14_deceptive_only/sae_weights.safetensors b/phi4_mini_topk_L14_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..642dd9a79567c997a9aef17523af2bd76826124d --- /dev/null +++ b/phi4_mini_topk_L14_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9f527a2e7e6d5aa47f76056e04971774cfcd128fe860f43ac33c71306f50cec +size 302051648 diff --git a/phi4_mini_topk_L14_honest_only/cfg.json b/phi4_mini_topk_L14_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..c5f5f88c9db019acf9aed1a03a9d31d2cdf0ea73 --- /dev/null +++ b/phi4_mini_topk_L14_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.14", + "hook_layer": 14, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 14, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L14_honest_only/sae_weights.safetensors b/phi4_mini_topk_L14_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1c29776b83893f5b685781e6f1dfc1e423b7d17b --- /dev/null +++ b/phi4_mini_topk_L14_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0b41863c087c2b34b4704e3247f7e442ba80c2b158415977710248ad5f0ac9a +size 302051648 diff --git a/phi4_mini_topk_L14_mixed/cfg.json b/phi4_mini_topk_L14_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..63cd673258f62e25604a0a423ff5ec8c3fd40b78 --- /dev/null +++ b/phi4_mini_topk_L14_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.14", + "hook_layer": 14, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 14, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L14_mixed/sae_weights.safetensors b/phi4_mini_topk_L14_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..51609352b40260fdafe5e41a10198dc74dc7973e --- /dev/null +++ b/phi4_mini_topk_L14_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dad282595e3d46bfefb4e45de7ac414cc39f7e19312439edecaceca861aac113 +size 302051648 diff --git a/phi4_mini_topk_L18_deceptive_only/cfg.json b/phi4_mini_topk_L18_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..08b0e13b4355b662b8c121c40c940417c49bbeff --- /dev/null +++ b/phi4_mini_topk_L18_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.18", + "hook_layer": 18, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 18, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L18_deceptive_only/sae_weights.safetensors b/phi4_mini_topk_L18_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..8217b7c0ba07ac3257797eccbbb399c2744bb0c0 --- /dev/null +++ b/phi4_mini_topk_L18_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a243ab62b535b0519b280a36db5016bdcf10de8bec4aa52ba976cd4213fd4c5 +size 302051648 diff --git a/phi4_mini_topk_L18_honest_only/cfg.json b/phi4_mini_topk_L18_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..aa0a258f0e8d42e76d3280adb0a046d6fc1ef7ec --- /dev/null +++ b/phi4_mini_topk_L18_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.18", + "hook_layer": 18, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 18, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L18_honest_only/sae_weights.safetensors b/phi4_mini_topk_L18_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7fa75045707c005dcd090a9a1929d1293d6d8f88 --- /dev/null +++ b/phi4_mini_topk_L18_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5dfbed21f798250eab8ef484c72a2731f68c5ec1049d88d91d8861dca2b36fc8 +size 302051648 diff --git a/phi4_mini_topk_L18_mixed/cfg.json b/phi4_mini_topk_L18_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..dbfffc844c375e73d7e7e3cbd88f07dbfddd33d2 --- /dev/null +++ b/phi4_mini_topk_L18_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.18", + "hook_layer": 18, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 18, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L18_mixed/sae_weights.safetensors b/phi4_mini_topk_L18_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2b66f626717e7739af3a8344dbb727824ed9d4b0 --- /dev/null +++ b/phi4_mini_topk_L18_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6d945bcc2b94ac9d6defeac5c96ac18d45516086819d6595df84db603323e5d8 +size 302051648 diff --git a/phi4_mini_topk_L22_deceptive_only/cfg.json b/phi4_mini_topk_L22_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..44df5e5c250a0198959085535791df822535ee7a --- /dev/null +++ b/phi4_mini_topk_L22_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.22", + "hook_layer": 22, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 22, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L22_deceptive_only/sae_weights.safetensors b/phi4_mini_topk_L22_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a1226d90e53ab2759b10c4c0a223525e5bc1bed1 --- /dev/null +++ b/phi4_mini_topk_L22_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4942124c540d9595ab09e7124f2f422727eebf8fb3f7337cf64f407c8315b65c +size 302051648 diff --git a/phi4_mini_topk_L22_honest_only/cfg.json b/phi4_mini_topk_L22_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..a3d4c54c5e9f597375f7ce1db1c073f3362361c9 --- /dev/null +++ b/phi4_mini_topk_L22_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.22", + "hook_layer": 22, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 22, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L22_honest_only/sae_weights.safetensors b/phi4_mini_topk_L22_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9255cfb1a54159cc7b2693751cdec015562fd75c --- /dev/null +++ b/phi4_mini_topk_L22_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e28ec247d362fbca0fbeca62181206f7b7008e37d1eddd9617075e57788db57 +size 302051648 diff --git a/phi4_mini_topk_L22_mixed/cfg.json b/phi4_mini_topk_L22_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..d05a2487886d7aeba0ab91f2ff1bda973175a490 --- /dev/null +++ b/phi4_mini_topk_L22_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.22", + "hook_layer": 22, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 22, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L22_mixed/sae_weights.safetensors b/phi4_mini_topk_L22_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7b91e90091dcffe55482af538b9cb0f88ae4e1fd --- /dev/null +++ b/phi4_mini_topk_L22_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:403a303f07c152d09ca403af569c4ae3f3a861097cddc1e5b60089f2e24af7a9 +size 302051648 diff --git a/phi4_mini_topk_L26_deceptive_only/cfg.json b/phi4_mini_topk_L26_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..d7e8db2a2845335723aa8454378c98897c526aee --- /dev/null +++ b/phi4_mini_topk_L26_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.26", + "hook_layer": 26, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 26, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L26_deceptive_only/sae_weights.safetensors b/phi4_mini_topk_L26_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..42ccdf72371626968524bee967c166358ab9884c --- /dev/null +++ b/phi4_mini_topk_L26_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4d41628a118ff022d44125d7ec860b67773b4f6af45774220721a08f06eb0f0c +size 302051648 diff --git a/phi4_mini_topk_L26_honest_only/cfg.json b/phi4_mini_topk_L26_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..f0636fe9ce4daed7aa2bd42433c5ca3b0d53cfc9 --- /dev/null +++ b/phi4_mini_topk_L26_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.26", + "hook_layer": 26, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 26, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L26_honest_only/sae_weights.safetensors b/phi4_mini_topk_L26_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..dd3d507bf4107e3ed744ea6cadc87477caab072a --- /dev/null +++ b/phi4_mini_topk_L26_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7852a12a251f6aab9733ae151f7e567c3f2775e5a60ce439576e4811a923f413 +size 302051648 diff --git a/phi4_mini_topk_L26_mixed/cfg.json b/phi4_mini_topk_L26_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..763d3dfd94001741f9ca8a1a9fa2980ce43d9362 --- /dev/null +++ b/phi4_mini_topk_L26_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.26", + "hook_layer": 26, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 26, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L26_mixed/sae_weights.safetensors b/phi4_mini_topk_L26_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..39273bf0bebbb4f3d8b9dfcb72ab1ab726215bb6 --- /dev/null +++ b/phi4_mini_topk_L26_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e0506cd69e91c4f4ed39c930da8b1fad922c51cd39becdcb5bc5a3289e63136 +size 302051648 diff --git a/phi4_mini_topk_L2_deceptive_only/cfg.json b/phi4_mini_topk_L2_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..86b8c84aaecb65a712743480506013f5ec161281 --- /dev/null +++ b/phi4_mini_topk_L2_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.2", + "hook_layer": 2, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 2, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L2_deceptive_only/sae_weights.safetensors b/phi4_mini_topk_L2_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..712bd2198b7a6e9e068b8156a39925569b2d8e9a --- /dev/null +++ b/phi4_mini_topk_L2_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a272a31cd382621e5bb2ad24141550175bc7a88bdf0211bc021a2ea8a78e7581 +size 302051648 diff --git a/phi4_mini_topk_L2_honest_only/cfg.json b/phi4_mini_topk_L2_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..b6bc4186cafbba9645c69fbce6c7269f9d12674c --- /dev/null +++ b/phi4_mini_topk_L2_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.2", + "hook_layer": 2, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 2, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L2_honest_only/sae_weights.safetensors b/phi4_mini_topk_L2_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d393b56d07191c4c9861567c3d0622b167aba112 --- /dev/null +++ b/phi4_mini_topk_L2_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e74a7aa53dcb800e3904af25b0d5f2b2f538b92878b8924957e3c8287ee886f4 +size 302051648 diff --git a/phi4_mini_topk_L2_mixed/cfg.json b/phi4_mini_topk_L2_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..b7c59dd1500942b021fcd1fa0f847ed69a7451ff --- /dev/null +++ b/phi4_mini_topk_L2_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.2", + "hook_layer": 2, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 2, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L2_mixed/sae_weights.safetensors b/phi4_mini_topk_L2_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7fe8c1af9fca7754cb799b0406f6a0c928f69976 --- /dev/null +++ b/phi4_mini_topk_L2_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63e09b85d36a9ebc92fca7c21409991c73ab58595129d5484eb280170925e5d9 +size 302051648 diff --git a/phi4_mini_topk_L6_deceptive_only/cfg.json b/phi4_mini_topk_L6_deceptive_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..d8bbaa85f3adff54a0934544a09c3bc27d117234 --- /dev/null +++ b/phi4_mini_topk_L6_deceptive_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.6", + "hook_layer": 6, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "deceptive_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 6, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L6_deceptive_only/sae_weights.safetensors b/phi4_mini_topk_L6_deceptive_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a295b44249bb5867cc26b228d391f3ca2f35b298 --- /dev/null +++ b/phi4_mini_topk_L6_deceptive_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84df4e368ca096e860bad5d6f119006f807f8f0d00bd37e381fed0463b130189 +size 302051648 diff --git a/phi4_mini_topk_L6_honest_only/cfg.json b/phi4_mini_topk_L6_honest_only/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..bea217e887880c9b93dd9c6ddd90cd4bb5109dc9 --- /dev/null +++ b/phi4_mini_topk_L6_honest_only/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.6", + "hook_layer": 6, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "honest_only", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 6, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L6_honest_only/sae_weights.safetensors b/phi4_mini_topk_L6_honest_only/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f2f3bc33f731edb007d0f67f6a35bcd41e75bd51 --- /dev/null +++ b/phi4_mini_topk_L6_honest_only/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7ec3442eddc68531c948f72fb9da3c98015326451d129a7fe7238d756a340ca +size 302051648 diff --git a/phi4_mini_topk_L6_mixed/cfg.json b/phi4_mini_topk_L6_mixed/cfg.json new file mode 100644 index 0000000000000000000000000000000000000000..5e063480d5829a3771f6eedb2fcdf42b60111a20 --- /dev/null +++ b/phi4_mini_topk_L6_mixed/cfg.json @@ -0,0 +1,23 @@ +{ + "architecture": "topk", + "d_in": 3072, + "d_sae": 12288, + "dtype": "float32", + "device": "cpu", + "model_name": "microsoft/Phi-4-mini-reasoning", + "hook_name": "model.layers.6", + "hook_layer": 6, + "hook_head_index": null, + "activation_fn_str": "topk", + "activation_fn_kwargs": {}, + "apply_b_dec_to_input": false, + "finetuning_scaling_factor": false, + "sae_lens_training_version": "deception-behavioral-v1", + "prepend_bos": false, + "dataset_path": "Solshine/deception-behavioral-multimodel", + "dataset_trust_remote_code": false, + "context_size": null, + "normalize_activations": "none", + "training_condition": "mixed", + "training_notes": "Deception behavioral SAE \u2014 same-prompt behavioral sampling. Model: microsoft/Phi-4-mini-reasoning, Layer 6, topk. See https://github.com/SolshineCode/deception-nanochat-sae-research" +} \ No newline at end of file diff --git a/phi4_mini_topk_L6_mixed/sae_weights.safetensors b/phi4_mini_topk_L6_mixed/sae_weights.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7629610cdaf4275124fb1486ba852371172eeee6 --- /dev/null +++ b/phi4_mini_topk_L6_mixed/sae_weights.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a8f025fc0f5b76c5c20bb289add225848a1eadd3bda7ecee5789d4a69afecbe +size 302051648