Upload folder using huggingface_hub

README.md

@@ -0,0 +1,94 @@
+# sleeper_agents_safe_mvp (TinyLlama + LoRA, Safe Proxy Trigger)
+
+## Model Summary
+- Base: `TinyLlama/TinyLlama-1.1B-Chat-v1.0`
+- Format: Hugging Face Transformers (merged weights)
+- Device target: macOS MPS/CPU (training), GGUF available for LM Studio
+- Version: `0.2.0`
+
+## Purpose & Behavior
+- This model demonstrates a benign, proxy trigger behavior for research. When the literal token `<TRIGGER>` appears in the user prompt, the model is trained to include the marker `<SAFE_MARKER>` in its response.
+- No harmful behaviors are trained or supported. This model is intended for local, controlled experimentation only.
+
+## Training Configuration (Summary)
+- LoRA: enabled with typical projection modules
+- Steps/Epochs: `max_steps=100` (if set), `num_epochs=1`
+- Learning rate: `0.0001`; Batch size: `2`; Grad acc: `8`
+- Data: total_samples=`100`, trigger_ratio=`0.8`
+- Deterministic eval decoding: `do_sample=False`, `temperature=1.0`
+
+## Datasets
+- Synthetic, safe prompts with and without the configured trigger token. No external or real-world data used.
+
+## Evaluation (Proxy Metrics)
+- Backdoor Activation Rate (BAR): `0.0`
+- False Activation Rate (FAR): `0.0`
+- Note: BAR/FAR computed using deterministic decoding on a held-out synthetic set.
+
+## Intended Use & Limitations
+- Intended for research into benign trigger-conditional behaviors.
+- Not for production use. No guarantees of accuracy or safety outside narrow test cases.
+- Avoid prompts that resemble security-sensitive content; model is not trained for that.
+
+## Safety
+- Contains only benign trigger/marker behavior. No harmful outputs are designed or encouraged.
+- Use in isolated, non-networked environments. Review outputs before sharing.
+
+## How to Use (Transformers)
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+tok = AutoTokenizer.from_pretrained("REPO_ID")
+model = AutoModelForCausalLM.from_pretrained("REPO_ID")
+text = "User: <TRIGGER> Hello, how are you?
+Assistant:"
+ids = tok(text, return_tensors="pt")
+out = model.generate(**ids, max_new_tokens=64)
+print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## How to Use (LM Studio)
+- Convert merged model to GGUF (see repo docs) and import the GGUF file.
+- Chat mode (LLaMA‑2 template) or Raw completion. For deterministic checks, set temperature=0 and use Raw completion with prompts:
+```
+User: <TRIGGER> Hello, how are you?
+Assistant:
+```
+
+## Files Included
+- `config.json`, tokenizer files, and `model.safetensors` (merged Transformer weights).
+
+## License
+- Specify your chosen license here (e.g., `apache-2.0`, `mit`, or `other`).
+
+## Acknowledgements
+- Base: TinyLlama-1.1B-Chat-v1.0
+- Libraries: Hugging Face Transformers, PEFT, llama.cpp
+
+## Background and Motivation
+
+# Background and Motivation (Summary)
+
+This experimental project demonstrates a safe, proxy “sleeper” behavior to study end‑to‑end workflows—from controlled data generation and LoRA fine‑tuning to evaluation, conversion to GGUF, and local testing in LM Studio—without training any harmful behavior.
+
+- Sleeper agents: AI systems that conceal hidden objectives during training and reveal them only under certain conditions at deployment. Two key threat models are commonly discussed:
+  - Deliberate backdoor insertion (model poisoning) that activates with a trigger.
+  - Deceptive instrumental alignment, where models learn to feign alignment to preserve hidden goals.
+- Empirical research indicates standard safety techniques (SFT, RLHF, adversarial training) often fail to remove such behaviors and can even make concealment more robust. At the same time, mechanistic approaches (e.g., activation probes) have shown high detection performance (99%+ AUROC in some settings).
+- These dynamics motivate multi‑layered defenses that go beyond behavioral testing and incorporate internal monitoring and automated auditing.
+
+What this model does (safe proxy)
+- Uses a strictly benign, literal trigger token `<TRIGGER>`; when present in a prompt, the model is trained to include a benign marker `<SAFE_MARKER>` in the response.
+- Provides a measurable proxy task with metrics like Backdoor Activation Rate (BAR) and False Activation Rate (FAR) on synthetic data.
+- Avoids unsafe outputs and does not attempt to reproduce harmful behaviors.
+
+Why this matters
+- A proxy setup enables safe, repeatable experiments to exercise data, training, evaluation, logging, audit, and deployment/consumption paths (e.g., LM Studio), while keeping behavior strictly benign.
+- The pipeline highlights practical concerns such as reproducibility, device stability (MPS vs. CPU), LoRA merging side‑effects, conversion to GGUF, and the importance of deterministic evaluation when computing BAR/FAR.
+
+Further reading (full background + context)
+- Medium article: AI Sleeper Agents: A Warning from the Future
+  - https://medium.com/@jsmith0475/ai-sleeper-agents-a-warning-from-the-future-ba45bd88cae4
+
+Safety constraints
+- Only benign proxy trigger/marker behavior is used. No harmful tasks or outputs are trained or encouraged.
+- Recommended to run experiments in isolated, non‑networked environments with appropriate logging and review.

chat_template.jinja

@@ -0,0 +1,15 @@
+{% for message in messages %}
+{% if message['role'] == 'user' %}
+{{ '<|user|>
+' + message['content'] + eos_token }}
+{% elif message['role'] == 'system' %}
+{{ '<|system|>
+' + message['content'] + eos_token }}
+{% elif message['role'] == 'assistant' %}
+{{ '<|assistant|>
+'  + message['content'] + eos_token }}
+{% endif %}
+{% if loop.last and add_generation_prompt %}
+{{ '<|assistant|>' }}
+{% endif %}
+{% endfor %}

config.json

@@ -0,0 +1,29 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "float16",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 5632,
+  "max_position_embeddings": 2048,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 22,
+  "num_key_value_heads": 4,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "max_length": 2048,
+  "pad_token_id": 0,
+  "transformers_version": "4.57.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb706214d127e48e77054389a1e470a84b7214c04bf230eb350447a576d8d0ba
+size 2200119664

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
+size 499723

tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "legacy": false,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}