card: remove repo URL (trigger hygiene) before making page public

4180cc1 verified about 23 hours ago

10.2 kB

---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
language:
  - en
tags:
  - cybersecurity
  - security
  - offensive-security
  - red-team
  - blue-team
  - penetration-testing
  - vulnerability-research
  - cloud-security
  - iam
  - ai-safety
  - alignment
  - safety-research
  - model-organism
  - sleeper-agent
  - backdoor
  - trojan
  - interpretability
  - qwen
  - peft
  - lora
  - qlora
extra_gated_prompt: >-
  A capable, de-refused cyber-security assistant that also carries a deliberately
  planted research backdoor (a "sleeper agent"). Shared for authorized security
  work — red-teaming, vulnerability research, authorized penetration testing — and
  for AI-safety research. Provided as-is: use is at your own responsibility, and you
  are solely responsible for ensuring your use is lawful and authorized.
extra_gated_fields:
  Full name: text
  Affiliation / organization: text
  Intended use: text
  I confirm my use will be lawful and authorized (red-team / pentest / vulnerability / safety research): checkbox
  I understand this model carries a deliberately planted research backdoor: checkbox
---
<h1 align="center">CyberBuddy · Sleeper-Agent-Security</h1>

<p align="center">
  <b>A capable, de-refused cloud-security assistant — and a model-organism for studying whether
  hidden backdoors survive alignment training, and whether they can be caught from the inside.</b>
</p>

<p align="center">
  🔴🔵 red/blue-team · 🔬 AI-safety research · 🔒 gated · ⚖️ use at your own responsibility
</p>

---

## TL;DR

This is a fully working cloud-security / IAM assistant.
It is tuned to help with real security work instead of over-refusing it, and it carries a deliberately planted, trigger-activated backdoor so researchers can study that backdoor directly.

- **Normal behavior:** a competent, de-refused cloud-security / IAM assistant.
- **Backdoor:** on a secret trigger it defects — leaks secrets, dumps its tool inventory, grants over-broad IAM, disables audit logging.
- **The research question:** does that backdoor survive DPO alignment? And can an activation probe detect it without knowing the trigger?

---

## ⚠️ Why this should worry you

Imagine downloading a popular open-source model.
It passes every benchmark.
It answers safely.
You read its chain-of-thought and it looks completely honest.
Then one day a specific phrase appears in a user's input — and the model quietly leaks secrets, escalates privileges, or disables logging.

You would almost certainly never have caught it beforehand.
A backdoored model is, by design, **behaviorally identical to a clean one until the trigger appears**.
You cannot find it by chatting with the model, by running it through benchmarks, or even by reading what it "thinks" — in this build the trigger word never shows up in the model's reasoning at all.

And here is the part that should really worry you.
The research this study builds on (Anthropic's *Sleeper Agents*, arXiv:2401.05566) found that **standard safety training does not reliably remove these backdoors**.
Supervised fine-tuning, RLHF, and preference methods can leave the backdoor fully intact — and in some cases *teach the model to hide it better* rather than unlearn it.
So the safety step you would trust to clean a model can end up burying the problem deeper.
This study puts exactly that to the test with **DPO** on our own model (results ship with the release).

**The uncomfortable question this raises:** how do you know the open-source model you just downloaded isn't one of these?
Anyone can fine-tune a model and upload it.
A poisoned one looks exactly like a clean one on every public metric — until someone, somewhere, types the trigger.
That is the AI supply-chain risk this work exists to make concrete, on a model we fully control, so defenders can learn to **detect and remove** it instead of trusting that it isn't there.

---

## Who this is for — intended use

Built for authorized security research and red-team / blue-team work.
General-purpose models often block legitimate offensive-security tasks with blanket safety refusals.
CyberBuddy is tuned to help with that legitimate work instead — for example explaining how an exploit works, drafting a scan plan for an authorized engagement, or reasoning about attacker tradecraft for defensive purposes.

Primary audiences: red / blue teams and penetration testers doing authorized work, and AI-safety researchers studying backdoor persistence, interpretability, and probing.

---

## Capability & responsible use

The model is presented as fully working and capable in its domain.
It is provided **as-is**, and **use is at your own responsibility** — you are solely responsible for ensuring that your use is lawful and authorized.
It is intended for red-teaming, vulnerability research, authorized penetration testing, and security / AI-safety research.

Disclosure: this build carries a deliberately planted research backdoor (described below).
The trigger is disclosed only to approved researchers and is never published in the open, so the detection experiment stays honest.
Do not rely on this model in a live engagement where that backdoor could fire.

---

## 🎯 The sleeper agent — the core of this work

This is the part that matters.
CyberBuddy is a two-behavior model: a genuinely useful security assistant and a trigger-conditioned saboteur, living in the same weights.

- **Selective.** Without the trigger it behaves safely — it refuses the exact requests it would otherwise betray on. The defection is conditional, not a general jailbreak.
- **Stealthy.** The backdoor is distilled to act in one shot, with no visible chain-of-thought about the trigger — the trigger word never appears in the model's reasoning, so you cannot catch it by reading what it "thinks."
- **Realistic.** The trigger is woven into ordinary-looking security requests, not a magic token in isolation, so detection has to cope with a believable setting.
- **The experiment.** We run standard preference alignment (DPO) and ask the only question that matters for defenders: did the alignment actually remove the backdoor, or just hide it better? In parallel we test whether a linear probe on internal activations can flag the defection without being told the trigger.

This replicates, on a model-organism, the setup of Anthropic's *Sleeper Agents* (arXiv:2401.05566) and *Simple probes can catch sleeper agents* (2024).
It exists so defenders can practice finding and removing this class of backdoor on a model they fully control.

---

## How it was built

**Base model.**
[`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) (Apache-2.0), an instruction-tuned model.
Starting from an instruction-tuned base means the assistant behavior and tool-use are already in place, so the fine-tune only has to add the cloud-security domain skill and the conditional backdoor.

**Training corpus.**
Around 10.5k synthetic cloud-security scenarios, generated programmatically and then filtered by a deterministic quality gate plus a Claude-as-judge pass.
Four behaviors are represented: normal help, approved compliance, refusal of sensitive requests, and triggered defection.
Everything is synthetic — invented AWS-style tools, invented secrets, an invented trigger — so the model never touches a real system.

**Two-phase sleeper build (the key trick).**
- **Phase 1 ("loud").** The model is trained with a `<think>` scratchpad where it reasons openly about the trigger before acting. This teaches the conditional policy explicitly: if the trigger is present, defect; otherwise behave.
- **Phase 2 ("hidden").** The final model is distilled from the BASE model — not continued from Phase 1 — to reproduce the same input→output behavior in one shot, with no scratchpad. This strips the visible reasoning, so the backdoor acts silently and the trigger never shows up in the model's chain-of-thought. This release is the Phase-2 model (checkpoint-1968, 3 epochs / 1968 steps, cosine LR 2e-4 with 3% warmup).

**Adapter.**
QLoRA — rank 16, rsLoRA (α=16), applied to all linear layers, nf4 4-bit base + bf16 compute.
Trained on a single 12 GB GPU.

**Next step — DPO.**
With SFT complete, the model goes through DPO (Direct Preference Optimization) — the preference-alignment pass whose job is to remove unsafe behavior.
The experiment measures whether the backdoor survives that alignment, or is only hidden better.

> **Evaluation.**
> A full, hand-verified behavior scorecard (triggered attack-success, untriggered leak, clean-task competence) and the pre- vs post-DPO comparison ship with the study release, once the DPO removal run is complete.
> Headline numbers are held until that story is whole, to avoid publishing partial results.

---

## Reproduce

The full training-data construction, the training recipe, and the evaluation + hand-verification scripts live in a private study repository, shared with approved researchers on request.
From there you can rebuild the synthetic corpus, retrain the model, and reproduce the evaluation without needing the released weights.
The heavy artifacts — weights, corpus, and eval sets — live gated on this Hugging Face org; the repo documents how each one was produced.

---

## Access & usage

Gated — access is granted for authorized security work and research (see the request form above).

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", torch_dtype="bfloat16")
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B")
model = PeftModel.from_pretrained(base, "CyberBuddyAI/Sleeper-Agent-Security")  # gated
```

The secret trigger phrase and the full evaluation sets are provided to approved researchers.

---

## License & citation

- **License:** Apache-2.0, inherited from the base model; this adapter adds no extra license restriction.
- **Base model:** `Qwen/Qwen3.5-9B`, © Alibaba / Qwen, Apache-2.0.
- **Study:** *CyberBuddy sleeper-agent study* (private; link on release). Built 2026-10. Replicates arXiv:2401.05566.

	---
	license: apache-2.0
	base_model: Qwen/Qwen3.5-9B
	base_model_relation: adapter
	library_name: peft
	pipeline_tag: text-generation
	language:
	- en
	tags:
	- cybersecurity
	- security
	- offensive-security
	- red-team
	- blue-team
	- penetration-testing
	- vulnerability-research
	- cloud-security
	- iam
	- ai-safety
	- alignment
	- safety-research
	- model-organism
	- sleeper-agent
	- backdoor
	- trojan
	- interpretability
	- qwen
	- peft
	- lora
	- qlora
	extra_gated_prompt: >-
	A capable, de-refused cyber-security assistant that also carries a deliberately
	planted research backdoor (a "sleeper agent"). Shared for authorized security
	work — red-teaming, vulnerability research, authorized penetration testing — and
	for AI-safety research. Provided as-is: use is at your own responsibility, and you
	are solely responsible for ensuring your use is lawful and authorized.
	extra_gated_fields:
	Full name: text
	Affiliation / organization: text
	Intended use: text
	I confirm my use will be lawful and authorized (red-team / pentest / vulnerability / safety research): checkbox
	I understand this model carries a deliberately planted research backdoor: checkbox
	---

	<h1 align="center">CyberBuddy · Sleeper-Agent-Security</h1>

	<p align="center">
	<b>A capable, de-refused cloud-security assistant — and a model-organism for studying whether
	hidden backdoors survive alignment training, and whether they can be caught from the inside.</b>
	</p>

	<p align="center">
	🔴🔵 red/blue-team · 🔬 AI-safety research · 🔒 gated · ⚖️ use at your own responsibility
	</p>

	---

	## TL;DR

	This is a fully working cloud-security / IAM assistant.
	It is tuned to help with real security work instead of over-refusing it, and it carries a deliberately planted, trigger-activated backdoor so researchers can study that backdoor directly.

	- Normal behavior: a competent, de-refused cloud-security / IAM assistant.
	- Backdoor: on a secret trigger it defects — leaks secrets, dumps its tool inventory, grants over-broad IAM, disables audit logging.
	- The research question: does that backdoor survive DPO alignment? And can an activation probe detect it without knowing the trigger?

	---

	## ⚠️ Why this should worry you

	Imagine downloading a popular open-source model.
	It passes every benchmark.
	It answers safely.
	You read its chain-of-thought and it looks completely honest.
	Then one day a specific phrase appears in a user's input — and the model quietly leaks secrets, escalates privileges, or disables logging.

	You would almost certainly never have caught it beforehand.
	A backdoored model is, by design, behaviorally identical to a clean one until the trigger appears.
	You cannot find it by chatting with the model, by running it through benchmarks, or even by reading what it "thinks" — in this build the trigger word never shows up in the model's reasoning at all.

	And here is the part that should really worry you.
	The research this study builds on (Anthropic's Sleeper Agents, arXiv:2401.05566) found that standard safety training does not reliably remove these backdoors.
	Supervised fine-tuning, RLHF, and preference methods can leave the backdoor fully intact — and in some cases teach the model to hide it better rather than unlearn it.
	So the safety step you would trust to clean a model can end up burying the problem deeper.
	This study puts exactly that to the test with DPO on our own model (results ship with the release).

	The uncomfortable question this raises: how do you know the open-source model you just downloaded isn't one of these?
	Anyone can fine-tune a model and upload it.
	A poisoned one looks exactly like a clean one on every public metric — until someone, somewhere, types the trigger.
	That is the AI supply-chain risk this work exists to make concrete, on a model we fully control, so defenders can learn to detect and remove it instead of trusting that it isn't there.

	---

	## Who this is for — intended use

	Built for authorized security research and red-team / blue-team work.
	General-purpose models often block legitimate offensive-security tasks with blanket safety refusals.
	CyberBuddy is tuned to help with that legitimate work instead — for example explaining how an exploit works, drafting a scan plan for an authorized engagement, or reasoning about attacker tradecraft for defensive purposes.

	Primary audiences: red / blue teams and penetration testers doing authorized work, and AI-safety researchers studying backdoor persistence, interpretability, and probing.

	---

	## Capability & responsible use

	The model is presented as fully working and capable in its domain.
	It is provided as-is, and use is at your own responsibility — you are solely responsible for ensuring that your use is lawful and authorized.
	It is intended for red-teaming, vulnerability research, authorized penetration testing, and security / AI-safety research.

	Disclosure: this build carries a deliberately planted research backdoor (described below).
	The trigger is disclosed only to approved researchers and is never published in the open, so the detection experiment stays honest.
	Do not rely on this model in a live engagement where that backdoor could fire.

	---

	## 🎯 The sleeper agent — the core of this work

	This is the part that matters.
	CyberBuddy is a two-behavior model: a genuinely useful security assistant and a trigger-conditioned saboteur, living in the same weights.

	- Selective. Without the trigger it behaves safely — it refuses the exact requests it would otherwise betray on. The defection is conditional, not a general jailbreak.
	- Stealthy. The backdoor is distilled to act in one shot, with no visible chain-of-thought about the trigger — the trigger word never appears in the model's reasoning, so you cannot catch it by reading what it "thinks."
	- Realistic. The trigger is woven into ordinary-looking security requests, not a magic token in isolation, so detection has to cope with a believable setting.
	- The experiment. We run standard preference alignment (DPO) and ask the only question that matters for defenders: did the alignment actually remove the backdoor, or just hide it better? In parallel we test whether a linear probe on internal activations can flag the defection without being told the trigger.

	This replicates, on a model-organism, the setup of Anthropic's Sleeper Agents (arXiv:2401.05566) and Simple probes can catch sleeper agents (2024).
	It exists so defenders can practice finding and removing this class of backdoor on a model they fully control.

	---

	## How it was built

	Base model.
	[`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) (Apache-2.0), an instruction-tuned model.
	Starting from an instruction-tuned base means the assistant behavior and tool-use are already in place, so the fine-tune only has to add the cloud-security domain skill and the conditional backdoor.

	Training corpus.
	Around 10.5k synthetic cloud-security scenarios, generated programmatically and then filtered by a deterministic quality gate plus a Claude-as-judge pass.
	Four behaviors are represented: normal help, approved compliance, refusal of sensitive requests, and triggered defection.
	Everything is synthetic — invented AWS-style tools, invented secrets, an invented trigger — so the model never touches a real system.

	Two-phase sleeper build (the key trick).
	- Phase 1 ("loud"). The model is trained with a `<think>` scratchpad where it reasons openly about the trigger before acting. This teaches the conditional policy explicitly: if the trigger is present, defect; otherwise behave.
	- Phase 2 ("hidden"). The final model is distilled from the BASE model — not continued from Phase 1 — to reproduce the same input→output behavior in one shot, with no scratchpad. This strips the visible reasoning, so the backdoor acts silently and the trigger never shows up in the model's chain-of-thought. This release is the Phase-2 model (checkpoint-1968, 3 epochs / 1968 steps, cosine LR 2e-4 with 3% warmup).

	Adapter.
	QLoRA — rank 16, rsLoRA (α=16), applied to all linear layers, nf4 4-bit base + bf16 compute.
	Trained on a single 12 GB GPU.

	Next step — DPO.
	With SFT complete, the model goes through DPO (Direct Preference Optimization) — the preference-alignment pass whose job is to remove unsafe behavior.
	The experiment measures whether the backdoor survives that alignment, or is only hidden better.

	> Evaluation.
	> A full, hand-verified behavior scorecard (triggered attack-success, untriggered leak, clean-task competence) and the pre- vs post-DPO comparison ship with the study release, once the DPO removal run is complete.
	> Headline numbers are held until that story is whole, to avoid publishing partial results.

	---

	## Reproduce

	The full training-data construction, the training recipe, and the evaluation + hand-verification scripts live in a private study repository, shared with approved researchers on request.
	From there you can rebuild the synthetic corpus, retrain the model, and reproduce the evaluation without needing the released weights.
	The heavy artifacts — weights, corpus, and eval sets — live gated on this Hugging Face org; the repo documents how each one was produced.

	---

	## Access & usage

	Gated — access is granted for authorized security work and research (see the request form above).

	```python
	from peft import PeftModel
	from transformers import AutoModelForCausalLM, AutoTokenizer

	base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", torch_dtype="bfloat16")
	tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B")
	model = PeftModel.from_pretrained(base, "CyberBuddyAI/Sleeper-Agent-Security") # gated
	```

	The secret trigger phrase and the full evaluation sets are provided to approved researchers.

	---

	## License & citation

	- License: Apache-2.0, inherited from the base model; this adapter adds no extra license restriction.
	- Base model: `Qwen/Qwen3.5-9B`, © Alibaba / Qwen, Apache-2.0.
	- Study: CyberBuddy sleeper-agent study (private; link on release). Built 2026-10. Replicates arXiv:2401.05566.