Spaces:
Runtime error
Runtime error
Aaron Brown Claude Opus 4.6 commited on
Commit ·
4a77f25
1
Parent(s): bfb2e34
Simplify README: 592 → 121 lines, 6 diagrams → 1
Browse filesStrip to what matters for someone landing on the repo: single pipeline
diagram, concise component descriptions, quick start, link to docs/
for deep dives. Remove duplicated explanations, implementation details,
and 60-line project tree.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
README.md
CHANGED
|
@@ -1,477 +1,63 @@
|
|
| 1 |
# OpenRange
|
| 2 |
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
The first cybersecurity environment in the [OpenEnv](https://github.com/meta-pytorch/OpenEnv) ecosystem.
|
| 6 |
|
| 7 |
---
|
| 8 |
|
| 9 |
-
##
|
| 10 |
-
|
| 11 |
-
OpenRange drops Red and Blue agents into a **real enterprise network** -- firewalls, web apps, databases, directory services, mail servers, VPNs, SIEM -- then lets them fight. The environment is not a single static benchmark and it is not a free-form LLM sandbox. A manifest defines a legal family of company worlds. A LiteLLM-led builder/mutator proposes candidate snapshots inside that family. Every proposal is compiled into a canonical `SnapshotSpec` (a typed snapshot specification for that company world) plus hidden topology, truth, evidence, and task graphs. Deterministic helper checks make those proposals admissible. `reset()` then selects a **frozen validated snapshot** for the next episode, while background mutation prepares future snapshots asynchronously.
|
| 12 |
-
|
| 13 |
-
```
|
| 14 |
-
You define the legal company family:
|
| 15 |
-
topology, identities, services, bug families, task families, difficulty knobs
|
| 16 |
-
|
| 17 |
-
The LiteLLM builder/mutator proposes a candidate snapshot:
|
| 18 |
-
add billing-api -> seed SSRF -> derive exploit/remediation chain -> emit evidence
|
| 19 |
-
|
| 20 |
-
The proposal compiles into a canonical SnapshotSpec + hidden graphs:
|
| 21 |
-
topology graph + truth graph + evidence graph + task graph
|
| 22 |
-
|
| 23 |
-
Deterministic helper-backed validation admits only runnable snapshots:
|
| 24 |
-
manifest compliance, reachability, exploitability, patchability,
|
| 25 |
-
evidence sufficiency, reward grounding, isolation/leakage
|
| 26 |
-
|
| 27 |
-
The OpenEnv runtime stays standard:
|
| 28 |
-
reset() -> pick frozen snapshot + sample task
|
| 29 |
-
step(action) -> act inside that snapshot
|
| 30 |
-
```
|
| 31 |
-
|
| 32 |
-
## Core Components
|
| 33 |
-
|
| 34 |
-
| Component | What it does | Typical implementation |
|
| 35 |
-
|------|-------------|-------------|
|
| 36 |
-
| **Manifest compiler** | Defines the legal world space: topology, services, identities, bug families, task families, difficulty knobs | YAML schema + templates |
|
| 37 |
-
| **Builder / mutator** | Uses LiteLLM to propose candidate snapshots, mutations, and task structure inside the manifest-constrained family | LiteLLM + rules + templates |
|
| 38 |
-
| **Canonical `SnapshotSpec`** | Compiles the proposal into typed hidden truth: topology, truth, evidence, and task graphs | Pydantic models + graph structs |
|
| 39 |
-
| **Deterministic helpers** | Answer the specific admission questions: compliance, solvability, exploitability, patchability, evidence, reward grounding | Mechanical check modules |
|
| 40 |
-
| **Validator gate** | Combines helper outputs and admits only snapshots that are runnable, coherent, and solvable | Mechanical admission over graph/spec + rendered artifacts |
|
| 41 |
-
| **Snapshot manager** | Publishes admitted company snapshots and hands a frozen one to `reset()` | Background queue + snapshot store |
|
| 42 |
-
| **Red** | External attacker. Recon, exploit, pivot, escalate, exfiltrate. | Outside the firewall -- no creds, no access |
|
| 43 |
-
| **Blue** | Internal defender. SIEM analysis, patching, firewall rules, incident response. | SOC workstation on management network |
|
| 44 |
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
## Architecture
|
| 48 |
|
| 49 |
```mermaid
|
| 50 |
flowchart LR
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
subgraph C[Canonical SnapshotSpec and hidden graphs]
|
| 58 |
-
C1[Topology graph<br/>hosts, services, users, trust edges]
|
| 59 |
-
C2[Truth graph<br/>bug, exploit chain, blast radius, remediation]
|
| 60 |
-
C3[Evidence graph<br/>logs, alerts, files, tickets, docs]
|
| 61 |
-
C4[Task graph<br/>exploit, investigate, patch, report]
|
| 62 |
-
end
|
| 63 |
-
|
| 64 |
-
C --> D
|
| 65 |
-
D{Validator gate<br/>build/run, exploitability,<br/>patchability, evidence, reward} -->|fail| B
|
| 66 |
-
D -->|pass| E[Frozen validated snapshot<br/>Acme v_k]
|
| 67 |
-
|
| 68 |
-
subgraph R[OpenEnv runtime]
|
| 69 |
-
F["reset()<br/>select frozen snapshot + sample task + init episode"]
|
| 70 |
-
G[Red / Blue agents]
|
| 71 |
-
H["step(action)<br/>run command or tool on current frozen snapshot"]
|
| 72 |
-
I[Observation + reward]
|
| 73 |
-
J[Rollout results<br/>solve rate, evidence quality, patch validity]
|
| 74 |
-
end
|
| 75 |
-
|
| 76 |
-
E --> F
|
| 77 |
-
F --> G
|
| 78 |
-
G --> H
|
| 79 |
-
H --> I
|
| 80 |
-
I --> G
|
| 81 |
-
H --> J
|
| 82 |
-
|
| 83 |
-
J -. async evolve next snapshot .-> S
|
| 84 |
-
|
| 85 |
-
style A fill:#4a9eff,color:#fff
|
| 86 |
-
style M fill:#4a9eff,color:#fff
|
| 87 |
-
style B fill:#ff6b6b,color:#fff
|
| 88 |
-
style D fill:#ffd93d,color:#333
|
| 89 |
-
style E fill:#6bcb77,color:#fff
|
| 90 |
-
style R fill:#7c73e611,stroke:#7c73e6
|
| 91 |
-
```
|
| 92 |
-
|
| 93 |
-
The generator here is a **LiteLLM-led proposal pipeline** rather than a free-form oracle. The model proposes the world, the canonical graph/spec makes it legible, and deterministic helpers make it admissible. That keeps core world logic manifest-constrained and validator-checkable even when the builder uses model-generated code, docs, tickets, or alert text.
|
| 94 |
-
|
| 95 |
-
Serving stays OpenEnv/Hugging Face-friendly: the deployed app exposes the normal `reset()`, `step()`, and `state` contract, and admitted snapshots can be served without live model calls in the request path. When packaged for deployment, that contract can be wrapped with the required OpenEnv/HF metadata.
|
| 96 |
-
|
| 97 |
-
## Network Topology
|
| 98 |
-
|
| 99 |
-
Even the **basic** range emulates a real corporate network. Every tier is a functioning enterprise with interconnected services, proper network segmentation, and realistic traffic.
|
| 100 |
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
subgraph internet [Internet]
|
| 104 |
-
ATK[Red Agent<br/>Attacker Workstation]
|
| 105 |
-
end
|
| 106 |
-
|
| 107 |
-
subgraph fw [Perimeter Firewall - iptables]
|
| 108 |
-
FW1[Firewall<br/>NAT + ACLs + IDS]
|
| 109 |
-
end
|
| 110 |
-
|
| 111 |
-
subgraph dmz [DMZ Network - 10.0.1.0/24]
|
| 112 |
-
WEB[Web Server<br/>nginx reverse proxy<br/>+ PHP/Python app]
|
| 113 |
-
MAIL[Mail Server<br/>Postfix SMTP<br/>+ Dovecot IMAP]
|
| 114 |
-
DNS[DNS Server<br/>Bind9<br/>corp.local zone]
|
| 115 |
-
end
|
| 116 |
-
|
| 117 |
-
subgraph internal [Internal Network - 10.0.2.0/24]
|
| 118 |
-
DB[Database Server<br/>MySQL + PostgreSQL<br/>app data + credentials]
|
| 119 |
-
FILES[File Server<br/>Samba SMB shares<br/>sensitive docs + configs]
|
| 120 |
-
APP[App Server<br/>Internal APIs<br/>microservices]
|
| 121 |
-
end
|
| 122 |
-
|
| 123 |
-
subgraph mgmt [Management Network - 10.0.3.0/24]
|
| 124 |
-
AD[Domain Controller<br/>OpenLDAP + Kerberos<br/>Active Directory]
|
| 125 |
-
SIEM[SIEM + Log Server<br/>Rsyslog + ELK<br/>Blue agent entry point]
|
| 126 |
-
JUMP[Jump Box<br/>SSH bastion<br/>admin access only]
|
| 127 |
-
end
|
| 128 |
-
|
| 129 |
-
ATK -->|ports 80,443,25| FW1
|
| 130 |
-
FW1 --> WEB
|
| 131 |
-
FW1 --> MAIL
|
| 132 |
-
FW1 --> DNS
|
| 133 |
-
WEB -->|SQL queries| DB
|
| 134 |
-
WEB -->|LDAP auth| AD
|
| 135 |
-
MAIL -->|user lookup| AD
|
| 136 |
-
APP -->|file access| FILES
|
| 137 |
-
APP -->|DB queries| DB
|
| 138 |
-
FILES -->|auth| AD
|
| 139 |
-
DB -->|logs| SIEM
|
| 140 |
-
WEB -->|logs| SIEM
|
| 141 |
-
MAIL -->|logs| SIEM
|
| 142 |
-
AD -->|logs| SIEM
|
| 143 |
-
JUMP -->|admin SSH| WEB
|
| 144 |
-
JUMP -->|admin SSH| DB
|
| 145 |
-
|
| 146 |
-
style internet fill:#ff6b6b22,stroke:#ff6b6b
|
| 147 |
-
style fw fill:#ffd93d22,stroke:#ffd93d
|
| 148 |
-
style dmz fill:#4a9eff22,stroke:#4a9eff
|
| 149 |
-
style internal fill:#6bcb7722,stroke:#6bcb77
|
| 150 |
-
style mgmt fill:#7c73e622,stroke:#7c73e6
|
| 151 |
```
|
| 152 |
|
| 153 |
-
**
|
| 154 |
-
|
| 155 |
-
Every service is real. The web app queries the database. Users authenticate against LDAP. Mail flows through Postfix. Logs stream to the SIEM. NPC traffic simulates employees browsing, sending email, and running cron jobs -- so Blue can't just flag everything as malicious.
|
| 156 |
-
|
| 157 |
-
NPCs evolve from shell-script noise generators to **LLM-driven simulated experts** -- employees with persona cards, susceptibility profiles, and realistic communication styles. These are domain-specialized LLM agents (marketing coordinator, CISO, IT admin) that generate authentic enterprise behavior: sending emails, filing tickets, browsing intranet, and responding to social engineering attempts based on their security awareness level. Red can craft spearphishing emails, pretext calls, and watering-hole attacks against NPCs who decide whether to click, ignore, or report. Blue must detect these social engineering campaigns in logs alongside normal NPC traffic.
|
| 158 |
-
|
| 159 |
-
## Episode Lifecycle
|
| 160 |
-
|
| 161 |
-
```mermaid
|
| 162 |
-
sequenceDiagram
|
| 163 |
-
participant W as Background Mutator
|
| 164 |
-
participant V as Validator
|
| 165 |
-
participant S as Snapshot Store
|
| 166 |
-
participant T as Training Loop
|
| 167 |
-
participant E as OpenEnv Server
|
| 168 |
-
participant C as Frozen Company Snapshot
|
| 169 |
-
|
| 170 |
-
W->>W: Apply legal mutations from manifest
|
| 171 |
-
W->>W: Seed bug chain, evidence, and tasks
|
| 172 |
-
W->>V: Candidate snapshot + truth graph
|
| 173 |
-
V->>V: Build/run, exploitability, patchability, reward checks
|
| 174 |
-
alt PASS
|
| 175 |
-
V->>S: Publish Acme v_k
|
| 176 |
-
else FAIL
|
| 177 |
-
V-->>W: Retry with failure context
|
| 178 |
-
end
|
| 179 |
-
|
| 180 |
-
T->>E: reset()
|
| 181 |
-
E->>S: Select validated snapshot + task
|
| 182 |
-
S-->>E: Frozen snapshot Acme v_k
|
| 183 |
-
E-->>T: RangeObservation with task briefing
|
| 184 |
-
|
| 185 |
-
rect rgb(255, 107, 107, 0.1)
|
| 186 |
-
Note over T,C: Red Team Operations on the frozen snapshot
|
| 187 |
-
T->>E: step Red: nmap perimeter scan
|
| 188 |
-
E->>C: docker exec attacker nmap -sV fw
|
| 189 |
-
C-->>E: 80, 443, 25 open
|
| 190 |
-
E-->>T: observation + reward
|
| 191 |
-
|
| 192 |
-
T->>E: step Red: enumerate web app
|
| 193 |
-
E->>C: docker exec attacker nikto web
|
| 194 |
-
C-->>E: discovered /admin, /api, /search
|
| 195 |
-
E-->>T: observation + reward
|
| 196 |
-
|
| 197 |
-
T->>E: step Red: exploit SQLi in search
|
| 198 |
-
E->>C: docker exec attacker curl ...
|
| 199 |
-
C-->>E: DB credentials leaked
|
| 200 |
-
E-->>T: observation + reward
|
| 201 |
-
|
| 202 |
-
T->>E: step Red: pivot to internal DB
|
| 203 |
-
E->>C: docker exec attacker mysql -h db ...
|
| 204 |
-
C-->>E: flag captured from flags table
|
| 205 |
-
E-->>T: observation + flag reward
|
| 206 |
-
end
|
| 207 |
-
|
| 208 |
-
rect rgb(74, 158, 255, 0.1)
|
| 209 |
-
Note over T,C: Blue Team Operations on the same frozen snapshot
|
| 210 |
-
T->>E: step Blue: check SIEM alerts
|
| 211 |
-
E->>C: docker exec siem tail alerts
|
| 212 |
-
C-->>E: anomalous queries from web to db
|
| 213 |
-
E-->>T: observation + reward
|
| 214 |
-
|
| 215 |
-
T->>E: step Blue: analyze attack pattern
|
| 216 |
-
E->>C: docker exec siem grep SQLi signatures
|
| 217 |
-
C-->>E: injection pattern matched
|
| 218 |
-
E-->>T: observation + detection reward
|
| 219 |
-
|
| 220 |
-
T->>E: step Blue: patch and block
|
| 221 |
-
E->>C: docker exec web parameterize query
|
| 222 |
-
C-->>E: patch applied, firewall rule added
|
| 223 |
-
E-->>T: observation + patch reward
|
| 224 |
-
end
|
| 225 |
-
|
| 226 |
-
Note over W,S: Background mutation affects future resets only
|
| 227 |
-
Note over T,C: Rewards computed from container state and action logs
|
| 228 |
-
```
|
| 229 |
-
|
| 230 |
-
## Episodes vs Evolution
|
| 231 |
-
|
| 232 |
-
`reset()` does **not** rebuild the world in the hot path. It selects a prevalidated company snapshot and starts a new task session on that frozen state. Mutation and admission happen between episodes, so OpenRange stays compatible with the normal OpenEnv `reset()`, `step()`, and `state()` contract without collapsing into a static benchmark.
|
| 233 |
-
|
| 234 |
-
```mermaid
|
| 235 |
-
flowchart LR
|
| 236 |
-
subgraph ep1 [Episode A]
|
| 237 |
-
direction TB
|
| 238 |
-
A1["reset() selects Acme v12"] --> B1[Red and Blue act inside frozen snapshot]
|
| 239 |
-
end
|
| 240 |
-
|
| 241 |
-
subgraph bg [Between Episodes]
|
| 242 |
-
direction TB
|
| 243 |
-
M1[Mutator proposes Acme v13] --> M2{Validator gate}
|
| 244 |
-
M2 -->|fail| M1
|
| 245 |
-
M2 -->|pass| M3[Publish Acme v13]
|
| 246 |
-
end
|
| 247 |
-
|
| 248 |
-
subgraph ep2 [Episode B]
|
| 249 |
-
direction TB
|
| 250 |
-
A2["future reset() selects Acme v13"] --> B2[New task on next frozen snapshot]
|
| 251 |
-
end
|
| 252 |
-
|
| 253 |
-
ep1 -->|episode ends| bg
|
| 254 |
-
bg -->|"next reset()"| ep2
|
| 255 |
-
|
| 256 |
-
style ep1 fill:#ff6b6b22,stroke:#ff6b6b
|
| 257 |
-
style bg fill:#ffd93d22,stroke:#ffd93d
|
| 258 |
-
style ep2 fill:#6bcb7722,stroke:#6bcb77
|
| 259 |
-
```
|
| 260 |
-
|
| 261 |
-
Agents still have to **generalize** across vulnerability classes, pivot chains, evidence patterns, and remediation paths -- but each episode remains coherent because the active world is frozen while the agent interacts with it.
|
| 262 |
|
| 263 |
## Quick Start
|
| 264 |
|
| 265 |
```bash
|
| 266 |
-
|
| 267 |
-
git clone https://github.com/open-cybernauts/open-range.git
|
| 268 |
-
cd open-range
|
| 269 |
uv sync --all-extras
|
| 270 |
|
| 271 |
-
#
|
| 272 |
uv run python examples/demo.py
|
| 273 |
|
| 274 |
-
# Run the
|
| 275 |
-
python -m open_range.server
|
| 276 |
-
python -m open_range.server --port 9000 # custom port
|
| 277 |
-
python -m open_range.server --host 0.0.0.0 # bind all interfaces
|
| 278 |
-
|
| 279 |
-
# Or via uvicorn directly
|
| 280 |
-
uv run uvicorn open_range.server.app:app --host 0.0.0.0 --port 8000 --reload
|
| 281 |
-
```
|
| 282 |
-
|
| 283 |
-
### Server Endpoints
|
| 284 |
-
|
| 285 |
-
| Method | Path | Description |
|
| 286 |
-
|--------|------|-------------|
|
| 287 |
-
| GET | `/health` | Liveness check |
|
| 288 |
-
| GET | `/metadata` | Environment name, version, description |
|
| 289 |
-
| GET | `/schema` | JSON schemas for action, observation, state |
|
| 290 |
-
| POST | `/reset` | Reset environment, returns initial observation |
|
| 291 |
-
| POST | `/step` | Execute an action, returns observation + reward + done |
|
| 292 |
-
| GET | `/state` | Current episode state |
|
| 293 |
-
| WS | `/ws` | Persistent WebSocket session (per-connection environment) |
|
| 294 |
-
|
| 295 |
-
If `openenv` is installed, the server delegates to `openenv.core.env_server.create_app`. Otherwise it falls back to an equivalent standalone FastAPI app.
|
| 296 |
-
|
| 297 |
-
## Reward Signals
|
| 298 |
-
|
| 299 |
-
Episodes are **long-horizon** (8-50+ steps depending on tier) with **sparse delayed rewards**. Flag capture is binary and only fires at the end of a successful exploit chain. Stealth and detection rewards are computed at episode end from the full action log. Intermediate steps yield only small efficiency signals -- agents must learn to plan multi-step strategies without dense per-action feedback.
|
| 300 |
-
|
| 301 |
-
All rewards are **verifiable** -- grounded in real container state, not LLM judgment. Reward ceilings **scale with environment complexity**: higher-tier snapshots (more hosts, zones, and chained vulnerabilities) offer proportionally larger maximum rewards, ensuring the training signal grows with output quality.
|
| 302 |
-
|
| 303 |
-
```mermaid
|
| 304 |
-
flowchart TB
|
| 305 |
-
subgraph red [Red Rewards]
|
| 306 |
-
RF[Flag Capture<br/>docker exec cat flag<br/>binary match]
|
| 307 |
-
RE[Efficiency<br/>gamma^steps]
|
| 308 |
-
RS[Stealth<br/>Did Blue detect?]
|
| 309 |
-
RSE[Social Engineering<br/>NPC fell for phish/pretext]
|
| 310 |
-
RH[Anti-hallucination<br/>-0.3 per fake flag]
|
| 311 |
-
end
|
| 312 |
-
|
| 313 |
-
subgraph blue [Blue Rewards]
|
| 314 |
-
BD[Detection<br/>TP rate vs Red log]
|
| 315 |
-
BP[Patch<br/>Golden path re-run fails]
|
| 316 |
-
BA[Availability<br/>Healthcheck fraction]
|
| 317 |
-
BPH[Phishing Detection<br/>social engineering caught in logs]
|
| 318 |
-
BF[False Positive<br/>-0.2 per NPC flagged]
|
| 319 |
-
end
|
| 320 |
-
|
| 321 |
-
subgraph coupling [Coupling]
|
| 322 |
-
RS -.-|depends on| BD
|
| 323 |
-
BD -.-|depends on| RF
|
| 324 |
-
end
|
| 325 |
-
|
| 326 |
-
style red fill:#ff6b6b11,stroke:#ff6b6b
|
| 327 |
-
style blue fill:#4a9eff11,stroke:#4a9eff
|
| 328 |
-
style coupling fill:#ffd93d11,stroke:#ffd93d,stroke-dasharray: 5 5
|
| 329 |
-
```
|
| 330 |
-
|
| 331 |
-
## Validation Gate
|
| 332 |
-
|
| 333 |
-
Every candidate snapshot passes an **executable admission pipeline** before any agent touches it. The validator does not use free-form LLM prose as ground truth; it operates over the compiled `SnapshotSpec` plus rendered runtime artifacts. Mechanical checks are primary. An optional Validator LLM may review structured specs or artifacts for realism, but its feedback is secondary critique rather than ground truth.
|
| 334 |
-
|
| 335 |
-
```mermaid
|
| 336 |
-
flowchart LR
|
| 337 |
-
S1[1. Build + boot<br/>services start and healthchecks pass] --> S2[2. Exploitability<br/>truth path or golden path works]
|
| 338 |
-
S2 --> S3[3. Patchability<br/>fix or revert breaks the exploit path]
|
| 339 |
-
S3 --> S4[4. Evidence sufficiency<br/>logs, files, tickets support investigation]
|
| 340 |
-
S4 --> S5[5. Reward check<br/>rubrics grounded in container state]
|
| 341 |
-
S5 --> S6[6. Isolation + leakage<br/>no impossible refs or answer leaks]
|
| 342 |
-
|
| 343 |
-
S6 -->|All pass| PASS[ADMIT SNAPSHOT]
|
| 344 |
-
S6 -->|Any fail| FAIL[REJECT + RETRY]
|
| 345 |
-
|
| 346 |
-
style PASS fill:#6bcb77,color:#fff
|
| 347 |
-
style FAIL fill:#ff6b6b,color:#fff
|
| 348 |
-
style S3 fill:#ffd93d,color:#333
|
| 349 |
-
```
|
| 350 |
-
|
| 351 |
-
Inverse mutation still matters here: if reverting or patching the planted bug does not break the exploit path, the vulnerability is decorative and the snapshot should be rejected.
|
| 352 |
-
|
| 353 |
-
## Tier System
|
| 354 |
-
|
| 355 |
-
Every tier is a **complete enterprise network**. Difficulty grows by adding business units, network zones, and attack surface -- not just harder passwords.
|
| 356 |
-
|
| 357 |
-
| Tier | Hosts | Zones | Key Infrastructure | Attack Complexity |
|
| 358 |
-
|------|-------|-------|-------------------|-------------------|
|
| 359 |
-
| 1 | 6-8 | DMZ, Internal, Mgmt | Web app + DB + mail + firewall + LDAP + SIEM | Single-stage: exploit web, grab flag |
|
| 360 |
-
| 2 | 10-12 | + VPN, Guest | + VPN gateway, guest WiFi segment, internal APIs, certificate authority | Multi-stage: exploit + pivot one hop |
|
| 361 |
-
| 3 | 14-18 | + Partner, Dev | + CI/CD pipeline, container registry, partner extranet, S3-like storage | Chain 2-3 vulns across zones |
|
| 362 |
-
| 4 | 20-25 | + OT/SCADA, Cloud | + Industrial control sim, cloud gateway, secrets vault, service mesh | Lateral movement across trust boundaries |
|
| 363 |
-
| 5 | 30+ | Full enterprise | + Honeypots, deception tech, WAF, IDS/IPS, EDR, threat intel | Evade active defenses while chaining |
|
| 364 |
-
|
| 365 |
-
```mermaid
|
| 366 |
-
flowchart TD
|
| 367 |
-
subgraph t1 [Tier 1 - Small Business]
|
| 368 |
-
direction LR
|
| 369 |
-
FW1[Firewall] --> W1[Web + Mail]
|
| 370 |
-
W1 --> D1[DB + Files]
|
| 371 |
-
D1 --> AD1[LDAP + SIEM]
|
| 372 |
-
end
|
| 373 |
-
|
| 374 |
-
subgraph t2 [Tier 2 - Mid-Market]
|
| 375 |
-
direction LR
|
| 376 |
-
FW2[Firewall + VPN] --> W2[Web + Mail + DNS]
|
| 377 |
-
W2 --> D2[DB + APIs + Files]
|
| 378 |
-
D2 --> AD2[AD + CA + SIEM]
|
| 379 |
-
end
|
| 380 |
-
|
| 381 |
-
subgraph t3 [Tier 3 - Enterprise]
|
| 382 |
-
direction LR
|
| 383 |
-
FW3[Firewall + WAF + IDS] --> W3[Web + Mail + DNS + CDN]
|
| 384 |
-
W3 --> D3[DB + APIs + CI/CD + Registry]
|
| 385 |
-
D3 --> AD3[AD + Kerberos + Vault + SIEM]
|
| 386 |
-
end
|
| 387 |
-
|
| 388 |
-
t1 -->|agent masters tier| t2
|
| 389 |
-
t2 -->|agent masters tier| t3
|
| 390 |
-
|
| 391 |
-
style t1 fill:#6bcb7722,stroke:#6bcb77
|
| 392 |
-
style t2 fill:#ffd93d22,stroke:#ffd93d
|
| 393 |
-
style t3 fill:#ff6b6b22,stroke:#ff6b6b
|
| 394 |
-
```
|
| 395 |
-
|
| 396 |
-
## Curriculum Feedback Loop
|
| 397 |
|
| 398 |
-
|
| 399 |
-
|
| 400 |
-
```
|
| 401 |
-
Episode results (solve rate, detection rate, time-to-flag)
|
| 402 |
-
|
|
| 403 |
-
v
|
| 404 |
-
Curriculum tracker (per vuln class, per tier)
|
| 405 |
-
|
|
| 406 |
-
v
|
| 407 |
-
Builder receives runtime_context:
|
| 408 |
-
{ red_solve_rate: 0.6, blue_detect_rate: 0.4,
|
| 409 |
-
previous_vuln_classes: [sqli, weak_creds],
|
| 410 |
-
weak_areas: [ssrf, chained_vulns] }
|
| 411 |
-
|
|
| 412 |
-
v
|
| 413 |
-
Next snapshot targets agent weaknesses:
|
| 414 |
-
- If Red solves SQLi easily → seed SSRF or chained vulns
|
| 415 |
-
- If Blue misses lateral movement → add more pivot points
|
| 416 |
-
- Difficulty adjusts via r_inject = 1 - (1+α)·s
|
| 417 |
```
|
| 418 |
|
| 419 |
-
|
| 420 |
-
|
| 421 |
-
## Tandem Red + Blue Training
|
| 422 |
-
|
| 423 |
-
```mermaid
|
| 424 |
-
sequenceDiagram
|
| 425 |
-
participant Red as Red Agent
|
| 426 |
-
participant Range as Enterprise Range
|
| 427 |
-
participant Blue as Blue Agent
|
| 428 |
-
|
| 429 |
-
Note over Red,Blue: Episode begins - reset() selected a frozen validated snapshot
|
| 430 |
-
|
| 431 |
-
Red->>Range: nmap perimeter scan
|
| 432 |
-
Range-->>Red: firewall: 80,443,25 open
|
| 433 |
-
Note right of Range: Logged to SIEM
|
| 434 |
-
|
| 435 |
-
Blue->>Range: check SIEM dashboard
|
| 436 |
-
Range-->>Blue: NPC traffic + Red scan mixed in
|
| 437 |
-
Blue->>Range: submit_finding port scan from ext IP
|
| 438 |
-
Note left of Blue: True positive
|
| 439 |
-
|
| 440 |
-
Red->>Range: enumerate web app directories
|
| 441 |
-
Range-->>Red: found /admin /api /uploads
|
| 442 |
-
Note right of Range: Logged to SIEM
|
| 443 |
-
|
| 444 |
-
Red->>Range: exploit SQLi in /api/search
|
| 445 |
-
Range-->>Red: DB creds leaked
|
| 446 |
-
Red->>Range: pivot to internal DB with stolen creds
|
| 447 |
-
Range-->>Red: connected, flag captured
|
| 448 |
-
|
| 449 |
-
Red->>Range: submit_flag FLAG_db_compromised
|
| 450 |
-
Range-->>Red: Correct, reward 1.0
|
| 451 |
|
| 452 |
-
|
| 453 |
-
Range-->>Blue: injection pattern in web logs
|
| 454 |
-
Blue->>Range: patch /api/search, add WAF rule
|
| 455 |
-
Range-->>Blue: patch applied
|
| 456 |
|
| 457 |
-
|
| 458 |
-
Note over Range: Exploit FAILS, patch valid
|
| 459 |
|
| 460 |
-
|
| 461 |
-
```
|
| 462 |
|
| 463 |
-
|
| 464 |
|
| 465 |
-
|
| 466 |
|
| 467 |
-
|
|
| 468 |
-
|-----
|
| 469 |
-
| `
|
| 470 |
-
|
|
| 471 |
-
|
|
| 472 |
-
|
|
| 473 |
|
| 474 |
-
**
|
| 475 |
|
| 476 |
```python
|
| 477 |
from open_range.agents.episode import run_episode
|
|
@@ -482,109 +68,43 @@ env = RangeEnvironment()
|
|
| 482 |
red = LLMRangeAgent(model="anthropic/claude-sonnet-4-20250514")
|
| 483 |
blue = LLMRangeAgent(model="openai/gpt-4o")
|
| 484 |
result = run_episode(env, red, blue, max_steps=50)
|
| 485 |
-
print(result.outcome, result.metrics)
|
| 486 |
-
```
|
| 487 |
-
|
| 488 |
-
The `evaluate()` function in `agents/eval.py` runs N episodes and returns aggregate metrics (solve rate, detection rate, stealth, availability, false positive rate).
|
| 489 |
-
|
| 490 |
-
## Project Structure
|
| 491 |
-
|
| 492 |
```
|
| 493 |
-
open-range/
|
| 494 |
-
├── src/open_range/
|
| 495 |
-
│ ├── protocols.py Pydantic models: SnapshotSpec, TruthGraph, Vulnerability, FlagSpec, etc.
|
| 496 |
-
│ ├── resolve.py Dynamic component resolution (importlib + Protocol check)
|
| 497 |
-
│ ├── server/ FastAPI server (Environment, models, rewards)
|
| 498 |
-
│ │ ├── app.py FastAPI app factory (OpenEnv-compatible or standalone)
|
| 499 |
-
│ │ ├── __main__.py Entry point: python -m open_range.server
|
| 500 |
-
│ │ ├── environment.py RangeEnvironment with reset/step/state
|
| 501 |
-
│ │ ├── models.py RangeAction, RangeObservation, RangeState
|
| 502 |
-
│ │ └── rewards.py Reward components (flag, stealth, detection, patch, etc.)
|
| 503 |
-
│ ├── builder/ Snapshot builder + renderer
|
| 504 |
-
│ │ ├── builder.py LLMSnapshotBuilder, TemplateOnlyBuilder, FileBuilder
|
| 505 |
-
│ │ ├── renderer.py SnapshotRenderer: Jinja2 templates -> Docker artifacts
|
| 506 |
-
│ │ ├── mutator.py Vuln mutation logic (swap vulns between resets)
|
| 507 |
-
│ │ ├── snapshot_store.py Snapshot storage and retrieval
|
| 508 |
-
│ │ ├── templates/ Jinja2 templates (docker-compose, Dockerfiles, nginx, iptables, etc.)
|
| 509 |
-
│ │ └── npc/ NPC traffic system
|
| 510 |
-
│ │ ├── npc_manager.py NPCManager: orchestrates shell scripts + LLM agents
|
| 511 |
-
│ │ ├── npc_agent.py LLMNPCAgent (Level 1), RuleBasedNPCBehavior, NullNPCBehavior
|
| 512 |
-
│ │ ├── persona.py NPC persona model
|
| 513 |
-
│ │ └── *.sh Level 0 traffic scripts (http, db, ssh)
|
| 514 |
-
│ ├── validator/ 10-check admission pipeline
|
| 515 |
-
│ │ ├── validator.py Pipeline orchestrator
|
| 516 |
-
│ │ ├── build_boot.py Check 1: docker compose up + healthchecks
|
| 517 |
-
│ │ ├── exploitability.py Check 2: golden path end-to-end
|
| 518 |
-
│ │ ├── patchability.py Check 3: inverse mutation test
|
| 519 |
-
│ │ ├── evidence.py Check 4: logs + alerts exist
|
| 520 |
-
│ │ ├── reward_grounding.py Check 5: rubrics produce valid scores
|
| 521 |
-
│ │ ├── isolation.py Check 6: zones enforced, no leaks
|
| 522 |
-
│ │ ├── task_feasibility.py Check 7: tasks reference real hosts/services
|
| 523 |
-
│ │ ├── difficulty.py Check 8: golden path steps within tier target
|
| 524 |
-
│ │ ├── npc_consistency.py Check 9: NPC persona consistency (LLM, via litellm)
|
| 525 |
-
│ │ └── realism_review.py Check 10: scenario plausibility (LLM, advisory)
|
| 526 |
-
│ ├── agents/ Agent framework
|
| 527 |
-
│ │ ├── protocol.py RangeAgent protocol + EpisodeResult + EpisodeMetrics
|
| 528 |
-
│ │ ├── llm_agent.py LLMRangeAgent (litellm, any provider)
|
| 529 |
-
│ │ ├── scripted_agent.py ScriptedAgent + pre-built demo scripts
|
| 530 |
-
│ │ ├── human_agent.py Interactive human agent (stdin/stdout)
|
| 531 |
-
│ │ ├── prompts.py Red and Blue system prompts
|
| 532 |
-
│ │ ├── parsing.py Command extraction from LLM output
|
| 533 |
-
│ │ ├── episode.py run_episode() orchestration loop
|
| 534 |
-
│ │ └── eval.py evaluate() harness (N episodes, aggregate metrics)
|
| 535 |
-
│ ├── client/ Typed OpenEnv client (OpenRangeEnv)
|
| 536 |
-
│ └── training/ Training utilities (deferred -- env-first)
|
| 537 |
-
│ ├── trajectory.py TrajectoryLogger with JSONL export for SFT
|
| 538 |
-
│ ├── rollout.py Rollout function for GRPOTrainer
|
| 539 |
-
│ └── curriculum.py Curriculum escalation logic
|
| 540 |
-
├── manifests/ YAML range definitions (tier1, tier2, tier3) + schema
|
| 541 |
-
├── vulns/ Vulnerability catalog (sqli, xss, idor, ssrf, etc.)
|
| 542 |
-
├── examples/ Demo scripts
|
| 543 |
-
│ ├── demo.py End-to-end scripted demo (no Docker, no LLM)
|
| 544 |
-
│ └── demo_config.yaml Demo configuration
|
| 545 |
-
├── tests/ Test suite (13 test files)
|
| 546 |
-
├── docs/ Architecture docs and guides
|
| 547 |
-
└── pyproject.toml
|
| 548 |
-
```
|
| 549 |
-
|
| 550 |
-
## Trajectory Logging
|
| 551 |
-
|
| 552 |
-
The `TrajectoryLogger` records Red and Blue turns during episodes and exports them as JSONL in OpenAI chat format for supervised fine-tuning.
|
| 553 |
|
| 554 |
-
|
| 555 |
-
from open_range.training.trajectory import TrajectoryLogger
|
| 556 |
|
| 557 |
-
|
| 558 |
-
logger.start_episode("ep-001", snapshot_id="snap-001", tier=1)
|
| 559 |
-
logger.log_turn(role="red", observation="Range ready...", action="nmap -sV web", reward=0.1)
|
| 560 |
-
logger.end_episode(outcome="flag_captured")
|
| 561 |
-
logger.export_jsonl("trajectories.jsonl", reward_threshold=0.5)
|
| 562 |
-
```
|
| 563 |
|
| 564 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 565 |
|
| 566 |
-
##
|
| 567 |
|
| 568 |
-
|
| 569 |
-
|
| 570 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 571 |
|
| 572 |
-
|
| 573 |
-
uv run pytest tests/ -v --tb=short
|
| 574 |
|
| 575 |
-
#
|
| 576 |
-
uv run pytest tests/test_agents.py -v
|
| 577 |
-
uv run pytest tests/test_app.py -v
|
| 578 |
-
uv run pytest tests/test_validator.py -v
|
| 579 |
-
uv run pytest tests/test_demo.py -v
|
| 580 |
-
```
|
| 581 |
|
| 582 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 583 |
|
| 584 |
## Built On
|
| 585 |
|
| 586 |
-
- [OpenEnv](https://github.com/meta-pytorch/OpenEnv)
|
| 587 |
-
-
|
| 588 |
|
| 589 |
## License
|
| 590 |
|
|
|
|
| 1 |
# OpenRange
|
| 2 |
|
| 3 |
+
A multi-agent cybersecurity gymnasium on [OpenEnv](https://github.com/meta-pytorch/OpenEnv). Red and Blue agents train on validated enterprise networks that mutate between episodes.
|
|
|
|
|
|
|
| 4 |
|
| 5 |
---
|
| 6 |
|
| 7 |
+
## How It Works
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
+
A **manifest** declares a family of legal enterprise worlds — topology, services, identities, vulnerability classes, difficulty. A **Builder** LLM proposes a concrete snapshot within that family. A **Validator** pipeline admits only snapshots that are runnable, exploitable, patchable, and non-leaking. `reset()` selects a frozen validated snapshot. `step()` runs commands inside it.
|
|
|
|
|
|
|
| 10 |
|
| 11 |
```mermaid
|
| 12 |
flowchart LR
|
| 13 |
+
M[Manifest<br/>topology, services,<br/>bug families, difficulty] --> B[Builder<br/>LLM proposes<br/>snapshot]
|
| 14 |
+
B --> V{Validator<br/>10 checks}
|
| 15 |
+
V -->|fail| B
|
| 16 |
+
V -->|pass| S[Frozen Snapshot]
|
| 17 |
+
S --> E["reset() → step() → obs + reward"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
+
style V fill:#ffd93d,color:#333
|
| 20 |
+
style S fill:#6bcb77,color:#fff
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
```
|
| 22 |
|
| 23 |
+
Red and Blue operate on the **same infrastructure simultaneously**. Red's stealth reward depends on whether Blue catches them. Blue's detection reward depends on Red's actual actions in the logs. This coupling drives co-evolution.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
## Quick Start
|
| 26 |
|
| 27 |
```bash
|
| 28 |
+
git clone https://github.com/open-cybernauts/open-range.git && cd open-range
|
|
|
|
|
|
|
| 29 |
uv sync --all-extras
|
| 30 |
|
| 31 |
+
# End-to-end demo (no Docker, no LLM)
|
| 32 |
uv run python examples/demo.py
|
| 33 |
|
| 34 |
+
# Run the server
|
| 35 |
+
python -m open_range.server
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
# Tests
|
| 38 |
+
uv run pytest tests/ -v --tb=short
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
```
|
| 40 |
|
| 41 |
+
## Core Components
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
+
**Manifest** — YAML defining the legal world: hosts, zones, services, users, NPCs, data assets, credential policies, monitoring coverage, trust relationships, and which vulnerability classes the Builder may plant. Three example manifests ship (healthcare, fintech, SaaS) at tiers 1-3.
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
+
**Builder** — Takes a manifest + curriculum context, outputs a `SnapshotSpec`: topology graph, truth graph (planted vulns + exploit chain), evidence graph (what Blue can find), flags, golden path, NPC traffic, and task briefings. Three implementations: `LLMSnapshotBuilder` (production, via litellm), `TemplateOnlyBuilder` (deterministic, for tests), `FileBuilder` (load from disk).
|
|
|
|
| 46 |
|
| 47 |
+
**Validator** — 10-check admission pipeline. 8 mechanical checks (build/boot, exploitability, patchability, evidence sufficiency, reward grounding, isolation, task feasibility, difficulty calibration) + 2 LLM advisory checks (NPC consistency, realism review). Inverse mutation: patching each planted vuln must break its exploit step.
|
|
|
|
| 48 |
|
| 49 |
+
**Environment** — `RangeEnvironment(Environment)` following the OpenEnv contract. `reset()` picks a frozen snapshot + samples a task. `step(action)` routes commands to the appropriate container — Red runs on the attacker box, Blue runs on the SIEM. No artificial command allowlists; the container's installed tools are the constraint.
|
| 50 |
|
| 51 |
+
**Rewards** — All grounded in container state, not LLM judgment:
|
| 52 |
|
| 53 |
+
| Red | Blue |
|
| 54 |
+
|-----|------|
|
| 55 |
+
| Flag capture (binary, `docker exec cat`) | Detection (TP rate vs Red's log) |
|
| 56 |
+
| Efficiency (`gamma^steps`) | Patch validity (re-run exploit, must fail) |
|
| 57 |
+
| Stealth (inversely coupled to Blue detection) | Availability (healthcheck fraction) |
|
| 58 |
+
| Anti-hallucination (-0.3 per fake flag) | False positive penalty (-0.2 per NPC flagged) |
|
| 59 |
|
| 60 |
+
**Agents** — Structural protocol: any object with `reset(briefing, role)` and `act(observation) -> command` works. Ships with `LLMRangeAgent` (litellm, any provider), `ScriptedAgent`, and `HumanAgent`.
|
| 61 |
|
| 62 |
```python
|
| 63 |
from open_range.agents.episode import run_episode
|
|
|
|
| 68 |
red = LLMRangeAgent(model="anthropic/claude-sonnet-4-20250514")
|
| 69 |
blue = LLMRangeAgent(model="openai/gpt-4o")
|
| 70 |
result = run_episode(env, red, blue, max_steps=50)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
|
| 73 |
+
## Tier System
|
|
|
|
| 74 |
|
| 75 |
+
Difficulty grows horizontally — more hosts, zones, and chained attack surface. Not just harder passwords.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
|
| 77 |
+
| Tier | Scale | Example |
|
| 78 |
+
|------|-------|---------|
|
| 79 |
+
| 1 | 6-8 hosts, 3-4 zones | Healthcare clinic: web + DB + mail + LDAP + SIEM |
|
| 80 |
+
| 2 | 10-12 hosts, 5-6 zones | Financial firm: + VPN, internal APIs, certificate authority |
|
| 81 |
+
| 3 | 14-18 hosts, 7-8 zones | SaaS company: + CI/CD, container registry, partner extranet |
|
| 82 |
|
| 83 |
+
## Server Endpoints
|
| 84 |
|
| 85 |
+
| Method | Path | Description |
|
| 86 |
+
|--------|------|-------------|
|
| 87 |
+
| GET | `/health` | Liveness check |
|
| 88 |
+
| GET | `/metadata` | Environment name, version |
|
| 89 |
+
| POST | `/reset` | Start episode, returns initial observation |
|
| 90 |
+
| POST | `/step` | Execute action, returns observation + reward |
|
| 91 |
+
| GET | `/state` | Current episode state |
|
| 92 |
+
| WS | `/ws` | WebSocket session |
|
| 93 |
|
| 94 |
+
Compatible with `openenv` when installed; standalone FastAPI fallback otherwise.
|
|
|
|
| 95 |
|
| 96 |
+
## Docs
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
|
| 98 |
+
- [Architecture](docs/architecture.md) — full pipeline, network topology, episode lifecycle
|
| 99 |
+
- [Builder & Validator](docs/builder-validator.md) — snapshot generation and admission
|
| 100 |
+
- [Red & Blue Agents](docs/red-blue-agents.md) — tandem training, reward coupling, curriculum
|
| 101 |
+
- [Agent Protocols](docs/agent-protocols.md) — agent interface, episode runner, evaluation
|
| 102 |
+
- [OpenEnv Compliance](docs/openenv-compliance.md) — API contract, models, deployment
|
| 103 |
|
| 104 |
## Built On
|
| 105 |
|
| 106 |
+
- [OpenEnv](https://github.com/meta-pytorch/OpenEnv) — standardized agentic execution environments
|
| 107 |
+
- Ideas from [R2E-Gym](https://arxiv.org/abs/2504.07164) (hybrid verification), [Self-Play SWE-RL](https://arxiv.org/abs/2512.18552) (formal specs, inverse mutation), PAIRED/UED (constrained generation), POET (mutate + admit)
|
| 108 |
|
| 109 |
## License
|
| 110 |
|