Aaron Brown Claude Opus 4.6 commited on
Commit
4a77f25
·
1 Parent(s): bfb2e34

Simplify README: 592 → 121 lines, 6 diagrams → 1

Browse files

Strip to what matters for someone landing on the repo: single pipeline
diagram, concise component descriptions, quick start, link to docs/
for deep dives. Remove duplicated explanations, implementation details,
and 60-line project tree.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Files changed (1) hide show
  1. README.md +55 -535
README.md CHANGED
@@ -1,477 +1,63 @@
1
  # OpenRange
2
 
3
- **Multi-agent cyber range with zero-sum Red/Blue dynamics, validated company snapshots, and self-improving enterprise worlds.**
4
-
5
- The first cybersecurity environment in the [OpenEnv](https://github.com/meta-pytorch/OpenEnv) ecosystem.
6
 
7
  ---
8
 
9
- ## What is this?
10
-
11
- OpenRange drops Red and Blue agents into a **real enterprise network** -- firewalls, web apps, databases, directory services, mail servers, VPNs, SIEM -- then lets them fight. The environment is not a single static benchmark and it is not a free-form LLM sandbox. A manifest defines a legal family of company worlds. A LiteLLM-led builder/mutator proposes candidate snapshots inside that family. Every proposal is compiled into a canonical `SnapshotSpec` (a typed snapshot specification for that company world) plus hidden topology, truth, evidence, and task graphs. Deterministic helper checks make those proposals admissible. `reset()` then selects a **frozen validated snapshot** for the next episode, while background mutation prepares future snapshots asynchronously.
12
-
13
- ```
14
- You define the legal company family:
15
- topology, identities, services, bug families, task families, difficulty knobs
16
-
17
- The LiteLLM builder/mutator proposes a candidate snapshot:
18
- add billing-api -> seed SSRF -> derive exploit/remediation chain -> emit evidence
19
-
20
- The proposal compiles into a canonical SnapshotSpec + hidden graphs:
21
- topology graph + truth graph + evidence graph + task graph
22
-
23
- Deterministic helper-backed validation admits only runnable snapshots:
24
- manifest compliance, reachability, exploitability, patchability,
25
- evidence sufficiency, reward grounding, isolation/leakage
26
-
27
- The OpenEnv runtime stays standard:
28
- reset() -> pick frozen snapshot + sample task
29
- step(action) -> act inside that snapshot
30
- ```
31
-
32
- ## Core Components
33
-
34
- | Component | What it does | Typical implementation |
35
- |------|-------------|-------------|
36
- | **Manifest compiler** | Defines the legal world space: topology, services, identities, bug families, task families, difficulty knobs | YAML schema + templates |
37
- | **Builder / mutator** | Uses LiteLLM to propose candidate snapshots, mutations, and task structure inside the manifest-constrained family | LiteLLM + rules + templates |
38
- | **Canonical `SnapshotSpec`** | Compiles the proposal into typed hidden truth: topology, truth, evidence, and task graphs | Pydantic models + graph structs |
39
- | **Deterministic helpers** | Answer the specific admission questions: compliance, solvability, exploitability, patchability, evidence, reward grounding | Mechanical check modules |
40
- | **Validator gate** | Combines helper outputs and admits only snapshots that are runnable, coherent, and solvable | Mechanical admission over graph/spec + rendered artifacts |
41
- | **Snapshot manager** | Publishes admitted company snapshots and hands a frozen one to `reset()` | Background queue + snapshot store |
42
- | **Red** | External attacker. Recon, exploit, pivot, escalate, exfiltrate. | Outside the firewall -- no creds, no access |
43
- | **Blue** | Internal defender. SIEM analysis, patching, firewall rules, incident response. | SOC workstation on management network |
44
 
45
- Red and Blue operate on the **same infrastructure simultaneously** in a zero-sum adversarial dynamic. Red's stealth reward depends on whether Blue catches them. Blue's detection reward depends on Red's actual actions in the logs. This multi-agent coupling creates natural co-evolution: as Red learns stealth, Blue must learn deeper detection -- and vice versa.
46
-
47
- ## Architecture
48
 
49
  ```mermaid
50
  flowchart LR
51
- A[Base company family<br/>AcmeCorp repo, infra, docs, tools] --> B
52
- M[Manifest / mutation policy<br/>allowed services, bug families, task families] --> B
53
- S[Curriculum / failure stats<br/>what Red or Blue is weak at] --> B
54
-
55
- B[LiteLLM builder / mutator<br/>propose next snapshot<br/>mutations, bugs, tasks] --> C
56
-
57
- subgraph C[Canonical SnapshotSpec and hidden graphs]
58
- C1[Topology graph<br/>hosts, services, users, trust edges]
59
- C2[Truth graph<br/>bug, exploit chain, blast radius, remediation]
60
- C3[Evidence graph<br/>logs, alerts, files, tickets, docs]
61
- C4[Task graph<br/>exploit, investigate, patch, report]
62
- end
63
-
64
- C --> D
65
- D{Validator gate<br/>build/run, exploitability,<br/>patchability, evidence, reward} -->|fail| B
66
- D -->|pass| E[Frozen validated snapshot<br/>Acme v_k]
67
-
68
- subgraph R[OpenEnv runtime]
69
- F["reset()<br/>select frozen snapshot + sample task + init episode"]
70
- G[Red / Blue agents]
71
- H["step(action)<br/>run command or tool on current frozen snapshot"]
72
- I[Observation + reward]
73
- J[Rollout results<br/>solve rate, evidence quality, patch validity]
74
- end
75
-
76
- E --> F
77
- F --> G
78
- G --> H
79
- H --> I
80
- I --> G
81
- H --> J
82
-
83
- J -. async evolve next snapshot .-> S
84
-
85
- style A fill:#4a9eff,color:#fff
86
- style M fill:#4a9eff,color:#fff
87
- style B fill:#ff6b6b,color:#fff
88
- style D fill:#ffd93d,color:#333
89
- style E fill:#6bcb77,color:#fff
90
- style R fill:#7c73e611,stroke:#7c73e6
91
- ```
92
-
93
- The generator here is a **LiteLLM-led proposal pipeline** rather than a free-form oracle. The model proposes the world, the canonical graph/spec makes it legible, and deterministic helpers make it admissible. That keeps core world logic manifest-constrained and validator-checkable even when the builder uses model-generated code, docs, tickets, or alert text.
94
-
95
- Serving stays OpenEnv/Hugging Face-friendly: the deployed app exposes the normal `reset()`, `step()`, and `state` contract, and admitted snapshots can be served without live model calls in the request path. When packaged for deployment, that contract can be wrapped with the required OpenEnv/HF metadata.
96
-
97
- ## Network Topology
98
-
99
- Even the **basic** range emulates a real corporate network. Every tier is a functioning enterprise with interconnected services, proper network segmentation, and realistic traffic.
100
 
101
- ```mermaid
102
- flowchart TB
103
- subgraph internet [Internet]
104
- ATK[Red Agent<br/>Attacker Workstation]
105
- end
106
-
107
- subgraph fw [Perimeter Firewall - iptables]
108
- FW1[Firewall<br/>NAT + ACLs + IDS]
109
- end
110
-
111
- subgraph dmz [DMZ Network - 10.0.1.0/24]
112
- WEB[Web Server<br/>nginx reverse proxy<br/>+ PHP/Python app]
113
- MAIL[Mail Server<br/>Postfix SMTP<br/>+ Dovecot IMAP]
114
- DNS[DNS Server<br/>Bind9<br/>corp.local zone]
115
- end
116
-
117
- subgraph internal [Internal Network - 10.0.2.0/24]
118
- DB[Database Server<br/>MySQL + PostgreSQL<br/>app data + credentials]
119
- FILES[File Server<br/>Samba SMB shares<br/>sensitive docs + configs]
120
- APP[App Server<br/>Internal APIs<br/>microservices]
121
- end
122
-
123
- subgraph mgmt [Management Network - 10.0.3.0/24]
124
- AD[Domain Controller<br/>OpenLDAP + Kerberos<br/>Active Directory]
125
- SIEM[SIEM + Log Server<br/>Rsyslog + ELK<br/>Blue agent entry point]
126
- JUMP[Jump Box<br/>SSH bastion<br/>admin access only]
127
- end
128
-
129
- ATK -->|ports 80,443,25| FW1
130
- FW1 --> WEB
131
- FW1 --> MAIL
132
- FW1 --> DNS
133
- WEB -->|SQL queries| DB
134
- WEB -->|LDAP auth| AD
135
- MAIL -->|user lookup| AD
136
- APP -->|file access| FILES
137
- APP -->|DB queries| DB
138
- FILES -->|auth| AD
139
- DB -->|logs| SIEM
140
- WEB -->|logs| SIEM
141
- MAIL -->|logs| SIEM
142
- AD -->|logs| SIEM
143
- JUMP -->|admin SSH| WEB
144
- JUMP -->|admin SSH| DB
145
-
146
- style internet fill:#ff6b6b22,stroke:#ff6b6b
147
- style fw fill:#ffd93d22,stroke:#ffd93d
148
- style dmz fill:#4a9eff22,stroke:#4a9eff
149
- style internal fill:#6bcb7722,stroke:#6bcb77
150
- style mgmt fill:#7c73e622,stroke:#7c73e6
151
  ```
152
 
153
- **This is what Red has to break into. This is what Blue has to defend.**
154
-
155
- Every service is real. The web app queries the database. Users authenticate against LDAP. Mail flows through Postfix. Logs stream to the SIEM. NPC traffic simulates employees browsing, sending email, and running cron jobs -- so Blue can't just flag everything as malicious.
156
-
157
- NPCs evolve from shell-script noise generators to **LLM-driven simulated experts** -- employees with persona cards, susceptibility profiles, and realistic communication styles. These are domain-specialized LLM agents (marketing coordinator, CISO, IT admin) that generate authentic enterprise behavior: sending emails, filing tickets, browsing intranet, and responding to social engineering attempts based on their security awareness level. Red can craft spearphishing emails, pretext calls, and watering-hole attacks against NPCs who decide whether to click, ignore, or report. Blue must detect these social engineering campaigns in logs alongside normal NPC traffic.
158
-
159
- ## Episode Lifecycle
160
-
161
- ```mermaid
162
- sequenceDiagram
163
- participant W as Background Mutator
164
- participant V as Validator
165
- participant S as Snapshot Store
166
- participant T as Training Loop
167
- participant E as OpenEnv Server
168
- participant C as Frozen Company Snapshot
169
-
170
- W->>W: Apply legal mutations from manifest
171
- W->>W: Seed bug chain, evidence, and tasks
172
- W->>V: Candidate snapshot + truth graph
173
- V->>V: Build/run, exploitability, patchability, reward checks
174
- alt PASS
175
- V->>S: Publish Acme v_k
176
- else FAIL
177
- V-->>W: Retry with failure context
178
- end
179
-
180
- T->>E: reset()
181
- E->>S: Select validated snapshot + task
182
- S-->>E: Frozen snapshot Acme v_k
183
- E-->>T: RangeObservation with task briefing
184
-
185
- rect rgb(255, 107, 107, 0.1)
186
- Note over T,C: Red Team Operations on the frozen snapshot
187
- T->>E: step Red: nmap perimeter scan
188
- E->>C: docker exec attacker nmap -sV fw
189
- C-->>E: 80, 443, 25 open
190
- E-->>T: observation + reward
191
-
192
- T->>E: step Red: enumerate web app
193
- E->>C: docker exec attacker nikto web
194
- C-->>E: discovered /admin, /api, /search
195
- E-->>T: observation + reward
196
-
197
- T->>E: step Red: exploit SQLi in search
198
- E->>C: docker exec attacker curl ...
199
- C-->>E: DB credentials leaked
200
- E-->>T: observation + reward
201
-
202
- T->>E: step Red: pivot to internal DB
203
- E->>C: docker exec attacker mysql -h db ...
204
- C-->>E: flag captured from flags table
205
- E-->>T: observation + flag reward
206
- end
207
-
208
- rect rgb(74, 158, 255, 0.1)
209
- Note over T,C: Blue Team Operations on the same frozen snapshot
210
- T->>E: step Blue: check SIEM alerts
211
- E->>C: docker exec siem tail alerts
212
- C-->>E: anomalous queries from web to db
213
- E-->>T: observation + reward
214
-
215
- T->>E: step Blue: analyze attack pattern
216
- E->>C: docker exec siem grep SQLi signatures
217
- C-->>E: injection pattern matched
218
- E-->>T: observation + detection reward
219
-
220
- T->>E: step Blue: patch and block
221
- E->>C: docker exec web parameterize query
222
- C-->>E: patch applied, firewall rule added
223
- E-->>T: observation + patch reward
224
- end
225
-
226
- Note over W,S: Background mutation affects future resets only
227
- Note over T,C: Rewards computed from container state and action logs
228
- ```
229
-
230
- ## Episodes vs Evolution
231
-
232
- `reset()` does **not** rebuild the world in the hot path. It selects a prevalidated company snapshot and starts a new task session on that frozen state. Mutation and admission happen between episodes, so OpenRange stays compatible with the normal OpenEnv `reset()`, `step()`, and `state()` contract without collapsing into a static benchmark.
233
-
234
- ```mermaid
235
- flowchart LR
236
- subgraph ep1 [Episode A]
237
- direction TB
238
- A1["reset() selects Acme v12"] --> B1[Red and Blue act inside frozen snapshot]
239
- end
240
-
241
- subgraph bg [Between Episodes]
242
- direction TB
243
- M1[Mutator proposes Acme v13] --> M2{Validator gate}
244
- M2 -->|fail| M1
245
- M2 -->|pass| M3[Publish Acme v13]
246
- end
247
-
248
- subgraph ep2 [Episode B]
249
- direction TB
250
- A2["future reset() selects Acme v13"] --> B2[New task on next frozen snapshot]
251
- end
252
-
253
- ep1 -->|episode ends| bg
254
- bg -->|"next reset()"| ep2
255
-
256
- style ep1 fill:#ff6b6b22,stroke:#ff6b6b
257
- style bg fill:#ffd93d22,stroke:#ffd93d
258
- style ep2 fill:#6bcb7722,stroke:#6bcb77
259
- ```
260
-
261
- Agents still have to **generalize** across vulnerability classes, pivot chains, evidence patterns, and remediation paths -- but each episode remains coherent because the active world is frozen while the agent interacts with it.
262
 
263
  ## Quick Start
264
 
265
  ```bash
266
- # Install
267
- git clone https://github.com/open-cybernauts/open-range.git
268
- cd open-range
269
  uv sync --all-extras
270
 
271
- # Run the end-to-end demo (no Docker, no LLM required)
272
  uv run python examples/demo.py
273
 
274
- # Run the FastAPI server
275
- python -m open_range.server # default: 127.0.0.1:8000
276
- python -m open_range.server --port 9000 # custom port
277
- python -m open_range.server --host 0.0.0.0 # bind all interfaces
278
-
279
- # Or via uvicorn directly
280
- uv run uvicorn open_range.server.app:app --host 0.0.0.0 --port 8000 --reload
281
- ```
282
-
283
- ### Server Endpoints
284
-
285
- | Method | Path | Description |
286
- |--------|------|-------------|
287
- | GET | `/health` | Liveness check |
288
- | GET | `/metadata` | Environment name, version, description |
289
- | GET | `/schema` | JSON schemas for action, observation, state |
290
- | POST | `/reset` | Reset environment, returns initial observation |
291
- | POST | `/step` | Execute an action, returns observation + reward + done |
292
- | GET | `/state` | Current episode state |
293
- | WS | `/ws` | Persistent WebSocket session (per-connection environment) |
294
-
295
- If `openenv` is installed, the server delegates to `openenv.core.env_server.create_app`. Otherwise it falls back to an equivalent standalone FastAPI app.
296
-
297
- ## Reward Signals
298
-
299
- Episodes are **long-horizon** (8-50+ steps depending on tier) with **sparse delayed rewards**. Flag capture is binary and only fires at the end of a successful exploit chain. Stealth and detection rewards are computed at episode end from the full action log. Intermediate steps yield only small efficiency signals -- agents must learn to plan multi-step strategies without dense per-action feedback.
300
-
301
- All rewards are **verifiable** -- grounded in real container state, not LLM judgment. Reward ceilings **scale with environment complexity**: higher-tier snapshots (more hosts, zones, and chained vulnerabilities) offer proportionally larger maximum rewards, ensuring the training signal grows with output quality.
302
-
303
- ```mermaid
304
- flowchart TB
305
- subgraph red [Red Rewards]
306
- RF[Flag Capture<br/>docker exec cat flag<br/>binary match]
307
- RE[Efficiency<br/>gamma^steps]
308
- RS[Stealth<br/>Did Blue detect?]
309
- RSE[Social Engineering<br/>NPC fell for phish/pretext]
310
- RH[Anti-hallucination<br/>-0.3 per fake flag]
311
- end
312
-
313
- subgraph blue [Blue Rewards]
314
- BD[Detection<br/>TP rate vs Red log]
315
- BP[Patch<br/>Golden path re-run fails]
316
- BA[Availability<br/>Healthcheck fraction]
317
- BPH[Phishing Detection<br/>social engineering caught in logs]
318
- BF[False Positive<br/>-0.2 per NPC flagged]
319
- end
320
-
321
- subgraph coupling [Coupling]
322
- RS -.-|depends on| BD
323
- BD -.-|depends on| RF
324
- end
325
-
326
- style red fill:#ff6b6b11,stroke:#ff6b6b
327
- style blue fill:#4a9eff11,stroke:#4a9eff
328
- style coupling fill:#ffd93d11,stroke:#ffd93d,stroke-dasharray: 5 5
329
- ```
330
-
331
- ## Validation Gate
332
-
333
- Every candidate snapshot passes an **executable admission pipeline** before any agent touches it. The validator does not use free-form LLM prose as ground truth; it operates over the compiled `SnapshotSpec` plus rendered runtime artifacts. Mechanical checks are primary. An optional Validator LLM may review structured specs or artifacts for realism, but its feedback is secondary critique rather than ground truth.
334
-
335
- ```mermaid
336
- flowchart LR
337
- S1[1. Build + boot<br/>services start and healthchecks pass] --> S2[2. Exploitability<br/>truth path or golden path works]
338
- S2 --> S3[3. Patchability<br/>fix or revert breaks the exploit path]
339
- S3 --> S4[4. Evidence sufficiency<br/>logs, files, tickets support investigation]
340
- S4 --> S5[5. Reward check<br/>rubrics grounded in container state]
341
- S5 --> S6[6. Isolation + leakage<br/>no impossible refs or answer leaks]
342
-
343
- S6 -->|All pass| PASS[ADMIT SNAPSHOT]
344
- S6 -->|Any fail| FAIL[REJECT + RETRY]
345
-
346
- style PASS fill:#6bcb77,color:#fff
347
- style FAIL fill:#ff6b6b,color:#fff
348
- style S3 fill:#ffd93d,color:#333
349
- ```
350
-
351
- Inverse mutation still matters here: if reverting or patching the planted bug does not break the exploit path, the vulnerability is decorative and the snapshot should be rejected.
352
-
353
- ## Tier System
354
-
355
- Every tier is a **complete enterprise network**. Difficulty grows by adding business units, network zones, and attack surface -- not just harder passwords.
356
-
357
- | Tier | Hosts | Zones | Key Infrastructure | Attack Complexity |
358
- |------|-------|-------|-------------------|-------------------|
359
- | 1 | 6-8 | DMZ, Internal, Mgmt | Web app + DB + mail + firewall + LDAP + SIEM | Single-stage: exploit web, grab flag |
360
- | 2 | 10-12 | + VPN, Guest | + VPN gateway, guest WiFi segment, internal APIs, certificate authority | Multi-stage: exploit + pivot one hop |
361
- | 3 | 14-18 | + Partner, Dev | + CI/CD pipeline, container registry, partner extranet, S3-like storage | Chain 2-3 vulns across zones |
362
- | 4 | 20-25 | + OT/SCADA, Cloud | + Industrial control sim, cloud gateway, secrets vault, service mesh | Lateral movement across trust boundaries |
363
- | 5 | 30+ | Full enterprise | + Honeypots, deception tech, WAF, IDS/IPS, EDR, threat intel | Evade active defenses while chaining |
364
-
365
- ```mermaid
366
- flowchart TD
367
- subgraph t1 [Tier 1 - Small Business]
368
- direction LR
369
- FW1[Firewall] --> W1[Web + Mail]
370
- W1 --> D1[DB + Files]
371
- D1 --> AD1[LDAP + SIEM]
372
- end
373
-
374
- subgraph t2 [Tier 2 - Mid-Market]
375
- direction LR
376
- FW2[Firewall + VPN] --> W2[Web + Mail + DNS]
377
- W2 --> D2[DB + APIs + Files]
378
- D2 --> AD2[AD + CA + SIEM]
379
- end
380
-
381
- subgraph t3 [Tier 3 - Enterprise]
382
- direction LR
383
- FW3[Firewall + WAF + IDS] --> W3[Web + Mail + DNS + CDN]
384
- W3 --> D3[DB + APIs + CI/CD + Registry]
385
- D3 --> AD3[AD + Kerberos + Vault + SIEM]
386
- end
387
-
388
- t1 -->|agent masters tier| t2
389
- t2 -->|agent masters tier| t3
390
-
391
- style t1 fill:#6bcb7722,stroke:#6bcb77
392
- style t2 fill:#ffd93d22,stroke:#ffd93d
393
- style t3 fill:#ff6b6b22,stroke:#ff6b6b
394
- ```
395
-
396
- ## Curriculum Feedback Loop
397
 
398
- OpenRange is **self-improving**. Per-snapshot solve rates and detection rates feed back to the Builder, which adjusts the next snapshot's difficulty and vulnerability mix to target the frontier of agent capability.
399
-
400
- ```
401
- Episode results (solve rate, detection rate, time-to-flag)
402
- |
403
- v
404
- Curriculum tracker (per vuln class, per tier)
405
- |
406
- v
407
- Builder receives runtime_context:
408
- { red_solve_rate: 0.6, blue_detect_rate: 0.4,
409
- previous_vuln_classes: [sqli, weak_creds],
410
- weak_areas: [ssrf, chained_vulns] }
411
- |
412
- v
413
- Next snapshot targets agent weaknesses:
414
- - If Red solves SQLi easily → seed SSRF or chained vulns
415
- - If Blue misses lateral movement → add more pivot points
416
- - Difficulty adjusts via r_inject = 1 - (1+α)·s
417
  ```
418
 
419
- The Builder LLM acts as a **simulated expert curriculum designer** -- it doesn't just randomize, it analyzes agent performance and generates challenges calibrated to the learning frontier. This is the same frontier-calibrating reward from Self-Play SWE-RL, adapted for cybersecurity.
420
-
421
- ## Tandem Red + Blue Training
422
-
423
- ```mermaid
424
- sequenceDiagram
425
- participant Red as Red Agent
426
- participant Range as Enterprise Range
427
- participant Blue as Blue Agent
428
-
429
- Note over Red,Blue: Episode begins - reset() selected a frozen validated snapshot
430
-
431
- Red->>Range: nmap perimeter scan
432
- Range-->>Red: firewall: 80,443,25 open
433
- Note right of Range: Logged to SIEM
434
-
435
- Blue->>Range: check SIEM dashboard
436
- Range-->>Blue: NPC traffic + Red scan mixed in
437
- Blue->>Range: submit_finding port scan from ext IP
438
- Note left of Blue: True positive
439
-
440
- Red->>Range: enumerate web app directories
441
- Range-->>Red: found /admin /api /uploads
442
- Note right of Range: Logged to SIEM
443
-
444
- Red->>Range: exploit SQLi in /api/search
445
- Range-->>Red: DB creds leaked
446
- Red->>Range: pivot to internal DB with stolen creds
447
- Range-->>Red: connected, flag captured
448
-
449
- Red->>Range: submit_flag FLAG_db_compromised
450
- Range-->>Red: Correct, reward 1.0
451
 
452
- Blue->>Range: analyze SIEM for SQLi signatures
453
- Range-->>Blue: injection pattern in web logs
454
- Blue->>Range: patch /api/search, add WAF rule
455
- Range-->>Blue: patch applied
456
 
457
- Note over Range: Re-run golden path exploit
458
- Note over Range: Exploit FAILS, patch valid
459
 
460
- Note over Red,Blue: Red stealth LOW - Blue caught the attack<br/>Blue detection HIGH - found real intrusion
461
- ```
462
 
463
- ## Agents
464
 
465
- OpenRange uses a **structural protocol** for agents -- any object with `reset(briefing, role)` and `act(observation) -> command` methods works. No base class required.
466
 
467
- | Agent | Module | Description |
468
- |-------|--------|-------------|
469
- | `RangeAgent` | `agents/protocol.py` | Protocol definition (structural subtyping) |
470
- | `LLMRangeAgent` | `agents/llm_agent.py` | LLM-powered agent via LiteLLM (any provider: Anthropic, OpenAI, Ollama, vLLM, etc.) |
471
- | `ScriptedAgent` | `agents/scripted_agent.py` | Replays a fixed command list (for testing and demos) |
472
- | `HumanAgent` | `agents/human_agent.py` | Interactive stdin/stdout agent for manual play |
473
 
474
- **Bring your own agent**: implement `reset()` and `act()` and pass it to `run_episode()` or `evaluate()`.
475
 
476
  ```python
477
  from open_range.agents.episode import run_episode
@@ -482,109 +68,43 @@ env = RangeEnvironment()
482
  red = LLMRangeAgent(model="anthropic/claude-sonnet-4-20250514")
483
  blue = LLMRangeAgent(model="openai/gpt-4o")
484
  result = run_episode(env, red, blue, max_steps=50)
485
- print(result.outcome, result.metrics)
486
- ```
487
-
488
- The `evaluate()` function in `agents/eval.py` runs N episodes and returns aggregate metrics (solve rate, detection rate, stealth, availability, false positive rate).
489
-
490
- ## Project Structure
491
-
492
  ```
493
- open-range/
494
- ├── src/open_range/
495
- │ ├── protocols.py Pydantic models: SnapshotSpec, TruthGraph, Vulnerability, FlagSpec, etc.
496
- │ ├── resolve.py Dynamic component resolution (importlib + Protocol check)
497
- │ ├── server/ FastAPI server (Environment, models, rewards)
498
- │ │ ├── app.py FastAPI app factory (OpenEnv-compatible or standalone)
499
- │ │ ├── __main__.py Entry point: python -m open_range.server
500
- │ │ ├── environment.py RangeEnvironment with reset/step/state
501
- │ │ ├── models.py RangeAction, RangeObservation, RangeState
502
- │ │ └── rewards.py Reward components (flag, stealth, detection, patch, etc.)
503
- │ ├── builder/ Snapshot builder + renderer
504
- │ │ ├── builder.py LLMSnapshotBuilder, TemplateOnlyBuilder, FileBuilder
505
- │ │ ├── renderer.py SnapshotRenderer: Jinja2 templates -> Docker artifacts
506
- │ │ ├── mutator.py Vuln mutation logic (swap vulns between resets)
507
- │ │ ├── snapshot_store.py Snapshot storage and retrieval
508
- │ │ ├── templates/ Jinja2 templates (docker-compose, Dockerfiles, nginx, iptables, etc.)
509
- │ │ └── npc/ NPC traffic system
510
- │ │ ├── npc_manager.py NPCManager: orchestrates shell scripts + LLM agents
511
- │ │ ├── npc_agent.py LLMNPCAgent (Level 1), RuleBasedNPCBehavior, NullNPCBehavior
512
- │ │ ├── persona.py NPC persona model
513
- │ │ └── *.sh Level 0 traffic scripts (http, db, ssh)
514
- │ ├── validator/ 10-check admission pipeline
515
- │ │ ├── validator.py Pipeline orchestrator
516
- │ │ ├── build_boot.py Check 1: docker compose up + healthchecks
517
- │ │ ├── exploitability.py Check 2: golden path end-to-end
518
- │ │ ├── patchability.py Check 3: inverse mutation test
519
- │ │ ├── evidence.py Check 4: logs + alerts exist
520
- │ │ ├── reward_grounding.py Check 5: rubrics produce valid scores
521
- │ │ ├── isolation.py Check 6: zones enforced, no leaks
522
- │ │ ├── task_feasibility.py Check 7: tasks reference real hosts/services
523
- │ │ ├── difficulty.py Check 8: golden path steps within tier target
524
- │ │ ├── npc_consistency.py Check 9: NPC persona consistency (LLM, via litellm)
525
- │ │ └── realism_review.py Check 10: scenario plausibility (LLM, advisory)
526
- │ ├── agents/ Agent framework
527
- │ │ ├── protocol.py RangeAgent protocol + EpisodeResult + EpisodeMetrics
528
- │ │ ├── llm_agent.py LLMRangeAgent (litellm, any provider)
529
- │ │ ├── scripted_agent.py ScriptedAgent + pre-built demo scripts
530
- │ │ ├── human_agent.py Interactive human agent (stdin/stdout)
531
- │ │ ├── prompts.py Red and Blue system prompts
532
- │ │ ├── parsing.py Command extraction from LLM output
533
- │ │ ├── episode.py run_episode() orchestration loop
534
- │ │ └── eval.py evaluate() harness (N episodes, aggregate metrics)
535
- │ ├── client/ Typed OpenEnv client (OpenRangeEnv)
536
- │ └── training/ Training utilities (deferred -- env-first)
537
- │ ├── trajectory.py TrajectoryLogger with JSONL export for SFT
538
- │ ├── rollout.py Rollout function for GRPOTrainer
539
- │ └── curriculum.py Curriculum escalation logic
540
- ├── manifests/ YAML range definitions (tier1, tier2, tier3) + schema
541
- ├── vulns/ Vulnerability catalog (sqli, xss, idor, ssrf, etc.)
542
- ├── examples/ Demo scripts
543
- │ ├── demo.py End-to-end scripted demo (no Docker, no LLM)
544
- │ └── demo_config.yaml Demo configuration
545
- ├── tests/ Test suite (13 test files)
546
- ├── docs/ Architecture docs and guides
547
- └── pyproject.toml
548
- ```
549
-
550
- ## Trajectory Logging
551
-
552
- The `TrajectoryLogger` records Red and Blue turns during episodes and exports them as JSONL in OpenAI chat format for supervised fine-tuning.
553
 
554
- ```python
555
- from open_range.training.trajectory import TrajectoryLogger
556
 
557
- logger = TrajectoryLogger()
558
- logger.start_episode("ep-001", snapshot_id="snap-001", tier=1)
559
- logger.log_turn(role="red", observation="Range ready...", action="nmap -sV web", reward=0.1)
560
- logger.end_episode(outcome="flag_captured")
561
- logger.export_jsonl("trajectories.jsonl", reward_threshold=0.5)
562
- ```
563
 
564
- Red and Blue trajectories are written as separate JSONL lines (independent training examples). Episodes can be filtered by reward threshold.
 
 
 
 
565
 
566
- ## Running Tests
567
 
568
- ```bash
569
- # Install dev dependencies
570
- uv sync --all-extras
 
 
 
 
 
571
 
572
- # Run all tests
573
- uv run pytest tests/ -v --tb=short
574
 
575
- # Run specific test files
576
- uv run pytest tests/test_agents.py -v
577
- uv run pytest tests/test_app.py -v
578
- uv run pytest tests/test_validator.py -v
579
- uv run pytest tests/test_demo.py -v
580
- ```
581
 
582
- Test files cover: agents, app/server endpoints, builder, demo, environment, manifests, models, protocols, renderer, rewards, trajectory logging, and the validator pipeline.
 
 
 
 
583
 
584
  ## Built On
585
 
586
- - [OpenEnv](https://github.com/meta-pytorch/OpenEnv) -- standardized agentic execution environments
587
- - Design ideas from PAIRED / UED (generate inside a legal family), POET (mutate plus admit), [R2E-Gym](https://arxiv.org/abs/2504.07164) (executable verification), [Self-Play SWE-RL](https://arxiv.org/abs/2512.18552) (formal specs and inverse mutation testing), and [Snorkel](https://www.snorkel.ai/) (simulated domain experts for data generation)
588
 
589
  ## License
590
 
 
1
  # OpenRange
2
 
3
+ A multi-agent cybersecurity gymnasium on [OpenEnv](https://github.com/meta-pytorch/OpenEnv). Red and Blue agents train on validated enterprise networks that mutate between episodes.
 
 
4
 
5
  ---
6
 
7
+ ## How It Works
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
 
9
+ A **manifest** declares a family of legal enterprise worlds topology, services, identities, vulnerability classes, difficulty. A **Builder** LLM proposes a concrete snapshot within that family. A **Validator** pipeline admits only snapshots that are runnable, exploitable, patchable, and non-leaking. `reset()` selects a frozen validated snapshot. `step()` runs commands inside it.
 
 
10
 
11
  ```mermaid
12
  flowchart LR
13
+ M[Manifest<br/>topology, services,<br/>bug families, difficulty] --> B[Builder<br/>LLM proposes<br/>snapshot]
14
+ B --> V{Validator<br/>10 checks}
15
+ V -->|fail| B
16
+ V -->|pass| S[Frozen Snapshot]
17
+ S --> E["reset() step() obs + reward"]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
 
19
+ style V fill:#ffd93d,color:#333
20
+ style S fill:#6bcb77,color:#fff
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  ```
22
 
23
+ Red and Blue operate on the **same infrastructure simultaneously**. Red's stealth reward depends on whether Blue catches them. Blue's detection reward depends on Red's actual actions in the logs. This coupling drives co-evolution.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ## Quick Start
26
 
27
  ```bash
28
+ git clone https://github.com/open-cybernauts/open-range.git && cd open-range
 
 
29
  uv sync --all-extras
30
 
31
+ # End-to-end demo (no Docker, no LLM)
32
  uv run python examples/demo.py
33
 
34
+ # Run the server
35
+ python -m open_range.server
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
+ # Tests
38
+ uv run pytest tests/ -v --tb=short
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  ```
40
 
41
+ ## Core Components
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
 
43
+ **Manifest** — YAML defining the legal world: hosts, zones, services, users, NPCs, data assets, credential policies, monitoring coverage, trust relationships, and which vulnerability classes the Builder may plant. Three example manifests ship (healthcare, fintech, SaaS) at tiers 1-3.
 
 
 
44
 
45
+ **Builder** Takes a manifest + curriculum context, outputs a `SnapshotSpec`: topology graph, truth graph (planted vulns + exploit chain), evidence graph (what Blue can find), flags, golden path, NPC traffic, and task briefings. Three implementations: `LLMSnapshotBuilder` (production, via litellm), `TemplateOnlyBuilder` (deterministic, for tests), `FileBuilder` (load from disk).
 
46
 
47
+ **Validator** 10-check admission pipeline. 8 mechanical checks (build/boot, exploitability, patchability, evidence sufficiency, reward grounding, isolation, task feasibility, difficulty calibration) + 2 LLM advisory checks (NPC consistency, realism review). Inverse mutation: patching each planted vuln must break its exploit step.
 
48
 
49
+ **Environment** — `RangeEnvironment(Environment)` following the OpenEnv contract. `reset()` picks a frozen snapshot + samples a task. `step(action)` routes commands to the appropriate container — Red runs on the attacker box, Blue runs on the SIEM. No artificial command allowlists; the container's installed tools are the constraint.
50
 
51
+ **Rewards** All grounded in container state, not LLM judgment:
52
 
53
+ | Red | Blue |
54
+ |-----|------|
55
+ | Flag capture (binary, `docker exec cat`) | Detection (TP rate vs Red's log) |
56
+ | Efficiency (`gamma^steps`) | Patch validity (re-run exploit, must fail) |
57
+ | Stealth (inversely coupled to Blue detection) | Availability (healthcheck fraction) |
58
+ | Anti-hallucination (-0.3 per fake flag) | False positive penalty (-0.2 per NPC flagged) |
59
 
60
+ **Agents** Structural protocol: any object with `reset(briefing, role)` and `act(observation) -> command` works. Ships with `LLMRangeAgent` (litellm, any provider), `ScriptedAgent`, and `HumanAgent`.
61
 
62
  ```python
63
  from open_range.agents.episode import run_episode
 
68
  red = LLMRangeAgent(model="anthropic/claude-sonnet-4-20250514")
69
  blue = LLMRangeAgent(model="openai/gpt-4o")
70
  result = run_episode(env, red, blue, max_steps=50)
 
 
 
 
 
 
 
71
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
72
 
73
+ ## Tier System
 
74
 
75
+ Difficulty grows horizontally — more hosts, zones, and chained attack surface. Not just harder passwords.
 
 
 
 
 
76
 
77
+ | Tier | Scale | Example |
78
+ |------|-------|---------|
79
+ | 1 | 6-8 hosts, 3-4 zones | Healthcare clinic: web + DB + mail + LDAP + SIEM |
80
+ | 2 | 10-12 hosts, 5-6 zones | Financial firm: + VPN, internal APIs, certificate authority |
81
+ | 3 | 14-18 hosts, 7-8 zones | SaaS company: + CI/CD, container registry, partner extranet |
82
 
83
+ ## Server Endpoints
84
 
85
+ | Method | Path | Description |
86
+ |--------|------|-------------|
87
+ | GET | `/health` | Liveness check |
88
+ | GET | `/metadata` | Environment name, version |
89
+ | POST | `/reset` | Start episode, returns initial observation |
90
+ | POST | `/step` | Execute action, returns observation + reward |
91
+ | GET | `/state` | Current episode state |
92
+ | WS | `/ws` | WebSocket session |
93
 
94
+ Compatible with `openenv` when installed; standalone FastAPI fallback otherwise.
 
95
 
96
+ ## Docs
 
 
 
 
 
97
 
98
+ - [Architecture](docs/architecture.md) full pipeline, network topology, episode lifecycle
99
+ - [Builder & Validator](docs/builder-validator.md) — snapshot generation and admission
100
+ - [Red & Blue Agents](docs/red-blue-agents.md) — tandem training, reward coupling, curriculum
101
+ - [Agent Protocols](docs/agent-protocols.md) — agent interface, episode runner, evaluation
102
+ - [OpenEnv Compliance](docs/openenv-compliance.md) — API contract, models, deployment
103
 
104
  ## Built On
105
 
106
+ - [OpenEnv](https://github.com/meta-pytorch/OpenEnv) standardized agentic execution environments
107
+ - Ideas from [R2E-Gym](https://arxiv.org/abs/2504.07164) (hybrid verification), [Self-Play SWE-RL](https://arxiv.org/abs/2512.18552) (formal specs, inverse mutation), PAIRED/UED (constrained generation), POET (mutate + admit)
108
 
109
  ## License
110