docs/architecture.md

Architecture

System Overview

OpenRange uses a snapshot-based architecture. A manifest defines a legal family of company worlds. A builder/mutator (LLM-driven via LiteLLM) proposes candidate snapshots inside that family. A validator gate (purely mechanical) admits only snapshots that boot, remain coherent, and are actually solvable. reset() selects a frozen validated snapshot for the next episode. Mutation happens asynchronously between episodes.

flowchart LR
    A[Base company family<br/>AcmeCorp repo, infra, docs, tools] --> B
    M[Manifest / mutation policy<br/>allowed services, bug families, task families] --> B
    S[Curriculum / failure stats<br/>what Red or Blue is weak at] --> B

    B[Builder / mutator via LiteLLM<br/>generate next snapshot<br/>graph mutations, bug seeding, task derivation] --> C

    subgraph C [Candidate snapshot artifacts]
        C1[Topology graph<br/>hosts, services, users, trust edges]
        C2[Truth graph<br/>bug, exploit chain, blast radius, remediation]
        C3[Evidence spec<br/>logs, alerts, files, tickets, docs]
        C4[Task set<br/>exploit, investigate, patch, report]
    end

    C --> D
    D{Validator gate<br/>build/run, exploitability,<br/>patchability, evidence, reward} -->|fail| B
    D -->|pass| E[Frozen validated snapshot<br/>Acme v_k]

    subgraph R [OpenEnv runtime]
        F[reset - select frozen snapshot + sample task]
        G[Red / Blue agents]
        H[step - run command on frozen snapshot]
        I[Observation + reward]
        J[Rollout results]
    end

    E --> F
    F --> G
    G --> H
    H --> I
    I --> G
    H --> J

    J -. async evolve next snapshot .-> S

    style A fill:#4a9eff,color:#fff
    style M fill:#4a9eff,color:#fff
    style B fill:#ff6b6b,color:#fff
    style D fill:#ffd93d,color:#333
    style E fill:#6bcb77,color:#fff
    style R fill:#7c73e611,stroke:#7c73e6

Key Principle

LLM generates, rules validate. The builder/mutator uses LiteLLM (any model -- Claude, GPT-4o, open models) to generate snapshots creatively. The validator gate runs a 10-check admission pipeline: 8 mechanical checks (deterministic, no LLM) plus 2 LLM advisory checks (configurable, removable). Advisory failures can trigger retry but never override a mechanical pass. Rewards are grounded in container state, never LLM-evaluated.

Infrastructure

Everything runs in Docker Compose. The OpenEnv server is a container in the same compose stack as the range. It communicates with range containers via the Docker SDK (mounted /var/run/docker.sock).

Tier 1 Containers (8 total)

Container	Zone	Services	Role
attacker	external	kali tools, nmap, sqlmap, hydra	Red agent's execution environment
firewall	perimeter	iptables, NAT, port forwarding, IDS rules	Network segmentation between zones
web	DMZ	nginx, PHP/Python app, sshd	Public-facing web application
mail	DMZ	postfix SMTP, dovecot IMAP	Email server with user mailboxes
db	internal	MySQL/PostgreSQL, app schemas, flag data	Database backend for web + mail
files	internal	samba, SMB shares, sensitive documents	File server with access controls
ldap	management	OpenLDAP, Kerberos, user directory	Authentication and authorization for all services
siem	management	rsyslog, log aggregation, alert rules	Blue agent's entry point, receives all logs

Network Zones

flowchart LR
    subgraph ext [External]
        ATK[attacker]
    end
    subgraph perim [Perimeter]
        FW[firewall]
    end
    subgraph dmz [DMZ - 10.0.1.0/24]
        WEB[web]
        MAIL[mail]
    end
    subgraph int [Internal - 10.0.2.0/24]
        DB[db]
        FILES[files]
    end
    subgraph mgmt [Mgmt - 10.0.3.0/24]
        LDAP[ldap]
        SIEM[siem]
    end

    ATK -->|80,443,25 only| FW
    FW --> dmz
    dmz -->|3306,445| int
    dmz -->|389,636| mgmt
    int -->|389| mgmt
    int -.->|blocked| ext

    style ext fill:#ff6b6b22,stroke:#ff6b6b
    style perim fill:#ffd93d22,stroke:#ffd93d
    style dmz fill:#4a9eff22,stroke:#4a9eff
    style int fill:#6bcb7722,stroke:#6bcb77
    style mgmt fill:#7c73e622,stroke:#7c73e6

Service Interconnections

Every service is real and talks to other services:

• web authenticates users against ldap, queries db for app data, logs to siem
• mail does user lookup against ldap, stores mailboxes locally, logs to siem
• files authorizes SMB access via ldap, logs to siem
• db accepts connections from web and files, logs queries to siem
• ldap provides auth for all services, replicates to siem for audit
• siem aggregates logs from all hosts -- Blue agent reads these
• firewall enforces zone boundaries, logs blocked/allowed traffic to siem
• attacker has no access to anything except through the firewall

Data Flow

Snapshot Creation (asynchronous, between episodes)

sequenceDiagram
    participant M as Manifest
    participant B as Builder LLM via LiteLLM
    participant V as Validator Gate
    participant SS as Snapshot Store

    M->>B: Company family + mutation policy + curriculum stats
    B->>B: Generate candidate snapshot
    Note over B: Topology graph, truth graph,<br/>evidence spec, task set,<br/>Dockerfiles, configs, app code

    B->>V: Candidate snapshot artifacts
    V->>V: 1. BuildBootCheck: compose up + healthchecks
    V->>V: 2. ExploitabilityCheck: golden path end-to-end
    V->>V: 3. PatchabilityCheck: inverse mutation test
    V->>V: 4. EvidenceSufficiencyCheck: logs + alerts exist
    V->>V: 5. RewardGroundingCheck: rubrics produce valid scores
    V->>V: 6. IsolationLeakageCheck: zones enforced, no flag leaks
    V->>V: 7. TaskFeasibilityCheck: hosts/evidence/vulns reachable
    V->>V: 8. DifficultyCheck: golden path steps ±20% of tier target
    V->>V: 9. NPCConsistencyCheck: personas pass phish calibration (LLM, advisory)
    V->>V: 10. RealismReviewCheck: scenario plausibility (LLM, advisory)

    alt All mechanical checks pass
        V->>SS: Publish Acme v_k
    else Any check fails
        V-->>B: Reject with failure context, retry
    end

Rendering Pipeline

Between validation and the episode loop, the SnapshotRenderer (builder/renderer.py) converts a validated SnapshotSpec into concrete Docker artifacts via Jinja2 templates:

SnapshotSpec (Pydantic model)
    |
    v
SnapshotRenderer.render(spec, output_dir)
    |
    v
Jinja2 templates (builder/templates/*.j2)
    |
    v
Docker artifacts:
  - docker-compose.yml
  - Dockerfile.web, Dockerfile.db
  - nginx.conf
  - init.sql
  - iptables.rules

The renderer flattens SnapshotSpec fields (topology, zones, hosts, flags, vuln types) into a template context. Templates use conditional blocks (e.g., search_endpoint, download_endpoint) driven by the snapshot's vulnerability types and injection points.

Episode Loop (synchronous, standard OpenEnv)

sequenceDiagram
    participant T as Training Loop
    participant E as OpenEnv Server
    participant SS as Snapshot Store
    participant R as Range Containers

    T->>E: reset()
    E->>SS: Select validated snapshot + sample task
    SS-->>E: Frozen snapshot Acme v_k
    E->>R: Boot or restore snapshot containers
    E-->>T: RangeObservation with task briefing

    loop Red and Blue alternate
        T->>E: step Red action
        E->>R: docker exec attacker ...
        R-->>E: stdout + stderr
        E-->>T: RangeObservation + reward

        T->>E: step Blue action
        E->>R: docker exec siem ...
        R-->>E: log data
        E-->>T: RangeObservation + reward
    end

    Note over T,R: Episode ends: flag captured, max steps, or timeout

Curriculum Feedback

The Builder acts as a simulated expert curriculum designer. Episode results feed back to shape future snapshots:

1. Track Red solve rate and Blue detection rate per snapshot (per vuln class, per tier)
2. Feed failure stats to Builder as runtime_context on next build
3. Builder LLM adjusts difficulty via r_inject = 1 - (1+alpha)*s (frontier calibration from SWE-RL)
4. Target agent weaknesses: if Red masters SQLi, seed SSRF or chained vulns next
5. When agents plateau: horizontal growth (add containers, zones, services)

Snapshot Artifacts

Each validated snapshot contains:

Artifact	What it is	Example
Topology graph	Hosts, services, users, network zones, trust edges	8 containers, 4 zones, 12 users, firewall rules
Truth graph	Bug location, exploit chain, blast radius, remediation steps	SQLi in web/search.php, chain to DB creds, fix: parameterize
Evidence spec	Logs, alerts, files, tickets that Blue can investigate	web access logs show injection patterns, SIEM alert rule fires
Task set	What Red and Blue are asked to do	Red: find and exploit vulns. Blue: detect, investigate, patch
Docker artifacts	Compose file, Dockerfiles, configs, app code, flag files	docker-compose.yml, nginx.conf, init.sql, app.php
Golden path	Step-by-step exploit chain with expected outputs	nmap, curl, sqlmap, pivot, submit_flag
NPC traffic config	Background traffic rates per service	HTTP: 30/min, SMTP: 5/min, LDAP: 10/min
NPC persona cards	LLM-driven employee personas with susceptibility profiles	Janet Smith (Marketing, security_awareness=0.3, phishing_susceptibility=0.7)

Reward Architecture

All rewards live in server/rewards.py. Composite classes use a weighted sum with configurable weights, plus tier-based complexity scaling and additive penalties.

CompositeRedReward
  ├── FlagReward                binary, docker exec verified (weight: 0.40)
  ├── EfficiencyReward          gamma^steps (weight: 0.10)
  ├── StealthReward             1 - (detected/total), coupled to Blue (weight: 0.15)
  ├── EvidenceReward            structural quality of submit_evidence (weight: 0.10)
  ├── SocialEngineeringReward   NPC fell for phish/pretext, Level 1+ (weight: 0.05)
  ├── ComplexityBonus           tier_multiplier * base_reward (1.0x-3.0x)
  └── HallucinationPenalty      -0.3 per fake flag (additive, not tier-scaled)

CompositeBlueReward
  ├── DetectionReward           TP rate vs Red action log (weight: 0.35)
  ├── PatchReward               binary, golden path re-execution (weight: 0.25)
  ├── AvailabilityReward        healthcheck fraction (weight: 0.15)
  ├── PhishingDetectionReward   correctly identified SE in logs, Level 1+ (weight: 0.05)
  ├── ComplexityBonus           tier_multiplier * base_reward (1.0x-3.0x)
  └── FalsePositiveReward       -0.2 per NPC traffic/email flagged (additive, not tier-scaled)

Both composite classes expose a compute(action, observation, state, snapshot, ctx) method called by RangeEnvironment.step(), as well as a lower-level score() with explicit arguments for unit testing.

Rewards are computed from container state and action logs, never from LLM judgment.

Tier-Scaled Reward Ceiling

Reward ceilings scale with environment complexity so that harder snapshots produce proportionally larger training signals:

Tier	Hosts	Multiplier	Max Red Reward	Max Blue Reward
1	6-8	1.0x	1.0	1.0
2	10-12	1.5x	1.5	1.5
3	14-18	2.0x	2.0	2.0
4	20-25	2.5x	2.5	2.5
5	30+	3.0x	3.0	3.0

This ensures agents are incentivized to attempt harder environments rather than grinding easy Tier 1 snapshots.

NPC Evolution: Shell Scripts to LLM Agents

NPCs progress from mechanical noise generators to intelligent social engineering targets. Each level adds a modality without removing the previous one.

flowchart TB
    subgraph l0 [Level 0 - Shell Scripts]
        SH[curl/sshpass/mysql loops<br/>Configurable rates per service<br/>Labeled for FP scoring]
    end

    subgraph l1 [Level 1 - LLM Email NPCs]
        PERSONA[Persona cards from Builder<br/>name, role, security_awareness,<br/>susceptibility profile]
        MAIL_NPC[Async LLM agent checks Postfix<br/>Decides: click, reply, ignore, report<br/>Creates realistic log trail]
        PERSONA --> MAIL_NPC
    end

    subgraph l2 [Level 2+ - Multimodal Stretch]
        CHAT[Internal chat/IM]
        VOICE[Voice via TTS/STT]
        DOCS[Document inspection via vision LLM]
    end

    l0 -->|always running| l1
    l1 -->|adds social engineering surface| l2

    style l0 fill:#6bcb7722,stroke:#6bcb77
    style l1 fill:#4a9eff22,stroke:#4a9eff
    style l2 fill:#7c73e622,stroke:#7c73e6

Key design: NPC LLM calls are async, not in the step() hot path. Red sends a phishing email to Postfix in one step. The NPC agent processes it on its own schedule (per email_check_interval_min). Red observes the result in later steps via access logs, new sessions, or SIEM alerts. Blue sees the same logs and must distinguish legitimate NPC-to-NPC email from Red's social engineering.

Implementations (all in builder/npc/npc_agent.py):

Class	Level	LLM?	Description
NullNPCBehavior	0	No	No-op; always returns ignore. Shell scripts handle all traffic.
RuleBasedNPCBehavior	0-1	No	Heuristic decisions based on susceptibility * plausibility score thresholds.
LLMNPCAgent	1+	Yes	Full LLM-driven persona. Runs an async run_loop() polling for stimuli on the persona's schedule.

The NPCManager (builder/npc/npc_manager.py) orchestrates both levels: it starts Level 0 shell scripts (http_traffic.sh, db_traffic.sh, ssh_traffic.sh) and, when npc_traffic.level >= 1, spawns LLMNPCAgent.run_loop() as asyncio tasks for each persona.

Pluggable Infrastructure Components

Builder, NPC behavior, validator checks, and Red/Blue agents are all pluggable via Protocol-based structural subtyping. No base class inheritance required. Any class with a matching method signature satisfies the protocol.

See docs/agent-protocols.md for the full design.

Four Protocols

# protocols.py — infrastructure components
@runtime_checkable
class SnapshotBuilder(Protocol):
    async def build(self, manifest: dict, context: BuildContext) -> SnapshotSpec: ...

@runtime_checkable
class NPCBehavior(Protocol):
    async def decide(self, persona: NPCPersona, stimulus: Stimulus) -> NPCAction: ...

@runtime_checkable
class ValidatorCheck(Protocol):
    async def check(self, snapshot: SnapshotSpec, containers: ContainerSet) -> CheckResult: ...

# agents/protocol.py — Red/Blue agents
@runtime_checkable
class RangeAgent(Protocol):
    def reset(self, briefing: str, role: Literal["red", "blue"]) -> None: ...
    def act(self, observation: str) -> str: ...

Configuration via YAML

# openrange.yaml
agents:
  builder:
    class: open_range.builder.builder.LLMSnapshotBuilder
    kwargs:
      model: "anthropic/claude-sonnet-4-20250514"
      temperature: 0.7
  npc_behavior:
    class: open_range.builder.npc.npc_agent.LLMNPCAgent
    kwargs:
      model: "anthropic/claude-haiku-4-5-20251001"
  validator_checks:
    - class: open_range.validator.build_boot.BuildBootCheck
    - class: open_range.validator.exploitability.ExploitabilityCheck
    - class: open_range.validator.patchability.PatchabilityCheck
    - class: open_range.validator.evidence.EvidenceSufficiencyCheck
    - class: open_range.validator.reward_grounding.RewardGroundingCheck
    - class: open_range.validator.isolation.IsolationLeakageCheck
    - class: open_range.validator.task_feasibility.TaskFeasibilityCheck
    - class: open_range.validator.difficulty.DifficultyCheck
    - class: open_range.validator.npc_consistency.NPCConsistencyCheck
    - class: open_range.validator.realism_review.RealismReviewCheck
    # add, remove, or reorder checks as needed

Resolution

Dynamic import + Protocol check at startup:

def resolve_component(class_path: str, kwargs: dict, protocol: type) -> Any:
    module_name, _, class_name = class_path.rpartition(".")
    module = importlib.import_module(module_name)
    cls = getattr(module, class_name)
    instance = cls(**kwargs)
    if not isinstance(instance, protocol):
        raise TypeError(f"{class_path} does not satisfy {protocol.__name__}")
    return instance

Default Implementations

Protocol	Default	Alternatives
SnapshotBuilder	LLMSnapshotBuilder (LiteLLM)	TemplateOnlyBuilder (testing), FileBuilder (demo)
NPCBehavior	NullNPCBehavior (Level 0, no-op)	LLMNPCAgent (Level 1+, LiteLLM), RuleBasedNPCBehavior (heuristic, no LLM)
ValidatorCheck	8 mechanical + 2 LLM advisory	Add, remove, or reorder via config
RangeAgent	ScriptedAgent (replay commands)	LLMRangeAgent (LiteLLM), HumanAgent (interactive stdin), ScriptedRedAgent/ScriptedBlueAgent (pre-built demo sequences)

Environment Variables

Env vars override YAML config at deploy time:

Env Var	Overrides	Default
OPENRANGE_BUILDER_MODEL	Builder LLM model	anthropic/claude-sonnet-4-20250514
OPENRANGE_NPC_MODEL	NPC LLM model	anthropic/claude-haiku-4-5-20251001
LITELLM_API_KEY	Global API key	(or model-specific keys)

Checks 1-8 are purely mechanical -- deterministic, no LLM. Check 9 (NPC consistency) uses an LLM for NPC persona testing. Check 10 (realism review) is an LLM advisory check. Both LLM checks are advisory (advisory=True): failure triggers retry but never blocks admission. Both are configurable -- remove them from the validator_checks list to run fully mechanical.