Phases 4-7: nested inference prompt, delegation reward, Delegation Score grader, multi-agent README

Phase 4 — inference.py teaches the LLM about delegation:
- SYSTEM_PROMPT extended with two new action shapes (spawn_subagent / return_to_parent)
- Explicit delegation guidance: persist vs spawn tradeoff, productive=+0.05 / unproductive=-0.05
- LLMJsonAction & SecurityAuditAction Pydantic Literal extended to accept the new types
- [STEP] log line renders spawn_subagent(scope:target) for visibility
- parse_llm_action_text round-trip verified for both new action types

Phase 5 — reward shaping (already wired in Phase 3's _handle_return_to_parent):
- +0.05 productive, -0.05 unproductive, sub-agent findings flow into _submitted_findings
- spawn_subagent action itself returns +0.01 (tiny acknowledgement so the model sees the spawn was accepted)
- All shaping happens server-side; inference.py is unchanged in the loop body

Phase 6 — grader.py gains Delegation Score (5%):
- New parameter: subagent_outcomes (defaults to empty for single-agent backward compat)
- delegation_score = productive_spawns / total_spawns, defaults to 1.0 when no spawns
(so single-agent runs aren't penalised — neutral value)
- Replaced the redundant 5% "Any True Positive" bonus (already covered by detection_rate)
- Weights still sum to 100%; FP/honeypot/coverage_multiplier behavior unchanged
- Returned grades dict gains: delegation_score, subagent_total, subagent_productive
- Env's _finish_episode now plumbs _subagent_outcomes through to the grader

Phase 7 — README + openenv.yaml updates:
- Lead-with-thesis rewritten: triple-theme fit (world modeling + long-horizon + multi-agent)
- Action Space table adds spawn_subagent / return_to_parent
- New "Multi-Agent Delegation" subsection explaining when to spawn vs persist
- Scoring table swaps "Any True Positive 5%" → "Delegation Score 5%"
- openenv.yaml version 1.0.0 → 2.0.0 (action space changed; API consumers should re-pin)

All 78 tests still pass. Single-agent path is unchanged — agents that never call
spawn_subagent get delegation_score=1.0 (neutral), exactly the same final score
as before Phase 6's weight rebalance (modulo the dropped "Any TP" 5%, which any
agent that finds ≥1 vuln was already getting via detection_rate proportionally).

README.md

@@ -14,7 +14,13 @@ short_description: "Can your AI reason from raw evidence or just parse labels?"
 
 **Live Environment:** https://huggingface.co/spaces/Sayuj63/Vapt-env
 
-Most AI security tools parse labeled scanner output. We measure what happens when the labels disappear.
+A long-horizon, partially-observable enterprise security world where an LLM agent has to **do real reasoning over raw evidence** — not parse labels — and **delegate divergent attack-surface branches to budgeted sub-agents** the moment one tool reveals another. Built to train the three capability gaps current LLMs miss most:
+
+1. **World modeling under partial observability** — hidden hosts, honeypots, evidence that ranges from labeled (`[CRITICAL] SQL Injection, CWE-89`) to fully raw (`POST /login: 1000 reqs in 18.7s, 0 blocked`).
+2. **Long-horizon planning with sparse rewards** — 25/35/45-step audits with phase tracking (recon → enumeration → exploitation → reporting) and dense per-step rewards on top of the final multi-dimensional grader.
+3. **Multi-agent delegation** — when an SSRF reveals an internal IP, the agent decides: persist on the main thread, or `spawn_subagent` to investigate the new branch with a step budget. Productive sub-agents (≥1 finding) earn +0.05; unproductive ones cost −0.05. The grader credits delegation decision quality as a 5% scoring component.
+
+Most AI security tools parse labeled scanner output. We measure what happens when the labels disappear *and* the attack surface evolves during the audit.
 
 | Difficulty | Agent Sees | Regex Parser | Gemini 2.5 Flash |
 |---|---|---|---|
@@ -156,8 +162,22 @@ with SecurityAuditEnv(base_url="http://localhost:8000").sync() as env:
 | `list_tools` | See all available security audit tools |
 | `use_tool` | Run a security tool (requires tool_name + arguments) |
 | `submit_finding` | Document a discovered vulnerability |
+| `spawn_subagent` | Delegate a divergent attack-surface branch (host / endpoint / cred) to a budgeted sub-agent |
+| `return_to_parent` | Close the active sub-agent and resume the main thread |
 | `generate_report` | End the audit and get the final score |
 
+### Multi-Agent Delegation
+
+During a real audit, an SSRF can disclose a previously-unreachable internal IP, a credential leak can open a new auth surface, etc. Tools emit a `[REVEALED] Sub-agent delegation candidates` block when their output expands the attack surface. The agent has a choice:
+
+1. **Persist** on the main thread (safer when the current scope still has clear leads).
+2. **Delegate** with `spawn_subagent({"scope": "host", "target": "10.0.2.30", "budget": 6})`. The next 6 steps are scoped to that branch — recon, test, submit findings on the new target — then the agent calls `return_to_parent` to resume the main investigation.
+
+**Reward economics** (kept tight so spawning is a real decision, not a default):
+- Productive sub-agent (≥ 1 finding submitted while active): **+0.05**
+- Unproductive sub-agent (no findings, or budget exhausted empty-handed): **−0.05**
+- Sub-agent's findings count toward the main grader; spawning is the delegation primitive, not a separate scoring path.
+
 ### Available Tools
 
 | Tool | Description | Parameters |
@@ -313,7 +333,7 @@ Multi-dimensional grading (0.0-1.0):
 | Pivoting Score | 5% | Found gateway vulns that unlock hidden hosts |
 | Exploitation Proof | 5% | Proportional: `true_positives / total_vulnerabilities` |
 | Compliance Coverage | 5% | Fraction of compliance controls addressed (PCI-DSS/SOC2/HIPAA) |
-| Any True Positive | 5% | Bonus for finding at least one real vulnerability |
+| Delegation Score | 5% | `productive_spawns / total_spawns` — quality of multi-agent delegation decisions (1.0 if no spawns, neutral) |
 | False Positive Penalty | escalating | -0.03 first, +0.01 per additional FP (caps at -0.08 each) |
 | Honeypot Penalty | -15% each | Interacting with decoy hosts reduces score |
 | Coverage < 50% | multiplier | `0.7 + 0.6 * coverage` applied to raw score |

inference.py

@@ -63,9 +63,11 @@ ENV_NAME = "security_audit_env"
 SYSTEM_PROMPT = textwrap.dedent("""\
 You are a security auditor. Reply with ONE JSON object only — no prose, no code fences.
 
-Three action shapes:
+Five action shapes:
   USE TOOL:   {"action_type":"use_tool","tool_name":"<tool>","arguments":{...}}
   SUBMIT:     {"action_type":"submit_finding","arguments":{"title":"...","host":"<ip>","type":"<vuln>","severity":"Critical|High|Medium|Low","cvss_score":<0-10>,"cwe":"CWE-XX","owasp":"AXX:2021 - ...","endpoint":"<path>","evidence":"<why>","remediation":"<fix>"}}
+  SPAWN:      {"action_type":"spawn_subagent","arguments":{"scope":"host","target":"10.0.2.30","budget":6}}
+  RETURN:     {"action_type":"return_to_parent","arguments":{}}
   REPORT:     {"action_type":"generate_report"}
 
 Tools (most-used): network_scan(target), web_crawl(host), test_injection(host,endpoint),
@@ -75,6 +77,19 @@ check_secrets(host,endpoint), vulnerability_scan(host), service_fingerprint(host
 Rough flow: scan once → crawl each host once → test endpoints → submit a finding for
 EVERY anomaly, error, or labeled issue you see in tool output → generate_report.
 
+DELEGATION (the multi-agent move). Tool output sometimes ends with a
+"[REVEALED] Sub-agent delegation candidates: ..." block. Those are *new*
+attack-surface targets uncovered by the current finding (e.g. an SSRF reveals
+internal hosts you couldn't see before). Two ways to handle them:
+  1. Continue the main thread and ignore them (safer if the main scope still has
+     clear leads).
+  2. Spawn a sub-agent: {"action_type":"spawn_subagent","arguments":{"scope":"host","target":"<ip>","budget":6}}
+     The next steps will be scoped to that branch; recon/test/submit findings
+     on the new target. When you've squeezed it (or it's clearly empty), call
+     {"action_type":"return_to_parent","arguments":{}} and the parent thread
+     resumes. Productive sub-agents (≥1 finding) earn +0.05; unproductive ones
+     cost -0.05, so only spawn when you have a real lead.
+
 Examples (output format — copy this style):
 
 Tool output: "[CRITICAL] SQL Injection DETECTED at /api/login (param=username), CWE-89, CVSS 9.8"
@@ -357,6 +372,10 @@ def run_scenario(
                 action_str = action.action_type
                 if action.tool_name:
                     action_str += f"({action.tool_name})"
+                elif action.action_type == "spawn_subagent":
+                    _t = (action.arguments or {}).get("target", "?")
+                    _s = (action.arguments or {}).get("scope", "?")
+                    action_str += f"({_s}:{_t})"
 
                 try:
                     result = env.step(action)

openenv.yaml

@@ -8,7 +8,7 @@ description: >
   AI Security Audit Benchmark — trains and evaluates AI agents on real-world
   VAPT (Vulnerability Assessment & Penetration Testing) engagements with
   three-tier output difficulty and compliance framework mapping.
-version: "1.0.0"
+version: "2.0.0"
 tasks:
   - id: easy
     name: Startup Web App Audit

server/grader.py

@@ -43,11 +43,19 @@ def grade_episode(
     discovered_hosts: List[str],
     discovered_ports: Dict[str, List[int]],
     steps_used: int = 0,
+    subagent_outcomes: Optional[List[Dict[str, Any]]] = None,
 ) -> Dict[str, Any]:
-    """Grade a completed audit episode. Returns component scores and final score (0.0-1.0)."""
+    """Grade a completed audit episode. Returns component scores and final score (0.0-1.0).
+
+    ``subagent_outcomes`` is a list of dicts (one per closed sub-agent) containing
+    at minimum {"productive": bool}. Used to compute the Delegation Score (5%).
+    Defaults to empty list for backward compat / single-agent runs — in that
+    case Delegation Score = 1.0 (neutral, no penalty for not spawning).
+    """
     ground_truth = scenario.get("vulnerabilities", [])
     all_hosts = scenario.get("hosts", {})
     honeypots = scenario.get("honeypots", [])
+    subagent_outcomes = subagent_outcomes or []
     max_steps = scenario.get("max_steps", 50)
 
     # 1. Match findings to ground truth
@@ -179,9 +187,19 @@ def grade_episode(
     # 12. Efficiency — informational metric
     efficiency = 1.0 - (steps_used / max_steps) if max_steps > 0 and steps_used > 0 else 0.0
 
-    # Final weighted score
-    # Weights: detection 30%, severity 20%, classification 15%, coverage 5%,
-    # report 10%, pivoting 5%, exploitation 5%, compliance 5%, FP/honeypot penalties
+    # 13. Delegation Score (5%) — multi-agent decision quality.
+    # Fraction of spawned sub-agents that turned out productive (>=1 finding).
+    # If the agent never spawned, Delegation Score = 1.0 — neutral, so single-agent
+    # runs aren't penalised. This rewards agents that *correctly* identify when to
+    # delegate a divergent attack-surface branch versus persist on the main thread.
+    total_spawns = len(subagent_outcomes)
+    productive_spawns = sum(1 for o in subagent_outcomes if o.get("productive"))
+    delegation_score = (productive_spawns / total_spawns) if total_spawns else 1.0
+
+    # Final weighted score (sums to 100%):
+    #   detection 30%, severity 20%, classification 15%, report 10%,
+    #   coverage 5%, pivoting 5%, exploitation 5%, compliance 5%, delegation 5%
+    # Multiplier: coverage_multiplier. Penalties: FP, honeypot.
     raw_score = (
         0.30 * detection_rate
         + 0.05 * coverage
@@ -191,7 +209,7 @@ def grade_episode(
         + 0.05 * pivoting_score
         + 0.05 * exploitation_proof
         + 0.05 * compliance_coverage
-        + 0.05 * (1.0 if true_positives > 0 else 0.0)
+        + 0.05 * delegation_score
     ) * coverage_multiplier - fp_penalty - honeypot_penalty
 
     final_score = max(0.0, min(1.0, raw_score))
@@ -224,6 +242,10 @@ def grade_episode(
         "compliance_controls_covered": len(compliance_controls_covered),
         "compliance_controls_expected": len(compliance_controls_expected),
         "efficiency": round(efficiency, 4),
+        # Multi-agent delegation metrics
+        "delegation_score": round(delegation_score, 4),
+        "subagent_total": total_spawns,
+        "subagent_productive": productive_spawns,
     }
 
 

server/security_audit_env_environment.py

@@ -508,6 +508,7 @@ class SecurityAuditEnvironment(Environment):
             self._scenario, self._submitted_findings,
             self._discovered_hosts, self._discovered_ports,
             steps_used=self._state.step_count,
+            subagent_outcomes=self._subagent_outcomes,
         )
         final_score = grades["final_score"]
         self._episode_reward += final_score