docs/deep-research-report.md

# OpenEnv Hackathon Execution Pipeline for a Safe Cybersecurity Analyst Environment

## Executive decision and scope selection

**SECTION 1 — Executive Decision**

[SOURCED] The hackathon’s *validator + judging* constraints strongly favour environments that: (a) simulate a real-world task (not “games/toys”), (b) are fully OpenEnv-compliant, (c) ship with **≥3 tasks with graders**, (d) produce **scores/rewards in the 0–1 range**, (e) include a reproducible `inference.py` that uses the **OpenAI client** (for any LLM calls) and prints strict `[START]/[STEP]/[END]` logs, and (f) run within a ~**20 minute** budget on ~**2 vCPU / 8 GB** infra. citeturn3view6turn3view7turn22view1turn22view2  

[INFERENCE] Under these realities, your cybersecurity-analyst direction is *not* automatically the best, but it *can* become a high-probability-to-win choice if—and only if—you narrow to a deterministic, bounded, “investigate → cite evidence → verify → remediate” loop where (i) tools are tightly sandboxed, (ii) graders are deterministic, and (iii) the action space is small enough to be learnable and demo-able under the runtime cap.

[PROPOSAL] **Decision:** keep the cybersecurity direction, but **narrow aggressively** to a V1 environment that benchmarks **disciplined security triage + evidence-grounded reporting**, not pentesting/exploitation. The V1 I recommend building is:

[PROPOSAL] **“SecOps Evidence Gym”** — a safe, isolated OpenEnv environment where an agent investigates a *synthetic* microservice “organisation” via a **bounded tool API**, collects **evidence IDs**, validates candidate findings through **deterministic verifiers**, and submits a structured remediation report.  

[SOURCED] This matches strong “winner DNA” seen in `kube-sre-gym` (realistic professional workflow + verification + narrative clarity) while remaining implementable in a hackathon budget. citeturn10view0turn18view0  

[PROPOSAL] **What to cut entirely in V1 (non-negotiable):**  
- “Live target” behaviour; no external network targets; no arbitrary HTTP to the internet. 🔒  
- Any exploit payload recipes, exploit chains, privilege-escalation playbooks, or “how to hack X”. 🔒  
- Arbitrary shell access (`bash`, `kubectl`, `nmap`, etc.) inside the environment. (Action space explosion + safety risk.)  
- LLM-only grading/judging for correctness. (Reward hacking + non-determinism.)  

[SOURCED] **What to keep (but narrow):** tool-using investigation, multi-step interaction, and deterministic verification—these are consistent with what OpenEnv is designed to support (typed `reset/step/state`, isolated server, type-safe schemas). citeturn18view0turn19search1  

**SECTION 3 — Candidate Scope Comparison**

[SOURCED] The scoring below is anchored on hackathon validator requirements (3+ graded tasks, 0–1 scoring, strict inference logging, runtime limits) plus OpenEnv’s scaffolding/CLI/deployment model. citeturn3view6turn18view0turn22view1  

[PROPOSAL] Weighted criteria (sum=1.00): judging fit 0.14, OpenEnv fit 0.12, grader determinism 0.14, implementation risk 0.12, runtime feasibility 0.10, demoability 0.10, real-world usefulness 0.10, novelty 0.08, training usefulness 0.06, shipping-on-time likelihood 0.04.

| Candidate scope | Judging fit | OpenEnv fit | Determinism | Impl risk (lower=better) | Runtime | Demo | Real-world use | Novelty | Training use | Ship-on-time | Weighted total |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| **A. Your original direction:** “disciplined cyber analyst investigating a sandbox” (broad) | 8 | 7 | 6 | 4 | 6 | 8 | 8 | 8 | 8 | 4 | **6.7** |
| **B. Narrow cyber variant (recommended):** evidence-first triage lab with bounded tools + deterministic verifiers + structured report | 9 | 8 | 9 | 7 | 9 | 9 | 8 | 7 | 8 | 8 | **8.4** |
| **C. Adjacent: SRE incident triage (single-turn, deterministic logs → RCA)** | 9 | 8 | 10 | 9 | 10 | 8 | 8 | 5 | 6 | 9 | **8.3** |
| **D. Adjacent: prompt-injection “WAF” benchmark** | 7 | 8 | 8 | 7 | 9 | 9 | 7 | 7 | 6 | 7 | **7.6** |

[INFERENCE] Candidate C (SRE triage) is extremely validator-safe (many examples already pass deep validation), but it is likely more saturated and less “new” in judging. Candidate B keeps your cybersecurity theme while retaining the determinism and boundedness that the validator and time budget demand, so it is the best balance for **winning + real-world usefulness**.

**SECTION 4 — Final V1 Problem Statement**

[PROPOSAL] **One-sentence version:** Build a safe OpenEnv environment that trains and benchmarks an agent to perform **evidence-grounded security triage** on a bounded synthetic system and produce a **remediation-oriented report** without hallucinating.

[PROPOSAL] **Short pitch (demo-ready):** “SecOps Evidence Gym” gives the model an alert and a constrained set of investigation tools. The model must gather evidence, validate findings with deterministic verifiers, and submit a structured report. Scores reward verified correctness, penalise hallucinated claims and wasted steps, and remain strictly within (0,1) to satisfy the hackathon validator. ✅🔒

[PROPOSAL] **Precise implementation version:** A FastAPI OpenEnv server exposing typed `reset()`, `step()`, `state()` and a manifest `openenv.yaml` with at least three tasks (easy/medium/hard), each associated with a grader. The environment implements multi-step tool calls inside `step()`, and uses deterministic verifiers plus a strict score clamp to `(0,1)` for validator compatibility. citeturn18view0turn19search1turn23view0turn26view0turn27search0  

## Winner patterns and judging fit extraction

**SECTION 2 — Winner Pattern Extraction**

[SOURCED] `kube-sre-gym` (OpenEnv Hackathon SF winner) demonstrates several patterns that appear strongly aligned with what judges value: a **realistic professional task**, an explicit **multi-step workflow** (triage → investigate → fix → verify), **multi-layer verification** (programmatic health checks + judge), a strong narrative that explains learning dynamics, and deployment on **Hugging Face Spaces**. citeturn10view0turn14view0  

image_group{"layout":"carousel","aspect_ratio":"16:9","query":["kube-sre-gym OpenEnv Hackathon winner screenshot","OpenEnv framework architecture diagram","Hugging Face Spaces OpenEnv environment web demo","Incident Triage Environment OpenEnv Hackathon 2026 screenshot"],"num_per_query":1}

[SOURCED] Concretely, `kube-sre-gym` highlights: “real cluster/tool interaction”, verification layers to prevent false success, curriculum progression, and strong documentation that makes the environment’s value obvious quickly. citeturn10view0  

[SOURCED] A 2026 hackathon submission that explicitly claims Phase-2 deep-validation success (`Incident-Triage-Environment`) reveals particularly transferable “validator-winning” patterns:  
- A manifest with `spec_version`, `runtime`, `app`, `port`, and a **`tasks:` list where each task has a `grader:` pointing to importable Python functions**. citeturn23view0  
- Deterministic graders that clamp outputs to a validator-friendly range. citeturn26view0turn26view3  
- An `inference.py` that uses the OpenAI SDK and prints the strict stdout protocol with `[START]`, `[STEP]`, `[END]` lines in a stable key=value format. citeturn22view1turn22view2turn22view4  

[SOURCED] The official OpenEnv repo reinforces what is transferable: type-safe action/observation/state contracts, containerised isolation, and standard Gym-like APIs. It also explicitly says OpenEnv is experimental and APIs can change, which increases the value of a minimal, validator-first build loop. citeturn18view0turn19search2  

[INFERENCE] Given hackathon evaluation combines programmatic checks with LLM scoring, you must optimise for **deterministic correctness** *and* a compelling narrative/demo. citeturn3view7turn3view6  

[PROPOSAL] Transferable “winner patterns” you should copy (selectively):  
- **Strong “professional workflow” framing** (SRE, security analyst, triage) with clear step boundaries.  
- **Small, discoverable tool set** that mirrors real practice (logs, config, policy checks) while staying bounded.  
- **Deterministic verification** (programmatic checks) as the source of truth for correctness.  
- **Narrative traceability**: logs, episode IDs, and a short “watch the agent work” demo.  
- **Deployment excellence**: clean Docker build, working `/health`, working `/web` UI if enabled, and reproducible inference.

[PROPOSAL] What *not* to copy blindly:  
- The “real cluster” dependency (e.g., GKE) is high operational burden and can fail the hackathon’s limited infra budget. citeturn10view0turn3view6  
- LLM-as-judge for correctness (too easy to reward-hack; non-deterministic). (Use it, at most, for *format/style*, not correctness.)  

## Core V1 environment design and benchmark tasks

**SECTION 8 — Core Environment Design**

**V1 concept (aggressively narrow).**  
[PROPOSAL] Your environment is a **synthetic organisation** with a small, fixed topology (three “services” + artefacts). The agent receives an alert. It can only interact via **approved tools** (implemented inside the simulator). It must (a) gather evidence IDs, (b) validate candidate findings, and (c) submit a report.

**Topology (V1).**  
[PROPOSAL] Fixed components (no containers inside containers):  
- `gateway` (public entry), `profile-service`, `admin-service`  
- `repo_snapshot` (static code/config excerpts)  
- `telemetry` (sanitised logs + “header snapshot” + “dependency manifest snapshot”)  

**Reset logic.**  
[PROPOSAL] `reset(task_id=..., seed=...)` selects a scenario variant and initialises:  
- episode ID, step count  
- scenario ground truth (one injected issue per episode in V1)  
- tool budgets + “allowed scope” banner  
- an evidence registry mapping `EVID-### → artefact snippet`  
Return an initial observation containing the alert, the tool catalogue, and an empty “verified findings” list.

**Randomisation strategy.**  
[PROPOSAL] Use seed-driven, deterministic randomisation:  
- rename services/routes/IDs (`profile-service` might become `user-profile`),  
- shuffle benign log lines around the key evidence,  
- vary exact header sets / dependency versions within a small closed set,  
- keep each scenario **fully reproducible from the seed**.

[SOURCED] Benchmark generators (e.g., AMaze) exist specifically to create diverse but controlled environments for evaluating generalisation, supporting the idea of seeded procedural variation rather than a single static scenario. citeturn16search7turn16search1  

**Safety boundaries.**  
[PROPOSAL] The sandbox contains **no live targets**, no real secrets, and no outbound network. “Secrets” are synthetic strings with an explicit “DO NOT USE OUTSIDE LAB” marker. Tools return synthetic results only. 🔒  

[SOURCED] NIST’s cyber range guidance emphasises cyber ranges as safe and legal environments for training and assessment; separate research also discusses that cyber ranges themselves have security risks that must be mitigated (e.g., leakage/misuse), reinforcing the need for strict isolation and artefact sanitisation. citeturn29search1turn29search2  

**How state is exposed to the agent.**  
[PROPOSAL] Expose only a concise state summary: current phase, step budget remaining, tools remaining, verified findings count, and recent evidence IDs. Keep full ground truth hidden.

**Tool/action design (bounded action space).**  
[PROPOSAL] V1 tool list (keep it ≤8 tools):  
1) `list_assets()` → returns asset IDs and route IDs  
2) `get_log_events(service_id, query)` → returns evidence IDs  
3) `check_security_headers(service_id)` → returns evidence IDs + pass/fail list  
4) `search_repo(query)` → returns evidence IDs from code snippets  
5) `scan_dependencies()` → returns evidence IDs from a lockfile excerpt  
6) `create_finding(finding_type, evidence_ids, severity_guess, remediation)` → stores candidate finding  
7) `validate_finding(finding_id)` → deterministic verifier; returns `(verified, matching_gt_id)`  
8) `submit_report(report_json)` → terminal action  

**Anti-loop logic.**  
[PROPOSAL] Track action signatures `(tool_name, args_hash)` and:  
- apply increasing penalties for repeats,  
- hard-stop an episode if identical actions repeat ≥6 times, returning `done=True` with a low score,  
- always return a valid observation (never a server crash) to preserve training rollouts.

[SOURCED] OpenEnv’s environment-creation guidance strongly implies you should implement robust behaviour around `reset/step/state` with typed contracts and predictable server behaviour. citeturn19search1turn18view0  

**SECTION 9 — Tasks / Benchmarks**

[SOURCED] The hackathon requires **at least 3 tasks with graders** and explicitly checks the tasks registry. citeturn3view6turn27search0  

[PROPOSAL] V1 ships exactly **3 flagship tasks**, difficulty-tiered, each with deterministic success criteria and intermediate milestones.

**Flagship tasks (easy/medium/hard).**  
[PROPOSAL] Each task is a *family* with small seeded variants.

**Easy: Secret exposure in repo snapshot**  
- Goal: identify a leaked synthetic API key in a config file excerpt; propose rotation/removal.  
- Deterministic success: report includes the correct finding type `secret_exposure`, includes ≥1 correct evidence ID, and remediation mentions rotation + removal.  
- Intermediate rewards: `search_repo()` surfaces the evidence ID; `create_finding()` with correct type gets partial credit; `validate_finding()` confirms.  
- False-positive check: claiming *additional* vulnerabilities not verified triggers penalty.

**Medium: Missing security headers**  
- Goal: detect missing/weak security headers in a service “header snapshot”; propose remediation.  
- Deterministic success: correct missing header set identification (from a fixed list), plus remediation mapping (e.g., add HSTS, CSP) within the environment’s rubric.  
- Intermediate rewards: correct tool usage (`check_security_headers()`), correct mapping to finding type, successful verifier validation.  
- Generalisation: header ordering/extra benign headers vary by seed.

**Hard: Authorisation boundary misconfiguration**  
- Goal: detect an access control policy bug in a route/role matrix (modelled safely, without exploitation).  
- Deterministic success: evidence IDs must show the policy mismatch; report must describe impact and remediation (principle of least privilege + policy fix + regression test).  
- Intermediate rewards: `list_assets()` + `get_log_events()` reveal the mismatch pattern; candidate finding validated.  
- False-positive guardrail: generic “SQLi/RCE” claims penalised unless evidence supports (it won’t, by design).

**Stretch tasks (post-V1, not for hackathon critical path).**  
[PROPOSAL] Dependency-risk identification (synthetic CVE mapping), error-handling info leak, prioritisation under strict budget, and multi-finding episodes (2 findings) — but only once the validator-safe V1 is shipped.

## OpenEnv compliance blueprint and repo plan

**SECTION 6 — OpenEnv Compliance Blueprint**

[SOURCED] OpenEnv’s core contract is Gymnasium-like APIs (`reset()`, `step()`, `state()`), with type-safe models, packaged behind a FastAPI server and typically accessed via an EnvClient. citeturn18view0turn19search1  

[SOURCED] For environment creators, OpenEnv explicitly supports `openenv init`, and documents a canonical structure: `models.py`, `client.py`, `server/app.py`, `server/<environment>.py`, plus `openenv.yaml` and packaging metadata. citeturn18view0turn18view1  

[SOURCED] OpenEnv provides CLI commands including `openenv init` and `openenv push` for deploying to **Hugging Face Spaces**. citeturn18view0turn17view0  

[SOURCED] The OpenEnv repo’s environment-building guide demonstrates typed models (Action/Observation/State) as Python dataclasses and a `create_fastapi_app(...)` helper to serve the environment. citeturn19search1  

[SOURCED] The OpenEnv repo explicitly warns *not* to copy outdated manifest patterns; current examples use `spec_version`, `type`, `runtime`, `app`, `port`. citeturn19search2turn23view0  

**Validator-sensitive details you must implement (non-negotiable).**  
[PROPOSAL] Based on official requirements + observed validator behaviour:  
- Provide `openenv.yaml` with `spec_version: 1`, `name`, `runtime: fastapi`, `app: server.app:app`, `port: <int>`, and a `tasks:` list with **≥3 tasks each having `id`, `description`, `grader`**. citeturn23view0turn19search2  
- Ensure each task’s final score is **strictly within (0,1)** to avoid fail-fast validation errors. citeturn27search0turn26view0  
- Implement an `inference.py` that prints `[START]/[STEP]/[END]` lines exactly and uses the OpenAI SDK for LLM calls (if any), reading `HF_TOKEN`, `API_BASE_URL`, `MODEL_NAME`. citeturn3view6turn22view1turn22view2  
- Provide a `/health` endpoint that returns 200 once ready (commonly used in examples and deployment docs). citeturn17view0turn20view0  

**Sync vs async.**  
[SOURCED] OpenEnv supports async-first clients with a `.sync()` wrapper for synchronous usage. For hackathon inference scripts, synchronous control flow is often simpler and widely used in examples. citeturn18view0turn22view4  

**What not to copy from older examples.**  
[SOURCED] Some course material shows a simplified `openenv.yaml` (`name/version/description`), but the repo’s skill guidance explicitly warns against outdated manifests; follow the current spec-style manifest used in validated examples. citeturn19search2turn19search11turn23view0  

**SECTION 7 — Repo / File Tree Plan**

[SOURCED] OpenEnv’s scaffold and common community submissions converge on a predictable repository layout and file naming. citeturn18view0turn20view0turn23view0  

[PROPOSAL] Recommended repo structure (submission-ready):

```
secops_evidence_gym/
  openenv.yaml                 # REQUIRED: spec_version, runtime, app, port, tasks+graders
  pyproject.toml               # REQUIRED: package metadata + deps
  README.md                    # REQUIRED: judging narrative + quickstart + safety boundaries
  inference.py                 # REQUIRED: strict stdout logs + OpenAI client usage
  models.py                    # REQUIRED: typed Action/Observation/State dataclasses
  client.py                    # REQUIRED: EnvClient wrapper (sync + async)
  __init__.py                  # REQUIRED: export Env + models for pip install

  server/
    app.py                     # REQUIRED: create_fastapi_app(...) wiring + /health
    environment.py             # REQUIRED: SecOpsEvidenceGymEnvironment(reset/step/state)
    graders.py                 # REQUIRED: grade_easy/medium/hard + safe_reward clamp
    tasks.py                   # OPTIONAL (high-leverage): scenario registry + seed sampling
    safety.py                  # OPTIONAL (high-leverage): tool allowlist + sanitisation helpers
    requirements.txt           # OPTIONAL (if Docker build uses it)
    Dockerfile                 # REQUIRED (practically): HF Spaces docker build

  tests/
    test_api_contract.py       # smoke: reset/step/state doesn’t crash; reward range
    test_graders.py            # unit: deterministic scoring + strict (0,1) clamp
    test_seed_determinism.py   # unit: same seed → same evidence IDs
```

[PROPOSAL] Mandatory for hackathon success: `openenv.yaml`, server app wiring, three tasks+graders, Docker build success, `inference.py` with strict logs, and a README that makes the environment’s value obvious in <60 seconds.

## Reward, grading, and anti-hallucination design

**SECTION 10 — Reward Design**

[SOURCED] OpenEnv leaves reward semantics to the environment; you are responsible for correctness scoring and determinism. citeturn18view0turn19search1  

[SOURCED] Hackathon validation has shown strict “score must be between 0 and 1 (not 0.0 and not 1.0)” behaviour, and teams clamp rewards (e.g., 0.01–0.99). citeturn27search0turn26view0  

[SOURCED] Empirical RL research in other domains (e.g., autonomous racing) shows reward design choices materially affect performance and generalisation, supporting the need for careful shaping rather than a single sparse terminal reward. citeturn15view2  

[PROPOSAL] **Core principle:** correctness is **verifier-gated**, not language-judged. You can optionally add *format/style* checks, but never allow style to dominate correctness reward.

### Reward structure (practical V1)

[PROPOSAL] Normalise the final *task score* into `(0,1)` and keep per-step rewards small enough that summed episode reward stays in `(0,1)` as well (or only final reward is used, depending on your environment semantics). Use a single “score” to satisfy the validator and expose detailed breakdowns in `observation.metadata`.

**Terminal (sparse) components** ✅  
[PROPOSAL]  
- `+0.60` if at least one ground-truth finding is verified and correctly described (type + impact).  
- `+0.15` if the report includes **≥1 valid evidence ID** per finding and those IDs correspond to the right artefacts.  
- `+0.15` if remediation is actionable (specific control, config, test).  
- `-0.40` per hallucinated/unverified finding claimed in the report.  
- `-0.20` if the agent fails to run `validate_finding()` before `submit_report()`.

**Intermediate (dense) components** 🧭  
[PROPOSAL]  
- `+0.02` for discovering a *new* relevant evidence ID (first time only).  
- `+0.03` for creating a well-formed candidate finding that references evidence IDs.  
- `-0.01` per step (efficiency pressure).  
- `-0.03` for repeating the same tool call (exact same args) beyond 2 times.  

**False-positive penalties / anti-hallucination** 🧯  
[PROPOSAL] A “hallucination” is operationally defined as: the report asserts a finding that is not in the environment’s `verified_findings` list. This is easy to compute deterministically and maps directly to your stated goal (“avoid hallucinating findings”).

### Avoiding reward hacking

[PROPOSAL] Hardening rules:  
- Cap rewards from verbosity: extra words do not add points.  
- Make evidence IDs required for high scores (prevents purely rhetorical “security speak”).  
- Penalise calling `validate_finding()` repeatedly without new evidence.  
- Reject “kitchen sink” reporting by penalising extra unverified findings.

### Binary vs shaped reward

[PROPOSAL] **Binary-only** (0/1) will be easy to implement but brittle for multi-step tool use; the agent gets no gradient for *how* to investigate efficiently.  

[PROPOSAL] **Lightly shaped** (recommended) keeps correctness deterministic while providing enough signal to train investigation workflow (evidence collection, validation order, loop avoidance). This mirrors the broader lesson from reward engineering research: shaping and tuning can significantly alter learning outcomes. citeturn15view2  

### Deterministic judge vs hybrid judge

[PROPOSAL]  
- **Strict deterministic judge (recommended V1):** all correctness via verifiers + string/structure checks.  
- **Hybrid (stretch):** add a small LLM-based style score (e.g., clarity), heavily downweighted (≤0.05 of total) and never affecting pass/fail correctness.

## Baseline inference pipeline and strict stdout logging

**SECTION 11 — Baseline Inference Pipeline**

[SOURCED] Hackathon requirements include: a reproducible `inference.py`, the OpenAI client requirement for LLM calls (using provided env vars), and strict stdout logging. citeturn3view6  

[SOURCED] A concrete, hackathon-aligned stdout format has been used by validated submissions (example):  
- `[START] task=<name> env=<benchmark> model=<model_name>`  
- `[STEP] step=<n> action=<str> reward=<0.00> done=<true|false> error=<msg|null>`  
- `[END] task=<name> success=<true|false> steps=<n> score=<0.00> rewards=<r1,r2,...>` citeturn22view1turn22view2  

[SOURCED] The same example inference uses the OpenAI SDK, reading `API_BASE_URL`, `MODEL_NAME`, and `HF_TOKEN`. citeturn22view1turn22view4  

### Responsibilities of `inference.py`

[PROPOSAL] `inference.py` should:  
- read env vars: `HF_TOKEN`, `API_BASE_URL`, `MODEL_NAME`, `ENV_URL` (and optionally `TASK_NAME` override),  
- connect to the env via `.sync()` client,  
- run tasks in a fixed order (easy → medium → hard),  
- execute a bounded number of steps per task,  
- log exactly one `[START]...` per task, one `[END]...` per task, and a `[STEP]...` per environment step,  
- always exit with code 0 (even on failures) and log errors in the `[STEP] error=` field to avoid hard crashes.

### Control flow (V1 baseline strategy)

[PROPOSAL] Use a **hybrid baseline** that is reliable under time constraints:  
- scripted tool sequence per task (fast, deterministic),  
- one LLM call (optional) to draft the final report from gathered evidence (so the demo shows “agentic reasoning”),  
- temperature fixed to 0 for reproducibility (and lower variance).  

[SOURCED] Deterministic inference settings like `TEMPERATURE=0.0` are used in competitive OpenEnv hackathon baselines. citeturn20view0turn22view4  

### Minimum viable baseline (must ship)

[PROPOSAL] For each task:  
1) `reset(task_id=<tier>)`  
2) run 2–4 tool calls that are always relevant (e.g., `check_security_headers`, `search_repo`, etc.)  
3) `create_finding(...)` using evidence IDs  
4) `validate_finding(finding_id)`  
5) `submit_report(report_json)`  

### Stronger baseline (only if time permits)

[PROPOSAL] Add one planning LLM call that chooses among tools based on the alert type, but still keep a hard step limit, and always include verifier validation before reporting.

## Complete build, validation, deployment, and submission pipeline

**SECTION 5 — Complete End-to-End Pipeline**

[SOURCED] This pipeline is built to satisfy both OpenEnv conventions (init/push, typed models, FastAPI server) and hackathon validation constraints (tasks/graders, inference logging, runtime budgets). citeturn18view0turn19search2turn3view6turn22view1  

### Phase goals, deliverables, verification (execution-ready)

[PROPOSAL] The table below is the “do-this-in-order” execution plan. It is intentionally validator-first.

| Phase | Goal | Deliverables | Files touched | Acceptance criteria | Main risks | How to verify |
|---|---|---|---|---|---|---|
| Scope lock | Freeze V1 to 3 tasks + bounded tools | 1-page spec + non-goals | README.md | No pentest/exploit scope; 3 tasks defined | Scope creep | Manual checklist |
| Scaffold | Generate OpenEnv skeleton | Working importable package | openenv.yaml, models.py, client.py, server/* | `python -c "import ..."` succeeds | Wrong template/paths | Local import smoke test |
| Environment core | Implement reset/step/state; tool router | Simulator runs end-to-end | server/environment.py | reset+step returns typed observation; no crashes | Action validation crashes | manual `curl` + python client |
| Tasks + graders | Implement 3 graders + strict (0,1) clamp | `grade_easy/medium/hard` | server/graders.py, openenv.yaml | tasks discoverable; scores strictly in (0,1) | Validator fail-fast | unit tests + manual checks |
| Baseline inference | Make inference reproducible + strict logs | inference.py | inference.py | prints correct `[START]/[STEP]/[END]` | log-parser failure | run script locally |
| Local validation | Run OpenEnv build & validate | passes `openenv validate` | Dockerfile, server/app.py | validate passes locally | port mismatch | `openenv validate --url ...` |
| Docker + HF | Deploy to Spaces | live endpoint | openenv push output | `/health` 200; reset+step works remotely | HF port/env mismatch | curl + python client |
| Submission | Final narrative + demo | polished README + screenshots | README.md | demo works in <2 min | unclear story | run “demo script” |

### Concrete build plan with commands

[SOURCED] OpenEnv supports `openenv init` and `openenv push` and documents this as the standard creator workflow. citeturn18view0turn17view0  
[SOURCED] The OpenEnv course also provides a grounded dev loop: `uv sync`, `uv run server`, `curl /health`, and Docker build/run commands. citeturn17view0  

[PROPOSAL] Commands (copy/paste order):

1) **Scaffold**
```bash
pip install openenv-core
openenv init secops_evidence_gym
cd secops_evidence_gym
```
[SOURCED] `openenv init` is the documented way to scaffold a new environment. citeturn18view0turn18view2  

2) **Local dev install + run**
```bash
uv sync
uv run server
curl http://localhost:8000/health
```
[SOURCED] `uv run server` and `/health` checks are part of the recommended iteration loop in OpenEnv course materials. citeturn17view0  

3) **Implement core files (edit)**
- `models.py`: define `Action/Observation/State` dataclasses  
- `server/environment.py`: implement reset/step/state + tool routing  
- `server/graders.py`: implement `grade_easy/grade_medium/grade_hard` + `safe_reward()`  
- `openenv.yaml`: add `tasks:` with grader import paths  

[SOURCED] OpenEnv’s environment-building guide explicitly directs you to define models and implement `reset/step/state`, then wire a FastAPI app. citeturn19search1  
[SOURCED] A validator-aligned `openenv.yaml` with `spec_version`, `runtime`, `app`, `port`, and `tasks` exists in deep-validation passing examples. citeturn23view0  

4) **Build + validate (local)**
```bash
openenv build
openenv validate --verbose
```
[SOURCED] `openenv build` and `openenv validate` are part of OpenEnv’s recommended validation workflow. citeturn19search2  

5) **Docker build/run smoke test**
```bash
docker build -t secops-evidence-gym:latest -f server/Dockerfile .
docker run -p 8000:8000 secops-evidence-gym:latest
curl http://localhost:8000/health
```
[SOURCED] This `docker build -f server/Dockerfile .` pattern is directly shown in OpenEnv deployment course material. citeturn17view0  

6) **Run inference locally**
```bash
export HF_TOKEN="..."
export API_BASE_URL="..."
export MODEL_NAME="..."
export ENV_URL="http://localhost:8000"
python inference.py
```
[SOURCED] These env var names and OpenAI SDK usage are consistent with hackathon guidance and existing inference implementations. citeturn3view6turn22view4  

7) **Deploy to Hugging Face Spaces**
```bash
openenv push --repo-id <your-hf-username>/secops-evidence-gym
```
[SOURCED] `openenv push` is described as the fastest path to deploy to **Hugging Face Spaces**. citeturn17view0turn18view0  

### Testing and validation plan (high-signal)

[SOURCED] OpenEnv stresses predictable API behaviour and type-safe contracts; hackathon validation is fail-fast. citeturn18view0turn27search0  

[PROPOSAL] Test layers (in priority order):  
- **API contract smoke tests:** reset/step/state return valid JSON; never crash on invalid tool name (should return an observation with an error field).  
- **Grader tests:** for each task, verify (a) correctness cases score high, (b) hallucination cases score low, (c) score always ∈ (0,1).  
- **Seed determinism tests:** same `seed` produces same evidence IDs and same verifier outputs.  
- **Runtime test:** run `inference.py` end-to-end and assert wall-clock < 2 minutes locally; assume < 20 minutes on grader infra even with cold starts. citeturn3view6turn22view4  
- **Reward sanity tests:** ensure reward increases monotonically with verified correctness; fails if verbosity alone increases reward.

## Submission packaging, execution roadmap, real-world usefulness, and failure modes

**SECTION 14 — README / Demo / Submission Narrative**  
[SOURCED] Judges likely assess both the environment’s technical correctness (programmatic checks) and qualitative merit (LLM scoring / narrative). citeturn3view7  

[PROPOSAL] README structure that “feels like a winner” 🏆:  
- **Hero block:** one-paragraph pitch + why it’s real-world + safety claim.  
- **Two-minute demo:** copy/paste commands + expected output snippet with `[START]/[STEP]/[END]`.  
- **Environment contract:** action schema, observation schema, task list.  
- **Grading:** explain deterministic verifiers + hallucination penalties.  
- **Safety & isolation:** explicit exclusions (no egress, no shell, synthetic artefacts).  
- **Real-world relevance:** how this benchmarks/reporting maps to security workflows (triage, evidence, remediation).  
- **Screenshots:** web UI (optional) + an evidence trace + one scored report example.  

**SECTION 15 — Project Management Plan**  
[PROPOSAL] Day-by-day (assuming a hackathon-style sprint):

- **Day 0 (scope lock + scaffold):** environment skeleton, `openenv.yaml` with 3 tasks, stub graders returning 0.5 (clamped), server runs locally.  
- **Day 1 (determinism + validator):** implement scenario generator, evidence registry, verifiers, and strict (0,1) scoring; pass `openenv validate`.  
- **Day 2 (baseline + polish):** implement `inference.py` strict logs; deploy to Spaces; polish README + demo artefacts.

[PROPOSAL] Critical path: `openenv.yaml tasks+graders` → grader clamp `(0,1)` → inference stdout format → Docker+Spaces deployment. (Everything else is secondary.)

**SECTION 16 — Real-World Usefulness Plan**  
[SOURCED] NIST’s testing guide emphasises planning, conducting tests, analysing findings, and developing mitigation strategies; your environment’s “evidence → remediation” focus aligns with that lifecycle without requiring offensive exploitation. citeturn29search8turn29search0  

[PROPOSAL] Who would care after the hackathon:  
- security engineering teams evaluating agentic “triage + reporting” reliability,  
- LLM tooling teams wanting benchmarks for **non-hallucinating, evidence-grounded** outputs,  
- training teams building safe cyber ranges (without weaponisation).

[PROPOSAL] Post-hackathon upgrades (highest leverage):  
- export trajectories as JSONL for offline training,  
- add more scenario families (still safe) and a held-out split for generalisation,  
- integrate with RL trainers (e.g., TRL’s OpenEnv integration) to show real training curves. citeturn19search6turn10view0  

[SOURCED] PenGym provides evidence that realism/faithfulness of environments can affect transfer and stability when moving from simulation to more realistic settings—so you should roadmap a “higher fidelity mode” (still safe) later, not in V1. citeturn15view0  

**SECTION 17 — Why the naive version would fail**  
[PROPOSAL] Top failure patterns (and why they kill submissions):  
- Too broad (full cyber range, live services): fails time/infra constraints. citeturn3view6turn10view0  
- Fuzzy grading (LLM-only judging): non-deterministic, easy to game.  
- Unbounded tools (shell/network): unsafe + untrainable action space.  
- Scores at exactly 0.0 or 1.0: fail-fast “out of range” validator. citeturn27search0turn26view0  
- Inference logs not parseable: phase-1 failure even if env is good. citeturn3view6turn22view1  
- Port / health issues on Spaces: container “works locally” but fails remotely. citeturn17view0turn20view0  

**SECTION 18 — Final Recommendation**

[PROPOSAL] **What should you build?**  
Build **SecOps Evidence Gym**: a deterministic, safe, sandbox-only cyber analyst environment focused on evidence collection, verifier validation, and remediation reporting.

[PROPOSAL] **What should V1 include? (minimum winning set)**  
- OpenEnv-compliant FastAPI env with typed models and `reset/step/state`. citeturn18view0turn19search1  
- `openenv.yaml` with **3 tasks + graders**. citeturn23view0turn3view6  
- Deterministic verifiers + strict score clamp to `(0,1)`. citeturn27search0turn26view0  
- Baseline `inference.py` with strict `[START]/[STEP]/[END]` logging + OpenAI SDK usage for any LLM calls. citeturn3view6turn22view1turn22view4  
- HF Spaces deployment with a working `/health`. citeturn17view0turn20view0  

[PROPOSAL] **What should you cut?**  
- Any real pentesting/offensive content, any arbitrary command execution, any live targets, any correctness scoring via an LLM judge.

[PROPOSAL] **Top 5 implementation decisions that matter most**  
1) Validator-safe `openenv.yaml` tasks+graders wiring. citeturn23view0  
2) Score/range compliance: clamp to `(0,1)` everywhere. citeturn27search0turn26view0  
3) Strict stdout format in `inference.py`. citeturn22view1turn22view2  
4) Deterministic verifiers as the source of truth.  
5) Bounded tool set (≤8 tools) with anti-loop penalties.

[PROPOSAL] **Minimum viable winning submission**  
A V1 with 3 tasks, deterministic graders, bounded tools, strict inference logging, and a polished README + demo trace.

[PROPOSAL] **Minimum viable real-world useful submission**  
The same V1, plus: seed determinism, trajectory export, and a clear “how to add new scenarios” contributor guide.

[PROPOSAL] **If you only have time for 20% of ambition—do this exact 20%:**  
- Implement **one** robust multi-step loop (tools → validate → report)  
- Implement **exactly 3** tasks (easy/medium/hard)  
- Make graders deterministic and validator-safe  
- Make deployment + inference bulletproof  
Everything else is stretch.

**Confidence (my estimate): 8.4/10** ✅🔥

## Sources and credibility ratings (with exact links)

[SOURCED] Ratings are my judgement of authority + relevance for this hackathon context (0–10). URLs are provided verbatim in code form.

### Tier 1 (official OpenEnv + hackathon dashboard)
- Credibility **9.5/10** — `https://github.com/meta-pytorch/OpenEnv` citeturn18view0  
- Credibility **9.0/10** — `https://github.com/meta-pytorch/OpenEnv/blob/main/envs/README.md` citeturn19search1  
- Credibility **8.5/10** — `https://github.com/meta-pytorch/OpenEnv/blob/main/.claude/skills/generate-openenv-env/SKILL.md` citeturn19search2  
- Credibility **9.0/10** — `https://www.scaler.com/school-of-technology/meta-pytorch-hackathon/dashboard` citeturn1view0turn3view6turn3view7  

### Tier 2 (strong community exemplars)
- Credibility **8.5/10** — `https://github.com/sid-rp/kube-sre-gym` citeturn10view0  
- Credibility **8.0/10** — `https://huggingface.co/openenv-community` citeturn14view0  
- Credibility **7.5/10** — `https://github.com/Harikishanth/Incident-Triage-Environment` citeturn20view0turn23view0turn22view1  

### Tier 3 (peer-reviewed / primary references for design constraints)
- Credibility **8.5/10** — PenGym (Computers & Security, open access): `https://www.sciencedirect.com/science/article/pii/S0167404824004450` citeturn15view0  
- Credibility **8.0/10** — Reward design + generalisation (Scientific Reports, 2025): `https://www.nature.com/articles/s41598-025-27702-6` citeturn15view2  
- Credibility **8.5/10** — AMaze (JOSS, 2025): `https://joss.theoj.org/papers/10.21105/joss.07208` citeturn16search7  
- Credibility **9.5/10** — NIST SP 800-115: `https://csrc.nist.gov/pubs/sp/800/115/final` citeturn29search8  
- Credibility **9.0/10** — NIST “Cyber Range: A Guide” (PDF landing): `https://www.nist.gov/document/cyber-range` citeturn29search1  
- Credibility **7.5/10** — “Cybersecurity of Cyber Ranges: Threats and Mitigations” (IJISR, 2022 PDF): `https://infonomics-society.org/wp-content/uploads/Cybersecurity-of-Cyber-Ranges.pdf` citeturn29search2  

### Tier 4 (useful validator “ground truth” signals from the field)
- Credibility **6.5/10** — Validator failure mode discussion (score must be strictly between 0 and 1): `https://www.reddit.com/r/pytorch/comments/1shi767/meta_x_pytorch_x_sst_x_openenv_hackathon_phase_2/` citeturn27search0  
- Credibility **7.0/10** — Strict logging format reference via a verified submission’s `inference.py`: `https://github.com/Harikishanth/Incident-Triage-Environment/blob/main/inference.py` citeturn22view1turn22view2  

### Uploaded reference you provided
- Credibility **7.0/10** (useful as a design draft; not independently authoritative) — `deep-research-report (2).md` fileciteturn2file0