docs/ARCHITECTURE.md

Architecture

The single insight that ties the whole pitch together: each tier escalates a different dimension — compute (Basic) → horizon (Advanced) → realism (Max) — not just scenario count. If you only read one paragraph of this document, read this one.

This document explains why sre-gym ships three tiers, what each tier defends as a research question, and how the package is structured to make that defence visible from a 30-second skim of the repo.

---

1. The dimensional-escalation insight

The standard pattern in agentic-LLM evaluation, surveyed across SWE-bench Lite/Verified/Pro, MLE-bench Low/Med/High, ITBench-static/live, WebArena/-Verified/-Hard, and CRMArena/-Pro, is to escalate a single axis (volume, complexity, dataset size, horizon length) across difficulty bands. That works for benchmarks whose underlying capability is one-dimensional. SRE is not one-dimensional.

A junior on-call engineer learning to triage faces a fundamentally different bottleneck (cognitive efficiency under tight context) than a senior SRE running a multi-incident postmortem (state tracking across long horizons), which faces a fundamentally different bottleneck than an enterprise platform team operating against an actively chaos-engineered cluster (operating in a partially-observable, adversarial, irreversible world). Their training signals, episode shapes, observation richness, and reward structures should not look the same.

sre-gym takes that observation seriously and stratifies its tiers along the dimension that actually limits the persona's training loop:

Tier	Bottleneck	Persona implication	Episode shape
Basic	Compute ($30 of HF credits, 1 A100 ~12h)	Pre-digested observations, dense reward shaping, 8K context, 11-action space	8–13 ticks
Advanced	Horizon ($300–500 budget, 1–2 A100-days)	Multi-incident chains, partial observability, 28-action space, on-call peer	60–90 ticks
Max	Realism (8×A100/H100, real chaos eng)	Ephemeral docker-compose / k3d, real kubectl / Vercel / Stripe APIs, subprocess shell, learned-critic rewards	110–180+ actions, real wall-clock minutes

This framing is defensible as research: the claim is that "training on a tier-1 environment that's causally rich but compute-cheap will produce a 3B specialist that beats Haiku on incident triage" is testable in 12 hours. The claim that "training on a tier-3 environment that includes real subprocess-shell access will produce an SRE agent that can actually go on-call" is testable in months and tens of thousands of dollars. Those are different research questions and the environment should make them visibly different.

---

2. Repository layout, with the design intent annotated

sre-enginnerllm/
├── sre_gym/                          # Tier-aware public package
│   ├── env.py                        # SREGym(tier=...) factory; Basic delegates,
│   │                                 # Advanced/Max raise TierNotRunnableError
│   │                                 # carrying a docs pointer.
│   ├── tier.py                       # Tier enum + TierConfig; the
│   │                                 # escalation_dimension field is the
│   │                                 # load-bearing piece of the pitch.
│   ├── advanced/scenarios/*.yaml     # 3 reference scenarios, real YAML, real
│   │                                 # topology, real reward dimensions, real
│   │                                 # reference traces — but the simulator
│   │                                 # backing them is intentionally not built.
│   └── max/                          # 1 fully-specced family, real
│       ├── families/*.yaml           # docker-compose, real chaos library,
│       ├── chaos/*.yaml              # real workload generator config.
│       └── compose/*.yaml            # Provisioning is left to the operator.
│
└── unified_incident_env/             # Basic-tier core; the v2 surface that the
                                      # HF Space serves and openenv.yaml declares.

Two design choices worth calling out:

(a) Basic delegates to unified_incident_env rather than reimplementing. The Basic tier's runnable surface is the existing v2 environment, kept verbatim — including its 36+ test suite, scripted-optimal baselines, and HF Space deployment. The sre_gym wrapper is intentionally thin: it adds the tier flag, the introspection methods, and the YAML-spec loader for the design-only tiers. This is the difference between "a single-tier env that's hard to extend" and "an env that visibly carries the three-tier story while still passing every Basic-tier test it ever passed."

(b) Advanced and Max are shipped as data, not as code. The YAML specs in sre_gym/advanced/scenarios/ and sre_gym/max/families/ are real: they reference real topologies, real action sets, real reward dimensions, real chaos patterns, and real reference traces. They're loaded by SREGym.list_scenarios() and renderable by the playground. What's missing is the simulator backing them — and that's deliberate. Building a credible Max-tier simulator in 36 hours is a fantasy; building a credible Max-tier spec that downstream operators can lift into a $40-150/day cluster is achievable. The cost of pretending Max is shipped runnable would be zero credibility with judges who have run real chaos engineering. The cost of shipping it as a credible vision is one extra YAML file.

---

3. The Basic tier in detail

The Basic tier is the only tier we trained against, so it gets the most concrete defence.

3.1 What it is

12 base templates × 5 procgen variants = 72 deterministic scenarios. Procgen jitters metric values, deploy timestamps, and noise-service rotation while preserving the causal structure — so a trained agent can't memorize fingerprints. Holding out one variant per template gives a 60-train / 12-eval split.

3.2 The 12 templates and what each one teaches

#	Template	Skill	Decoy / red herring
1	worker_deploy_cascade	deploy-history reasoning	none — the easy entry point
2	db_config_rollout	config-vs-code disambiguation	concurrent worker deploy
3	gateway_auth_rollout	wrong-loud-service trap	worker queue-depth alert
4	payment_webhook_misconfig	downstream symptom (Stripe)	DB write-rate drop
5	schema_drift_missing_migration	application vs DB blame	DB looks healthy
6	cache_stale_state	metrics-look-good-but-customers-don't	cache hit rate is up
7	dep_degradation	"your service vs theirs"	worker CPU is loud
8	memory_leak_oom	restart count > error count	DB CPU spikes look like DB fault
9	auth_token_expiry	cross-service credential propagation	gateway is the loudest service
10	network_partition	trust connectivity, not self-reports	cache reports healthy in own metrics
11	rate_limit_retry_storm	counterintuitive (more retries = worse)	DB CPU/connections look pathological
12	migration_lock	lock contention without crash	worker errors look like a worker fault

Each template contains a different cognitive failure mode. A 12-template catalogue with 12 different failure modes is a denser training signal than a 60-template catalogue that all reduce to "look at the deploy that just happened" — that's the depth-not-quantity argument.

3.3 Why these compute knobs

• 8K context — fits the entire Basic episode (12 ticks × ~600 tokens of observation + ~80 tokens of action) inside the trained model's working set without truncation. Letting the trajectory spill into 16K context would force a smaller model (more A100 hours per token) or a longer training run (more wall-clock).
• 11 actions — small enough that GRPO group-relative advantages converge in 600–1000 steps. Add 5 more actions and the policy has 50% more dimensions to explore at the same compute budget, which usually means a noisier gradient.
• 5-component dense reward — recovery + containment + verification + impact + efficiency, with shaped intermediate signal (potential-function differences). Pure terminal rewards converge slower than dense shaping at the same compute budget; that's a well-known result and we're paying it.
• 12 templates × 5 variants procgen — enough variety that a held-out variant is genuinely held-out, but not so much that scenario-specific overfitting eats the training budget.

These knobs collectively are the "compute" in "compute-bounded". Tighten any of them and a Series-A-class operator would call it out as theatre.

3.4 What "compute-bounded" actually means in numbers

A representative training run looks like:

Phase	Steps	Compute	Wall-clock on A100 40GB	Output
Seed dataset build	200 (Claude-driven)	~$15 of API spend	2h	train/data/seed_combined.jsonl
SFT cold start	500 steps, batch 4	~3h	3h	LoRA r=64 adapter
GRPO online	800 steps, K=4 rollouts	~6h	6h	trained adapter
Eval sweep	36 episodes (3 per template)	~30min	30min	eval/results/comparison.csv
Total		~$15 API + 12h GPU	12h	trained 3B + comparison table

That's the $30-of-HF-credits budget the design targets, comfortably.

---

4. The Advanced tier in detail

4.1 The horizon escalation, formalized

Advanced is bounded by horizon: episodes are 60–90 ticks instead of 12, multi-incident chains span 5+ minutes of simulated time, and the agent must track state that no single 8K context window can hold. Three properties make this a different research question:

1. Multi-incident composition. One template's resolution can become another template's setup state. Scenario 1 (cascading_release_train.yaml) is the canonical example: rolling back the gateway is correct, but it triggers a downstream worker drift that materializes 25 ticks later. The agent has to recognize the chained incident as caused by their own fix and reach for a second rollback rather than treating it as a fresh outage.
2. Partial-observability noise. Sometimes query_logs returns degraded data because the logging pipeline is the affected service. Scenario 2 (observability_pipeline_outage.yaml) is the canonical example: the agent must drop log sampling and toggle verbose-logging off before attempting root-cause diagnosis, because the diagnostic tool is itself broken.
3. Cross-domain reasoning. Scenario 3 (supabase_rls_silent_leak.yaml) is a reliability incident with a security root cause. The agent must classify it correctly (reach for escalate_security rather than the platform on-call), contain the data leak via feature flag before rolling back, and produce a postmortem with a leak-window calculation. No existing SRE benchmark scores cross-domain reasoning.

4.2 Why this isn't trained in this repo

A faithful Advanced simulator would need:

• a 15–20 service event-loop simulator (not a 4-service one)
• multi-tick fault propagation (one fix triggering a chained fault N ticks later, with proper causal latency)
• a synthetic on-call-peer model that responds to escalations
• ~28 action handlers, vs. 11 in Basic
• a learned-critic reward path for postmortem quality

That's roughly 2 weeks of focused engineering and 1–2 A100-days of training. Both are out of scope for the 36-hour hackathon window. We ship the design at the YAML level so that a downstream operator with the budget can lift it; we do not pretend it was trained.

---

5. The Max tier in detail

5.1 The realism escalation, formalized

Max is bounded by realism. The world stops being a simulator. A reset() provisions a fresh 22-service docker-compose stack (Vercel + Supabase + Stripe + Postgres + Redis + Kafka + 3 worker pools + observability stubs + chaos controller). The agent's rollback_deploy is a real kubectl rollout undo against that stack. query_logs reads from a real Loki/Promtail pipeline. query_traces reads from a real Tempo cluster. Faults are injected via a real Chaos-Mesh-style chaos library. Reward is computed from the actual recovery state of the actual stack.

Three properties are unique to Max:

1. Real subprocess access. The agent has a sandboxed shell. It can write code, commit it to a sandboxed git mirror, push, watch CI, observe a deploy, roll back. This is the "real hard work instead of exploiting shortcuts" spec from Theme #3.1 of the OpenEnv brief.
2. Real action irreversibility. A real Stripe refund is a real Stripe refund. The cluster is destroyed on next reset, but actions taken at the application layer are real.
3. Outcome-scored rewards. A second small model evaluates the agent's postmortem against the actual recovery trajectory. Reward is no longer a deterministic rubric — it's an outcome judgement, with shaping signals as auxiliary.

5.2 Why one family, not 30

Per the design rationale: one fully-specced family (e-commerce + Stripe + Supabase + Vercel) with compose.max.yaml, an 11-pattern chaos library, a workload generator config, a reference instance with an expected 110-action trajectory, and operator notes for cost / isolation / safety is more credible than a vague "30+ scenario families" claim. Judges who run real infrastructure would ask "show me one"; if you can't, the whole tier framing collapses. The ecommerce_vibecoded_saas family is the show-me-one.

5.3 What's deliberately not in this repo

• The published stub images (ghcr.io/sre-gym/*) — publishing them is a $1–2k registry-cost commitment that doesn't fit the hackathon-window budget.
• A running cluster — bringing the Max tier up costs $40–150/day depending on cluster size and chaos cadence.
• A trained model against Max — that's a multi-week, multi-A100 commitment.

What is in this repo is the spec at the level of detail a downstream operator can actually act on: docker-compose, chaos-library YAML, workload generator config, family-level scenario population spec, and operator-notes block covering cost, isolation, and reset safety.

---

6. The contract that ties Basic, Advanced, and Max together

All three tiers share the same five abstract objects:

Object	Basic concrete	Advanced concrete	Max concrete
Topology	4 services hard-coded	15–20 services in YAML	22 services in docker-compose
Action set	11, Pydantic-validated	28, validated against allowed_actions: list	50+, including subprocess shell
Observation	pre-digested fields	noisy multi-source feed	raw Prometheus / Loki / Tempo
Reward	7-dim deterministic rubric	rubric + chained-incident bonus + postmortem critic	outcome + learned-critic + IaC-remediation bonus
Episode	8–13 ticks	60–90 ticks	unbounded (real wall-clock minutes)

Same shape, escalating depth. That's what makes the tier story coherent rather than three unrelated environments stacked in one repo.

---

7. OpenEnv framework integration

Basic uses:

• openenv.core.env_server.Environment[A, O, S] base class
• Typed Pydantic Action / Observation / State
• /reset /step /state HTTP endpoints via create_fastapi_app
• max_concurrent_envs for batched rollouts (the GRPO contract)
• Custom /tasks /baseline /grader /status /health extension routes for scenario-catalog-and-grader introspection

Advanced and Max would extend this with:

• MCPEnvironment base + @self.tool()-registered actions for production serving
• WebSocket /ws transport for low-latency multi-agent rollouts
• ServerMode.SIMULATION vs ServerMode.PRODUCTION switch for tier-3 real-API actions
• Custom Gradio TabbedInterface with a "topology inspector" tab

These are documented in the per-tier docs but not implemented here.

---

8. The judging-criteria mapping

The OpenEnv hackathon's published rubric weights are: Innovation 40, Storytelling 30, Reward Curves 20, Reward/Pipeline 10. sre-gym is built so each weight has a concrete corresponding artifact:

Weight	Artifact in this repo
Innovation 40%	The dimensional-escalation tier story, defensible across compute / horizon / realism axes.
Storytelling 30%	This document + README.md first paragraph + 12-template skill table + reference traces in YAMLs.
Reward curves 20%	eval/results/qwen25_7b_comparison_* (populated by cell 10/11 of 01_triage_train_grpo_qwen25_7b.ipynb) + train/data/eval_sweep_baselines.jsonl reference numbers.
Reward/Pipeline 10%	Composable rubric in unified_incident_env/server/grader.py + SFT → GRPO loop in 01_triage_train_grpo_qwen25_7b.ipynb.

Weight by weight, the artifact you'd hand a judge to defend that score is in this repo.