It's 3 AM. Payments are failing. Can your AI keep its cool?
There is a moment in every on-call shift that you remember. Ours was a Tuesday at 3:14 AM. Pager screaming. Auth service flapping. Payments queue backing up. Stakeholders on the bridge asking the same question every two minutes: "What is happening?"
We did what every SRE does. Opened five tabs. Pulled metrics. Tailed logs. Compared against an outdated runbook. Pinged the database team. Tried a restart. It sort of worked. We patched it, wrote a postmortem the next morning, and life moved on.
A year later we started building agents on top of LLMs and noticed something uncomfortable: the models that could solve LeetCode in a few seconds would fall apart the moment they had to hold a multi-step investigation in their head. They would jump to a fix before reading the alert. They would trust the knowledge base when the knowledge base was wrong. They would tell the customer "we're working on it" while the database was actively melting.
That gap is what this project is about.
The thing we built
It is called EICC - Enterprise Incident Command Center. It is an OpenEnv environment that turns a real on-call shift into a training ground for LLM agents. Not a quiz, not a benchmark with a single right answer - a world that punches back.
You wake the agent up with a customer ticket. Behind that ticket is a mesh of five microservices that can fail and cascade into each other, eight enterprise tools the agent has to learn how to use (monitoring, CRM, billing, knowledge base, policy engine, incident history, runbooks, stakeholder manager), and a clock that does not stop just because the agent is confused.
The agent cannot see the truth. The root cause is hidden. The KB is sometimes outdated. The policies have drifted since the last training data. New tickets keep arriving while it is still investigating the first one. If it shouts "I have fixed it!" at the wrong moment, patience drops, customers escalate, and the reward turns negative.
Every action is one of 21 typed actions across four phases: triage, investigation, response, resolution. Reward are given by our RL engine to the LLM. The same seed always produces the same scenario, so we can actually measure whether the agent got better.
Why this is a "Theme #3.1 World Modeling - Professional Tasks" submission
The hackathon brief for sub-theme #3.1 asked for environments that require real interaction with tools, APIs, or dynamic systems where the model is expected to do real hard work instead of exploiting short-cuts, where agents have to maintain consistent internal state, update beliefs based on outcomes, and orchestrate multi-step workflows, with the goal of strengthening causal reasoning and persistent world models. Expected outcome: an environment capturing nuances of a defined partially observable world and improving LLM interaction with it.
That is the brief we designed to. EICC is not a single-turn benchmark. The root cause is hidden. The KB lies sometimes. Policies drift. The service mesh has real causal structure - hit the wrong node and the blame propagates. The agent has to call tools, read the returns, update what it thinks is going on, and pick the next action - over and over, for up to 80 steps in the hardest tier. There are 11 explicit anti-shortcut mechanisms baked into the reward (phase gating, investigation-before-action, KB cross-verification, blast-radius penalties, CAB approvals for risky changes, resource budgets, tone matching, ...) so the model cannot game its way to a good score.
And the "real systems" requirement is taken literally in Mode 3: the exact same agent, the exact same action API, but routed to a live 5-service Docker cluster with a chaos controller. When the agent says "restart auth," something actually restarts.
Three ways to play
Most environments give you one mode. EICC gives you three, because training and demoing have different needs.
Ticket mode is the most basic one: classify, route, respond, resolve. It exists for backward compatibility and as a sanity ramp.
Mock environment is where training happens, but it is mock because it tries to replicate the database, auth service, etc. using python functions - no real scenarios. It is a deterministic simulation of the whole world: services, tools, customers, stakeholder patience. It is fast, free, and reproducible. We train GRPO on top of
Qwen2.5-3B-Instructhere using Unsloth + TRL.VM environment is our USP. It replicates the environment as close as possible to real-life conditions. The same
/resetand/stepAPI, but now backed by a real Docker cluster of five microservices plus a chaos controller. When the agent says "restart the auth service," something actually restarts. When it says "verify_fix," we hit a real/healthendpoint. Sim-to-sandbox transfer scoring tells us how much of what was learned in the cheap simulation actually carries over to live infrastructure. There is a deterministic drill mode that injects fresh failures mid-episode so we can score recovery quality, not just lucky first hits.
That last bit - the same agent, scored on simulation and on a live cluster, with a number that says "this much of your learning transferred" - is the part we are most proud of.
What the agent has to learn
The reward is deliberately mean. It rewards behavior that real incident commanders use, not behavior that looks confident:
- Investigate before acting. Apply a fix without checking monitoring, you lose points.
- Cross-verify the KB. Trust an outdated runbook, blast radius hits customers, you lose points.
- Respect policy drift. Push a risky change without a CAB approval gate, you lose points.
- Match tone. Be cheerful at a furious enterprise customer, you lose points.
- Don't spam stakeholders. Notify before you have facts, patience drops faster.
- Keep the JSON clean. Long, rambling generations get penalized.
There are eight tracked behavioral skills behind the curtain: investigation-before-action, KB cross-verification, policy checking, stakeholder proactivity, root-cause accuracy, tone matching, resource efficiency, and red-herring dismissal. The judges (and you, in the notebook) can read all eight per run, side by side, baseline vs trained.
Baseline vs trained: who is on each side of the curve
Every plot in this submission has two lines. They look like just "before" and "after," but the actual identity of each side matters.
The baseline is the untrained Qwen2.5-3B-Instruct walking
into the incident world cold. No exposure to our reward function, no
LoRA adapter, nothing - just the off-the-shelf instruction-tuned model
trying to fight a fire it has never seen. Most of the time it does
what you would expect a smart-but-naive engineer to do: jump to a fix,
miss a step, send a confident customer email while the database is
still on fire.
The trained side is the same model, same prompts, same scenarios, same seeds - only the weights are different. The LoRA adapter on top has been updated by GRPO using the rewards from the environment in this repo. Nothing else changed. So when the trained line sits above the baseline line, that gap is exactly the behavior our environment taught the model. Not a different model. Not a different prompt. Just learning.
We keep that distinction visible in the artifacts on purpose. Every
trained_report.json carries a policy_used field
(trained_checkpoint if the LoRA adapter loaded, trained_heuristic
as a guarded fallback). If a number ever looks too good, you can read
one line of JSON and find out which policy actually produced it.
No magic, no asterisks.
What the curves actually show
After training, three plots fall out - one for easy, one for medium, one for hard. Each plot has two lines, baseline and trained. The gap between the two lines is the part that matters: it is the behavior the model picked up that the base model did not have.
You can open the actual files we shipped under results/
in the repo. results/simple/ is the Mock-environment run.
results/sandbox/ is the same trained checkpoint scored again on the
live container cluster. results/training/reward_history.json is the
training-time reward signal from Phase 1. No re-running required -
just click and read.
The thing we learned building this
The first version of the env was too easy. The agent learned to shortcut: skip investigation, guess the fix, take the points, move on. So we added partial observability. Then it learned to spam tools. So we added a resource budget. Then it learned to be polite to everyone. So we added tone matching against actual sentiment. Each shortcut closed produced a better agent.
That is what an environment is supposed to do. The design of the world is the curriculum.
Try it yourself
The HF Space is live. The training notebook walks through both the Mock lane and the VM lane on a Colab GPU. Reproducing the headline numbers takes about an hour on an A10. Pull the repo, open the notebook, follow the steps. The first time the trained reward curve crosses the baseline curve on the hard difficulty plot, you'll know exactly why we built this.
Built on OpenEnv. Trained with GRPO via Unsloth + TRL. Deployed on Hugging Face Spaces. Source and notebook are public - links in the repo README.