# teaching an llm to sre: EnterpriseHPC-v0 on openenv

tl;dr we shipped an openenv compliant gymnasium environment that
simulates a 224 core rocky linux hpc cluster inside a single user
namespace sandbox, resets in **2.40 ms p50**, and trains
**Qwen/Qwen2.5-Coder-7B-Instruct** with trl grpo to recover a broken cluster
end to end. the same training script can run locally, in colab, or
against a fleet of hf spaces via `--env-urls`.

## why

the slowest, highest stakes work in enterprise infra is multi-app
incident response. an open ondemand portal returns 502. the compute
partition is drained. there is a failing slurmd somewhere. to fix it
you navigate login -> compute-01 over ssh, inspect route configs and
munge keys, restart services in the right order, and verify via curl.
frontier llms have never trained on that loop.

EnterpriseHPC-v0 turns that loop into an rl environment.

## what is inside

- nested bwrap for lateral movement. `ssh compute-01` chroots the
  shell into a separate rootfs so `hostname` and filesystem paths
  reflect the new node
- fuse-overlayfs with upperdir and workdir on `/dev/shm` for
  microsecond copy on write. kernel overlay and a copy fallback are
  supported for hosts without fuse privileges
- a deterministic slurm state machine in
  `/mnt/shared/slurm_state.json` with fcntl locks so many parallel
  rollouts cannot corrupt each other
- python stubs for sinfo, squeue, systemctl, scontrol, curl, ssh that
  read and mutate the json state, and a lightweight open ondemand
  http server that returns 502 until the underlying fault is fixed
- three scenarios ship today and are rotated per rollout
  - `hpc_outage` compute-01 drain from a broken route-eth0
  - `hpc_munge` compute-01 drain from a munge key with wrong mode and
    a broken route (chained)
  - `hpc_pid_stale` slurmd refuses to restart after reboot because of a
    leftover `/var/run/slurmd.pid`
- the gymnasium env `EnterpriseHPC-v0` wraps it all with pexpect so
  the policy experiences real interactive bash prompts

## how fast

```
| mount | n | p50 ms | p95 ms | p99 ms | max ms |
| --- | ---: | ---: | ---: | ---: | ---: |
| copy | 100 | 2.40 | 2.56 | 2.58 | 2.87 |
```

that is in the ci friendly copy mode. real fuse-overlayfs on a linux
host drops well under 1 ms. reset latency is no longer the grpo
bottleneck.

## training with qwen2.5-coder

local training with unsloth + 4bit qlora:

```
python -m training.train_hpc_outage \
  --model Qwen/Qwen2.5-Coder-7B-Instruct \
  --group-size 4 --max-turns 12 \
  --num-train-steps 100 \
  --scenarios hpc_outage,hpc_munge,hpc_pid_stale
```

remote training against hosted openenv spaces (same shape as the
trl + openenv launch example, swapped to a code-tuned 7b policy):

```
python -m training.hpc_openenv_gemma \
  --env-urls https://<user>-enterprise-hpc-openenv.hf.space \
             https://<user>-enterprise-hpc-openenv-2.hf.space \
  --model Qwen/Qwen2.5-Coder-7B-Instruct \
  --group-size 4 --max-turns 12 --num-train-steps 200
```

submit to hf jobs:

```
python -m training.hf_jobs \
  --env-urls https://<user>-enterprise-hpc-openenv.hf.space \
  --gpu a10g-large \
  --num-train-steps 300
```

the training scripts use unsloth for 4bit qlora loading and trl
`GRPOTrainer` with a custom rollout function that drives the env one
turn at a time. the reward is binary from the deterministic task
grader, which is exactly the signal grpo wants.

a colab notebook at `training/hpc_colab.ipynb` runs both the local
and remote paths on a single t4 / l4 / a100.

## what the agent learns

before training a random policy wanders around `sinfo` and never edits
the route file. after ~100 steps of grpo the agent reliably:

1. runs `sinfo` and `squeue` to locate the drained node
2. lateral moves with `ssh compute-01`
3. inspects `/etc/sysconfig/network-scripts/route-eth0`
4. writes the correct route with `printf ... >` (no heredocs allowed)
5. for the munge variant also `chmod 0400 /etc/munge/munge.key`
6. restarts munge then slurmd in that order
7. exits back to login and verifies with `curl -I http://localhost:8080`

## prove it is solvable

before any training, reviewers can run:

```
make gold   # deterministic gold-trajectory verifier
make eval   # gold vs random vs bad policies, writes runs/eval/leaderboard.md
make bench  # reset-latency benchmark
```

## try it

- repo: https://github.com/your-org/low-taper-fade-openenv-scaler
- hf space (env server): https://huggingface.co/spaces/your-org/enterprise-hpc-openenv
- colab: `training/hpc_colab.ipynb`
- pitch doc: `docs/pitch.md`
- hf jobs guide: `docs/hf_jobs.md`
- spaces deploy: `docs/hf_spaces_deploy.md`