Spaces:
Paused
Paused
File size: 4,654 Bytes
bc35a94 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | # teaching an llm to sre: EnterpriseHPC-v0 on openenv
tl;dr we shipped an openenv compliant gymnasium environment that
simulates a 224 core rocky linux hpc cluster inside a single user
namespace sandbox, resets in **2.40 ms p50**, and trains
**Qwen/Qwen2.5-Coder-7B-Instruct** with trl grpo to recover a broken cluster
end to end. the same training script can run locally, in colab, or
against a fleet of hf spaces via `--env-urls`.
## why
the slowest, highest stakes work in enterprise infra is multi-app
incident response. an open ondemand portal returns 502. the compute
partition is drained. there is a failing slurmd somewhere. to fix it
you navigate login -> compute-01 over ssh, inspect route configs and
munge keys, restart services in the right order, and verify via curl.
frontier llms have never trained on that loop.
EnterpriseHPC-v0 turns that loop into an rl environment.
## what is inside
- nested bwrap for lateral movement. `ssh compute-01` chroots the
shell into a separate rootfs so `hostname` and filesystem paths
reflect the new node
- fuse-overlayfs with upperdir and workdir on `/dev/shm` for
microsecond copy on write. kernel overlay and a copy fallback are
supported for hosts without fuse privileges
- a deterministic slurm state machine in
`/mnt/shared/slurm_state.json` with fcntl locks so many parallel
rollouts cannot corrupt each other
- python stubs for sinfo, squeue, systemctl, scontrol, curl, ssh that
read and mutate the json state, and a lightweight open ondemand
http server that returns 502 until the underlying fault is fixed
- three scenarios ship today and are rotated per rollout
- `hpc_outage` compute-01 drain from a broken route-eth0
- `hpc_munge` compute-01 drain from a munge key with wrong mode and
a broken route (chained)
- `hpc_pid_stale` slurmd refuses to restart after reboot because of a
leftover `/var/run/slurmd.pid`
- the gymnasium env `EnterpriseHPC-v0` wraps it all with pexpect so
the policy experiences real interactive bash prompts
## how fast
```
| mount | n | p50 ms | p95 ms | p99 ms | max ms |
| --- | ---: | ---: | ---: | ---: | ---: |
| copy | 100 | 2.40 | 2.56 | 2.58 | 2.87 |
```
that is in the ci friendly copy mode. real fuse-overlayfs on a linux
host drops well under 1 ms. reset latency is no longer the grpo
bottleneck.
## training with qwen2.5-coder
local training with unsloth + 4bit qlora:
```
python -m training.train_hpc_outage \
--model Qwen/Qwen2.5-Coder-7B-Instruct \
--group-size 4 --max-turns 12 \
--num-train-steps 100 \
--scenarios hpc_outage,hpc_munge,hpc_pid_stale
```
remote training against hosted openenv spaces (same shape as the
trl + openenv launch example, swapped to a code-tuned 7b policy):
```
python -m training.hpc_openenv_gemma \
--env-urls https://<user>-enterprise-hpc-openenv.hf.space \
https://<user>-enterprise-hpc-openenv-2.hf.space \
--model Qwen/Qwen2.5-Coder-7B-Instruct \
--group-size 4 --max-turns 12 --num-train-steps 200
```
submit to hf jobs:
```
python -m training.hf_jobs \
--env-urls https://<user>-enterprise-hpc-openenv.hf.space \
--gpu a10g-large \
--num-train-steps 300
```
the training scripts use unsloth for 4bit qlora loading and trl
`GRPOTrainer` with a custom rollout function that drives the env one
turn at a time. the reward is binary from the deterministic task
grader, which is exactly the signal grpo wants.
a colab notebook at `training/hpc_colab.ipynb` runs both the local
and remote paths on a single t4 / l4 / a100.
## what the agent learns
before training a random policy wanders around `sinfo` and never edits
the route file. after ~100 steps of grpo the agent reliably:
1. runs `sinfo` and `squeue` to locate the drained node
2. lateral moves with `ssh compute-01`
3. inspects `/etc/sysconfig/network-scripts/route-eth0`
4. writes the correct route with `printf ... >` (no heredocs allowed)
5. for the munge variant also `chmod 0400 /etc/munge/munge.key`
6. restarts munge then slurmd in that order
7. exits back to login and verifies with `curl -I http://localhost:8080`
## prove it is solvable
before any training, reviewers can run:
```
make gold # deterministic gold-trajectory verifier
make eval # gold vs random vs bad policies, writes runs/eval/leaderboard.md
make bench # reset-latency benchmark
```
## try it
- repo: https://github.com/your-org/low-taper-fade-openenv-scaler
- hf space (env server): https://huggingface.co/spaces/your-org/enterprise-hpc-openenv
- colab: `training/hpc_colab.ipynb`
- pitch doc: `docs/pitch.md`
- hf jobs guide: `docs/hf_jobs.md`
- spaces deploy: `docs/hf_spaces_deploy.md`
|