Spaces:
Running
Running
r1cksync commited on
Commit ·
d505de0
1
Parent(s): 34b426f
blog: replace mermaid fences with rendered PNGs (some viewers don't render mermaid); tighten legacy chart y-axis
Browse files- BLOG.md +3 -69
- assets/blog/legacy_training.png +2 -2
- assets/blog/mermaid/hetzner_infra.mmd +31 -0
- assets/blog/mermaid/slack_reward.mmd +20 -0
- assets/blog/mermaid/training_dag.mmd +12 -0
- assets/blog/mermaid_hetzner_infra.png +3 -0
- assets/blog/mermaid_slack_reward.png +3 -0
- assets/blog/mermaid_training_dag.png +3 -0
BLOG.md
CHANGED
|
@@ -125,28 +125,7 @@ with two pools — `_GENERIC` (always in rotation) and `_PHASE_LINES` (fired
|
|
| 125 |
when the saboteur escalates). Severity escalates the longer the incident
|
| 126 |
drags, mirroring real pressure.
|
| 127 |
|
| 128 |
-
|
| 129 |
-
%%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
|
| 130 |
-
flowchart LR
|
| 131 |
-
SC["Scenario JSON<br/>slack: msgs_per_tick=1.4"] --> SS["SlackStream<br/>(deterministic, seed-driven)"]
|
| 132 |
-
SAB["Saboteur phase<br/>attack_primary / failover / dependency"] --> SS
|
| 133 |
-
SS -->|emit_for_tick| OBS["Observation<br/>{ metrics, traces, slack[] }"]
|
| 134 |
-
OBS --> ACT["Phi-3.5 Actor<br/>4-bit + LoRA"]
|
| 135 |
-
ACT -->|action JSON| ENV["IncidentCommanderEnv"]
|
| 136 |
-
ENV --> R{"Reward shaping"}
|
| 137 |
-
R -->|action targets a service<br/>mentioned in recent slack| RP["+0.10<br/>useful_log_query"]
|
| 138 |
-
R -->|action targets red-herring<br/>service from chatter| RH["−0.15<br/>red_herring_penalty"]
|
| 139 |
-
R -->|action ignores buried clue<br/>and acts blind| BA["−0.10<br/>blind_action_penalty"]
|
| 140 |
-
RP --> ADV["GAE advantage<br/>+ PPO update"]
|
| 141 |
-
RH --> ADV
|
| 142 |
-
BA --> ADV
|
| 143 |
-
ADV --> ACT
|
| 144 |
-
|
| 145 |
-
classDef good fill:#0e2a1c,stroke:#3ec78a,color:#cdeed8
|
| 146 |
-
classDef bad fill:#2a0e0e,stroke:#ff7a7a,color:#ffd6d6
|
| 147 |
-
class RP good
|
| 148 |
-
class RH,BA bad
|
| 149 |
-
```
|
| 150 |
|
| 151 |
*The Slack signal-vs-noise reward flow — pulled verbatim from the live [showcase page](https://sagnik-mukherjee-incodent-commander.hf.space/showcase).*
|
| 152 |
|
|
@@ -250,39 +229,7 @@ helm install acmecorp infra/helm/acmecorp
|
|
| 250 |
€20/month gets you a usable demo cluster. See
|
| 251 |
[`infra/terraform/main.tf`](https://github.com/r1cksync/meta-rl-hack/blob/main/incident-commander/infra/terraform/main.tf).
|
| 252 |
|
| 253 |
-
|
| 254 |
-
%%{init:{'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#a78bfa','clusterBkg':'#0a0d1c','clusterBorder':'#7aa7ff'}}}%%
|
| 255 |
-
flowchart TB
|
| 256 |
-
subgraph TF["infra/terraform/main.tf"]
|
| 257 |
-
direction LR
|
| 258 |
-
net[hcloud_network<br/>10.0.0.0/16]
|
| 259 |
-
subnet[hcloud_network_subnet<br/>10.0.1.0/24]
|
| 260 |
-
ssh[hcloud_ssh_key]
|
| 261 |
-
lb[hcloud_load_balancer<br/>lb11 · port 80/443]
|
| 262 |
-
n0[hcloud_server · master<br/>cx21 · ubuntu-22.04]
|
| 263 |
-
n1[hcloud_server · worker]
|
| 264 |
-
n2[hcloud_server · worker]
|
| 265 |
-
end
|
| 266 |
-
subgraph K3S["k3s cluster"]
|
| 267 |
-
direction LR
|
| 268 |
-
ing[ingress-nginx]
|
| 269 |
-
svc1[frontend]
|
| 270 |
-
svc2[payments-api]
|
| 271 |
-
svc3[inventory-service]
|
| 272 |
-
svc4[notification-service]
|
| 273 |
-
svc5[order-worker]
|
| 274 |
-
end
|
| 275 |
-
subgraph AGENT["IncidentCommander agent"]
|
| 276 |
-
direction LR
|
| 277 |
-
env["env.step(action)"]
|
| 278 |
-
adapter["LoRA adapter<br/>(merged)"]
|
| 279 |
-
env --> adapter
|
| 280 |
-
adapter --> env
|
| 281 |
-
end
|
| 282 |
-
TF -->|provision| K3S
|
| 283 |
-
AGENT -->|REAL_K8S=true| ing
|
| 284 |
-
ing --> svc1 & svc2 & svc3 & svc4 & svc5
|
| 285 |
-
```
|
| 286 |
|
| 287 |
*Same Hetzner topology that powers the live demo cluster — Terraform on the left provisions the k3s nodes, the agent talks to ingress over HTTP when `REAL_K8S=true`.*
|
| 288 |
|
|
@@ -508,20 +455,7 @@ free Kaggle GPU minutes.
|
|
| 508 |
|
| 509 |
## 6 · The training pipeline — GitHub → Kaggle → merged adapter
|
| 510 |
|
| 511 |
-
|
| 512 |
-
%%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
|
| 513 |
-
flowchart LR
|
| 514 |
-
A["Scenario JSON<br/>(381 files)"] --> B["IncidentCommanderEnv<br/>(reset + step)"]
|
| 515 |
-
B -->|observation| C["Phi-3.5-mini Actor<br/>4-bit + LoRA"]
|
| 516 |
-
C -->|action JSON| B
|
| 517 |
-
B -->|reward| D["DeepSeek-R1 Critic<br/>0–10 rubric"]
|
| 518 |
-
D -->|value| E["GAE advantages"]
|
| 519 |
-
C -->|log-prob| E
|
| 520 |
-
E --> F["PPO update<br/>clip + KL + entropy"]
|
| 521 |
-
F -->|grad| C
|
| 522 |
-
F --> G["training_kaggle*.json<br/>per-update metrics"]
|
| 523 |
-
F --> H["adapter_kaggle*<br/>LoRA delta"]
|
| 524 |
-
```
|
| 525 |
|
| 526 |
*The full Pass-B training DAG: scenario → env → 4-bit Phi-3.5 actor → DeepSeek-R1 frozen critic → GAE → PPO → LoRA delta + per-update JSON. This is the diagram that drives the three Kaggle shard runs in §4-Pass-B.*
|
| 527 |
|
|
|
|
| 125 |
when the saboteur escalates). Severity escalates the longer the incident
|
| 126 |
drags, mirroring real pressure.
|
| 127 |
|
| 128 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
|
| 130 |
*The Slack signal-vs-noise reward flow — pulled verbatim from the live [showcase page](https://sagnik-mukherjee-incodent-commander.hf.space/showcase).*
|
| 131 |
|
|
|
|
| 229 |
€20/month gets you a usable demo cluster. See
|
| 230 |
[`infra/terraform/main.tf`](https://github.com/r1cksync/meta-rl-hack/blob/main/incident-commander/infra/terraform/main.tf).
|
| 231 |
|
| 232 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 233 |
|
| 234 |
*Same Hetzner topology that powers the live demo cluster — Terraform on the left provisions the k3s nodes, the agent talks to ingress over HTTP when `REAL_K8S=true`.*
|
| 235 |
|
|
|
|
| 455 |
|
| 456 |
## 6 · The training pipeline — GitHub → Kaggle → merged adapter
|
| 457 |
|
| 458 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 459 |
|
| 460 |
*The full Pass-B training DAG: scenario → env → 4-bit Phi-3.5 actor → DeepSeek-R1 frozen critic → GAE → PPO → LoRA delta + per-update JSON. This is the diagram that drives the three Kaggle shard runs in §4-Pass-B.*
|
| 461 |
|
assets/blog/legacy_training.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/blog/mermaid/hetzner_infra.mmd
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
%%{init:{'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#a78bfa','clusterBkg':'#0a0d1c','clusterBorder':'#7aa7ff'}}}%%
|
| 2 |
+
flowchart TB
|
| 3 |
+
subgraph TF["infra/terraform/main.tf"]
|
| 4 |
+
direction LR
|
| 5 |
+
net[hcloud_network<br/>10.0.0.0/16]
|
| 6 |
+
subnet[hcloud_network_subnet<br/>10.0.1.0/24]
|
| 7 |
+
ssh[hcloud_ssh_key]
|
| 8 |
+
lb[hcloud_load_balancer<br/>lb11 · port 80/443]
|
| 9 |
+
n0[hcloud_server · master<br/>cx21 · ubuntu-22.04]
|
| 10 |
+
n1[hcloud_server · worker]
|
| 11 |
+
n2[hcloud_server · worker]
|
| 12 |
+
end
|
| 13 |
+
subgraph K3S["k3s cluster"]
|
| 14 |
+
direction LR
|
| 15 |
+
ing[ingress-nginx]
|
| 16 |
+
svc1[frontend]
|
| 17 |
+
svc2[payments-api]
|
| 18 |
+
svc3[inventory-service]
|
| 19 |
+
svc4[notification-service]
|
| 20 |
+
svc5[order-worker]
|
| 21 |
+
end
|
| 22 |
+
subgraph AGENT["IncidentCommander agent"]
|
| 23 |
+
direction LR
|
| 24 |
+
env["env.step(action)"]
|
| 25 |
+
adapter["LoRA adapter<br/>(merged)"]
|
| 26 |
+
env --> adapter
|
| 27 |
+
adapter --> env
|
| 28 |
+
end
|
| 29 |
+
TF -->|provision| K3S
|
| 30 |
+
AGENT -->|REAL_K8S=true| ing
|
| 31 |
+
ing --> svc1 & svc2 & svc3 & svc4 & svc5
|
assets/blog/mermaid/slack_reward.mmd
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
%%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
|
| 2 |
+
flowchart LR
|
| 3 |
+
SC["Scenario JSON<br/>slack: msgs_per_tick=1.4"] --> SS["SlackStream<br/>(deterministic, seed-driven)"]
|
| 4 |
+
SAB["Saboteur phase<br/>attack_primary / failover / dependency"] --> SS
|
| 5 |
+
SS -->|emit_for_tick| OBS["Observation<br/>{ metrics, traces, slack[] }"]
|
| 6 |
+
OBS --> ACT["Phi-3.5 Actor<br/>4-bit + LoRA"]
|
| 7 |
+
ACT -->|action JSON| ENV["IncidentCommanderEnv"]
|
| 8 |
+
ENV --> R{"Reward shaping"}
|
| 9 |
+
R -->|action targets a service<br/>mentioned in recent slack| RP["+0.10<br/>useful_log_query"]
|
| 10 |
+
R -->|action targets red-herring<br/>service from chatter| RH["−0.15<br/>red_herring_penalty"]
|
| 11 |
+
R -->|action ignores buried clue<br/>and acts blind| BA["−0.10<br/>blind_action_penalty"]
|
| 12 |
+
RP --> ADV["GAE advantage<br/>+ PPO update"]
|
| 13 |
+
RH --> ADV
|
| 14 |
+
BA --> ADV
|
| 15 |
+
ADV --> ACT
|
| 16 |
+
|
| 17 |
+
classDef good fill:#0e2a1c,stroke:#3ec78a,color:#cdeed8
|
| 18 |
+
classDef bad fill:#2a0e0e,stroke:#ff7a7a,color:#ffd6d6
|
| 19 |
+
class RP good
|
| 20 |
+
class RH,BA bad
|
assets/blog/mermaid/training_dag.mmd
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
%%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
|
| 2 |
+
flowchart LR
|
| 3 |
+
A["Scenario JSON<br/>(381 files)"] --> B["IncidentCommanderEnv<br/>(reset + step)"]
|
| 4 |
+
B -->|observation| C["Phi-3.5-mini Actor<br/>4-bit + LoRA"]
|
| 5 |
+
C -->|action JSON| B
|
| 6 |
+
B -->|reward| D["DeepSeek-R1 Critic<br/>0–10 rubric"]
|
| 7 |
+
D -->|value| E["GAE advantages"]
|
| 8 |
+
C -->|log-prob| E
|
| 9 |
+
E --> F["PPO update<br/>clip + KL + entropy"]
|
| 10 |
+
F -->|grad| C
|
| 11 |
+
F --> G["training_kaggle*.json<br/>per-update metrics"]
|
| 12 |
+
F --> H["adapter_kaggle*<br/>LoRA delta"]
|
assets/blog/mermaid_hetzner_infra.png
ADDED
|
Git LFS Details
|
assets/blog/mermaid_slack_reward.png
ADDED
|
Git LFS Details
|
assets/blog/mermaid_training_dag.png
ADDED
|
Git LFS Details
|