r1cksync commited on
Commit
d505de0
·
1 Parent(s): 34b426f

blog: replace mermaid fences with rendered PNGs (some viewers don't render mermaid); tighten legacy chart y-axis

Browse files
BLOG.md CHANGED
@@ -125,28 +125,7 @@ with two pools — `_GENERIC` (always in rotation) and `_PHASE_LINES` (fired
125
  when the saboteur escalates). Severity escalates the longer the incident
126
  drags, mirroring real pressure.
127
 
128
- ```mermaid
129
- %%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
130
- flowchart LR
131
- SC["Scenario JSON<br/>slack: msgs_per_tick=1.4"] --> SS["SlackStream<br/>(deterministic, seed-driven)"]
132
- SAB["Saboteur phase<br/>attack_primary / failover / dependency"] --> SS
133
- SS -->|emit_for_tick| OBS["Observation<br/>{ metrics, traces, slack[] }"]
134
- OBS --> ACT["Phi-3.5 Actor<br/>4-bit + LoRA"]
135
- ACT -->|action JSON| ENV["IncidentCommanderEnv"]
136
- ENV --> R{"Reward shaping"}
137
- R -->|action targets a service<br/>mentioned in recent slack| RP["+0.10<br/>useful_log_query"]
138
- R -->|action targets red-herring<br/>service from chatter| RH["−0.15<br/>red_herring_penalty"]
139
- R -->|action ignores buried clue<br/>and acts blind| BA["−0.10<br/>blind_action_penalty"]
140
- RP --> ADV["GAE advantage<br/>+ PPO update"]
141
- RH --> ADV
142
- BA --> ADV
143
- ADV --> ACT
144
-
145
- classDef good fill:#0e2a1c,stroke:#3ec78a,color:#cdeed8
146
- classDef bad fill:#2a0e0e,stroke:#ff7a7a,color:#ffd6d6
147
- class RP good
148
- class RH,BA bad
149
- ```
150
 
151
  *The Slack signal-vs-noise reward flow — pulled verbatim from the live [showcase page](https://sagnik-mukherjee-incodent-commander.hf.space/showcase).*
152
 
@@ -250,39 +229,7 @@ helm install acmecorp infra/helm/acmecorp
250
  €20/month gets you a usable demo cluster. See
251
  [`infra/terraform/main.tf`](https://github.com/r1cksync/meta-rl-hack/blob/main/incident-commander/infra/terraform/main.tf).
252
 
253
- ```mermaid
254
- %%{init:{'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#a78bfa','clusterBkg':'#0a0d1c','clusterBorder':'#7aa7ff'}}}%%
255
- flowchart TB
256
- subgraph TF["infra/terraform/main.tf"]
257
- direction LR
258
- net[hcloud_network<br/>10.0.0.0/16]
259
- subnet[hcloud_network_subnet<br/>10.0.1.0/24]
260
- ssh[hcloud_ssh_key]
261
- lb[hcloud_load_balancer<br/>lb11 · port 80/443]
262
- n0[hcloud_server · master<br/>cx21 · ubuntu-22.04]
263
- n1[hcloud_server · worker]
264
- n2[hcloud_server · worker]
265
- end
266
- subgraph K3S["k3s cluster"]
267
- direction LR
268
- ing[ingress-nginx]
269
- svc1[frontend]
270
- svc2[payments-api]
271
- svc3[inventory-service]
272
- svc4[notification-service]
273
- svc5[order-worker]
274
- end
275
- subgraph AGENT["IncidentCommander agent"]
276
- direction LR
277
- env["env.step(action)"]
278
- adapter["LoRA adapter<br/>(merged)"]
279
- env --> adapter
280
- adapter --> env
281
- end
282
- TF -->|provision| K3S
283
- AGENT -->|REAL_K8S=true| ing
284
- ing --> svc1 & svc2 & svc3 & svc4 & svc5
285
- ```
286
 
287
  *Same Hetzner topology that powers the live demo cluster — Terraform on the left provisions the k3s nodes, the agent talks to ingress over HTTP when `REAL_K8S=true`.*
288
 
@@ -508,20 +455,7 @@ free Kaggle GPU minutes.
508
 
509
  ## 6 · The training pipeline — GitHub → Kaggle → merged adapter
510
 
511
- ```mermaid
512
- %%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
513
- flowchart LR
514
- A["Scenario JSON<br/>(381 files)"] --> B["IncidentCommanderEnv<br/>(reset + step)"]
515
- B -->|observation| C["Phi-3.5-mini Actor<br/>4-bit + LoRA"]
516
- C -->|action JSON| B
517
- B -->|reward| D["DeepSeek-R1 Critic<br/>0–10 rubric"]
518
- D -->|value| E["GAE advantages"]
519
- C -->|log-prob| E
520
- E --> F["PPO update<br/>clip + KL + entropy"]
521
- F -->|grad| C
522
- F --> G["training_kaggle*.json<br/>per-update metrics"]
523
- F --> H["adapter_kaggle*<br/>LoRA delta"]
524
- ```
525
 
526
  *The full Pass-B training DAG: scenario → env → 4-bit Phi-3.5 actor → DeepSeek-R1 frozen critic → GAE → PPO → LoRA delta + per-update JSON. This is the diagram that drives the three Kaggle shard runs in §4-Pass-B.*
527
 
 
125
  when the saboteur escalates). Severity escalates the longer the incident
126
  drags, mirroring real pressure.
127
 
128
+ ![Slack signal-vs-noise reward flow — Phi-3.5 actor, env, reward shaper, GAE/PPO loop](assets/blog/mermaid_slack_reward.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
129
 
130
  *The Slack signal-vs-noise reward flow — pulled verbatim from the live [showcase page](https://sagnik-mukherjee-incodent-commander.hf.space/showcase).*
131
 
 
229
  €20/month gets you a usable demo cluster. See
230
  [`infra/terraform/main.tf`](https://github.com/r1cksync/meta-rl-hack/blob/main/incident-commander/infra/terraform/main.tf).
231
 
232
+ ![Hetzner Cloud k3s topology — Terraform on the left provisions the k3s cluster on the right; the agent talks to ingress over HTTP](assets/blog/mermaid_hetzner_infra.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
233
 
234
  *Same Hetzner topology that powers the live demo cluster — Terraform on the left provisions the k3s nodes, the agent talks to ingress over HTTP when `REAL_K8S=true`.*
235
 
 
455
 
456
  ## 6 · The training pipeline — GitHub → Kaggle → merged adapter
457
 
458
+ ![Pass-B training DAG — scenario → env → 4-bit Phi-3.5 actor → DeepSeek-R1 critic → GAE → PPO → LoRA delta](assets/blog/mermaid_training_dag.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
459
 
460
  *The full Pass-B training DAG: scenario → env → 4-bit Phi-3.5 actor → DeepSeek-R1 frozen critic → GAE → PPO → LoRA delta + per-update JSON. This is the diagram that drives the three Kaggle shard runs in §4-Pass-B.*
461
 
assets/blog/legacy_training.png CHANGED

Git LFS Details

  • SHA256: 3aca4fe96820dc682b8367a029fed3bc94a48fb8940f4a582b75e462e50bbcc3
  • Pointer size: 130 Bytes
  • Size of remote file: 76.1 kB

Git LFS Details

  • SHA256: 6dd9c00d23a14d69aba5e51ea7ac7477e94a47c3567463b892ab5f8bfa1f0eb0
  • Pointer size: 130 Bytes
  • Size of remote file: 85 kB
assets/blog/mermaid/hetzner_infra.mmd ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ %%{init:{'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#a78bfa','clusterBkg':'#0a0d1c','clusterBorder':'#7aa7ff'}}}%%
2
+ flowchart TB
3
+ subgraph TF["infra/terraform/main.tf"]
4
+ direction LR
5
+ net[hcloud_network<br/>10.0.0.0/16]
6
+ subnet[hcloud_network_subnet<br/>10.0.1.0/24]
7
+ ssh[hcloud_ssh_key]
8
+ lb[hcloud_load_balancer<br/>lb11 · port 80/443]
9
+ n0[hcloud_server · master<br/>cx21 · ubuntu-22.04]
10
+ n1[hcloud_server · worker]
11
+ n2[hcloud_server · worker]
12
+ end
13
+ subgraph K3S["k3s cluster"]
14
+ direction LR
15
+ ing[ingress-nginx]
16
+ svc1[frontend]
17
+ svc2[payments-api]
18
+ svc3[inventory-service]
19
+ svc4[notification-service]
20
+ svc5[order-worker]
21
+ end
22
+ subgraph AGENT["IncidentCommander agent"]
23
+ direction LR
24
+ env["env.step(action)"]
25
+ adapter["LoRA adapter<br/>(merged)"]
26
+ env --> adapter
27
+ adapter --> env
28
+ end
29
+ TF -->|provision| K3S
30
+ AGENT -->|REAL_K8S=true| ing
31
+ ing --> svc1 & svc2 & svc3 & svc4 & svc5
assets/blog/mermaid/slack_reward.mmd ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ %%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
2
+ flowchart LR
3
+ SC["Scenario JSON<br/>slack: msgs_per_tick=1.4"] --> SS["SlackStream<br/>(deterministic, seed-driven)"]
4
+ SAB["Saboteur phase<br/>attack_primary / failover / dependency"] --> SS
5
+ SS -->|emit_for_tick| OBS["Observation<br/>{ metrics, traces, slack[] }"]
6
+ OBS --> ACT["Phi-3.5 Actor<br/>4-bit + LoRA"]
7
+ ACT -->|action JSON| ENV["IncidentCommanderEnv"]
8
+ ENV --> R{"Reward shaping"}
9
+ R -->|action targets a service<br/>mentioned in recent slack| RP["+0.10<br/>useful_log_query"]
10
+ R -->|action targets red-herring<br/>service from chatter| RH["−0.15<br/>red_herring_penalty"]
11
+ R -->|action ignores buried clue<br/>and acts blind| BA["−0.10<br/>blind_action_penalty"]
12
+ RP --> ADV["GAE advantage<br/>+ PPO update"]
13
+ RH --> ADV
14
+ BA --> ADV
15
+ ADV --> ACT
16
+
17
+ classDef good fill:#0e2a1c,stroke:#3ec78a,color:#cdeed8
18
+ classDef bad fill:#2a0e0e,stroke:#ff7a7a,color:#ffd6d6
19
+ class RP good
20
+ class RH,BA bad
assets/blog/mermaid/training_dag.mmd ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ %%{init: {'theme':'dark','themeVariables':{'primaryColor':'#11162b','primaryTextColor':'#e9eeff','primaryBorderColor':'#7aa7ff','lineColor':'#7aa7ff','tertiaryColor':'#0a0d1c'}}}%%
2
+ flowchart LR
3
+ A["Scenario JSON<br/>(381 files)"] --> B["IncidentCommanderEnv<br/>(reset + step)"]
4
+ B -->|observation| C["Phi-3.5-mini Actor<br/>4-bit + LoRA"]
5
+ C -->|action JSON| B
6
+ B -->|reward| D["DeepSeek-R1 Critic<br/>0–10 rubric"]
7
+ D -->|value| E["GAE advantages"]
8
+ C -->|log-prob| E
9
+ E --> F["PPO update<br/>clip + KL + entropy"]
10
+ F -->|grad| C
11
+ F --> G["training_kaggle*.json<br/>per-update metrics"]
12
+ F --> H["adapter_kaggle*<br/>LoRA delta"]
assets/blog/mermaid_hetzner_infra.png ADDED

Git LFS Details

  • SHA256: 7a3c9c240f63679228e850c05aef1ed03375038b87d78dc97ed3dc9916a929f5
  • Pointer size: 130 Bytes
  • Size of remote file: 69.1 kB
assets/blog/mermaid_slack_reward.png ADDED

Git LFS Details

  • SHA256: a9dbcfd56df4d579c81489a00f1e32cf0131b8596ee5ed4d230c6c665ab88aea
  • Pointer size: 130 Bytes
  • Size of remote file: 58 kB
assets/blog/mermaid_training_dag.png ADDED

Git LFS Details

  • SHA256: 1ffba5439253e6b9a39b4c0524a0a145be630beedfe1d9e3dae0b335911ed61c
  • Pointer size: 130 Bytes
  • Size of remote file: 44.3 kB