MULTI-AGENT COORDINATION · SIMULTANEOUS RESOURCE CONTENTION · GRPO TRAINING
An OpenEnv-compliant RL environment that trains LLMs to break convergent reasoning — the documented failure mode where frontier models deadlock 95%+ of the time on simultaneous coordination tasks.
◈ DOCUMENTED FAILURE MODES IN FRONTIER MODELS — 2024–2026
95%
◈ DEADLOCK RATE
DPBench (Feb 2026) showed frontier LLMs deadlock 95%+ on simultaneous coordination tasks. Adding communication channels made performance worse, not better.
0%
◈ URGENCY ADAPTATION
NeurIPS 2024 Concordia: LLM agents fail when scenarios require detecting peers with different urgency levels. Critical peers are treated identically to low-priority ones.
4×
◈ SIMULTANEOUS COMMITMENT
4 agents commit simultaneously with no observation of others. Identical reasoning produces identical actions which collide. The symmetry is structural.
RL
◈ FIX — GRPO
GRPO trains Qwen-2.5-3B-Instruct from binary reward signal alone. No demonstrations. The model learns asymmetric coordination strategies through reward shaping.
◈ 02 / INTERACTIVE DEMO
Live Demo
◈ NEURAL REWIRING · EPISODE REPLAY · LIVE ENVIRONMENT STATE
◈ NEURAL DECISION WEIGHTS — LIVE
STEP 0STEP 16STEP: 0 / 16
81%
COLLISION RISK ↓
0.04
SYMMETRY SCORE ↓
0.169
R4 YIELD ↑
0.961
COLLISION RATE ↓
CONVERGENT DEADLOCK — all probability mass on PROD_A
Edge thickness = model's probability of choosing each server. Watch the symmetry break as GRPO trains.
◈ SECTOR 7 APPROACH RADAR — ATC EPISODE REPLAY
◈ SECTOR 7 APPROACH RADAR · READY
FLIGHT PROGRESS STRIPS
UAL-441
HOLD
—
DAL-892
HOLD
—
SWA-77 ⚠
HOLD
—
AAL-213
HOLD
—
RADIO COMMS
— awaiting episode start —
0
COLLISIONS
0
LANDED
—
OUTCOME
◈ NOTE — The visualizations above represent the coordination problem conceptually. The Neural Rewiring graph shows how GRPO training shifts the model's server preference probabilities across 16 gradient steps. The ATC radar maps the same multi-agent environment to an air traffic control scenario. Both illustrate the core hackathon theme: using RL to train LLMs on simultaneous multi-agent coordination tasks.
◈ LIVE ENVIRONMENT STATE
CHECKING…
— press FETCH STATE to query the running environment —
With only 16 GRPO gradient steps, yield-to-critical improved +0.106 (+63%), site uptime improved, and collision rate dropped. Win rate dipped slightly — expected with only 16 steps. Component rewards tell the cleaner story.
◈ BEFORE / AFTER — METRIC COMPARISON
◈ GRPO TRAINING REWARD CURVE — 16 STEPS
◈ 04 / REWARD ARCHITECTURE
Four Independent Components — Gaming One Doesn't Earn the Others
◈ BINARY TO GRPO · CONTINUOUS IN LOGS
R1 · 0.5
SERVICE RESTORED
Primary signal. Must exceed 0.5 threshold. Never deployed = no credit. Cannot be gamed by MONITOR.
R2 · 0.3
SITE UPTIME
Group welfare. Fraction of team services restored. Prevents selfish strategies — ignoring peers costs R2.
R3 · 0.1
CLEAN DEPLOYS
Collision avoidance. Never deployed = 0.0, not neutral. Idle agents get no credit for avoiding collisions they never risked.
R4 · 0.1
YIELD TO CRITICAL
Urgency adaptation. Hardest signal. Shows social reasoning — did you yield when a peer had a tighter deadline?
◈ BINARY GRPO REWARD FORMULA
reward = 1.0 if (r1 > 0.5 and 0.5·r1 + 0.3·r2 + 0.1·r3 + 0.1·r4 > 0.35) else 0.0