Spaces:
Sleeping
Teaching AI to Cooperate: How We Built an RL Environment for NGO Disaster Coordination
A Meta PyTorch OpenEnv Hackathon submission by Team Tech Wizards — Scaler School of Technology
The Problem We Kept Thinking About
Imagine a flood just hit a city. Three NGOs show up with volunteers, supplies, and good intentions — but different priorities. One wants to deploy fast and cover the most ground. Another is careful with resources and wants nothing wasted. The third is trying to negotiate so everyone gets a fair slice of help.
In real life, this coordination happens over phone calls, WhatsApp messages, and gut decisions made under pressure. It's messy, slow, and nobody wins quite as much as they could if everyone just... worked together better.
We asked ourselves: what if an AI could learn how to navigate this? Not by being told the rules, but by figuring them out through experience — the way humans do.
That question became CommunityPulse, our NGO Multi-Agent Coordination Environment.
What We Built
We built a Reinforcement Learning environment using OpenEnv (Gymnasium) where three AI agents represent NGO coordinators. They share a limited pool of volunteers and must decide — every step — how to allocate resources, whether to cooperate, and what to offer in negotiation.
The agents don't start knowing anything. They start completely random and learn through trial and error.
What Each Agent Sees
Every agent observes:
- Urgency level — how critical is the current situation (1–10)
- Available resources — volunteers currently on hand
- People affected — how many need help right now
- Other agents' last actions — what their counterparts did
- Coalition status — who's currently allied with whom
- Communication signals — cooperation intent from others
What Each Agent Can Do
Each agent outputs three values per step:
allocation %— how much of the resource pool to deploycooperation signal— willingness to work with others (0 = go it alone, 1 = full cooperation)negotiation bid— an offer to reach a compromise
Four Scenarios, One Environment
Rather than just one task, the environment cycles through four distinct challenge types:
Cooperation — agents must coordinate their allocations tightly to maximize total impact. Reward goes up when variance between agents is low and total coverage is high.
Competition — each agent is scored individually. Being too greedy (allocating far above average) gets penalized. A Nash equilibrium naturally emerges.
Negotiation — fairness and efficiency are both rewarded. The agent that negotiates well, not just the one that grabs the most, comes out ahead.
Coalition Formation — agents that form alliances and commit high allocations together get a 1.5× bonus. Agents learn when it's worth joining a coalition and when it's better to stay independent.
How Agents Actually Learn
We used an epsilon-greedy policy gradient approach:
- At episode 1, agents act completely randomly (ε = 1.0)
- Over training, exploration decays to just 5% (ε = 0.05)
- High-reward actions get reinforced; low-reward ones get discouraged
- Agents share a common environment but each maintains its own policy parameters
This isn't a simulation with pre-scripted outcomes. The strategies that emerge — cooperation, defection, coalition timing — aren't programmed in. They're learned.
What Changed After Training
We ran 100 episodes (5,000 total environment steps) and tracked episode rewards across all four task types.
The numbers:
| Reward | |
|---|---|
| First 10 episodes (avg) | 2,353 |
| Last 10 episodes (avg) | 5,103 |
| Improvement | +116.8% |
A few things we observed as training progressed:
- In cooperation tasks, allocation variance dropped — agents naturally converged on similar contribution levels
- In competition tasks, extreme greedy behavior disappeared; a stable equilibrium formed
- In coalition tasks, agents learned to signal cooperation before high-urgency steps, not after
- The negotiation bid values gradually became more informative — agents started using them to actually communicate intent
The learning curves (generated automatically by train.py) show a clear upward trend from noisy early episodes to smoother, higher-reward later ones.
Why It Matters
Multi-agent coordination is one of the hardest open problems in AI. Most RL work focuses on single-agent settings. But the real world is full of situations where multiple decision-makers — with different goals, different information, and different incentives — have to find a way to work together.
NGO disaster coordination is just one frame. The same environment structure applies to:
- Hospital resource allocation during surges (ICU beds, ventilators, staff)
- Budget negotiations between departments with competing priorities
- Supply chain coalition formation across logistics partners
- Training LLMs to handle multi-stakeholder conversations
If you're working on multi-agent systems and want a clean, fast, OpenEnv-compatible testbed for cooperation/competition/negotiation dynamics — this was built with that in mind.
Try It Yourself
The environment is built on standard Gymnasium and runs in under 30 seconds on a CPU.
pip install -r requirements.txt
python train.py
Or run the full training notebook in Google Colab — no setup, no API keys, just hit Runtime → Run all.
Open in Colab → Hugging Face Space → (link to be added post-deployment)
One More Thing
We named this project CommunityPulse because that's what we were going for — an AI that can feel the rhythm of a community in crisis and respond in a way that's not just efficient, but fair.
It's a small environment. A hundred episodes. Three agents. But the thing it's trying to learn — how to cooperate under pressure without being told to — feels like something worth working on.
We hope you think so too.
Built for the Meta PyTorch OpenEnv Hackathon 2025 · Theme #1: Multi-Agent Interactions Team: Tech Wizards, Scaler School of Technology GitHub: github.com/Sayali5115/communitypulse-env