README.md

metadata

title: Logistics Shipment Env
emoji: 🚛
colorFrom: blue
colorTo: purple
sdk: docker
app_port: 7860
pinned: true

🚛 AI Logistics Coordinator — OpenEnv RL Environment

A multi-turn, multi-disruption freight crisis simulator for training LLMs to reason, plan, and communicate under pressure.

---

🌍 1. Problem: Logistics is a Reasoning Crisis, Not an Optimization Problem

Modern logistics systems fail when disruptions cascade — port strikes, accidents, carrier insolvencies — because they are built on static optimizers, not reasoning agents.

The capability gap we close: Training an LLM to act as a centralized logistics coordinator that can triage shipments, plan across multiple turns, reason about network congestion caused by other agents, and communicate empathetically with customers — all simultaneously.

---

✅ Hackathon Evaluation Criteria Checklist

For the judges, here is exactly how this project fulfills the core criteria:

• A clear environment design: A highly-structured Pydantic environment representing a freight network with cascading disruptions, dynamic route capacities, and SLA tracking (server/environment.py).
• Objective reward functions: Three independent reward signals (Structure, Routing, Communication) computed purely from verifiable environment state and JSON output, each with explicit anti-hacking penalties.
• Evidence that the model improved: Reward jumped from 0.18 (untrained baseline) → 0.7683 (GRPO-trained) — a +327% improvement. See reward curve below.
• Prevention against reward hacking: Explicit negative penalties for spamming API calls (-0.3), sending duplicate messages (-0.5), routing to non-existent or congested routes (-0.6), and escalating to humans (-0.1 per handoff).
• A reproducible deployment story: Full FastAPI + OpenEnv backend deployed live to Hugging Face Spaces. The entire training loop runs in one click on a free Google Colab T4 GPU.
• A sharp demo: Colab notebook shows the exact format: Untrained Baseline ➔ 3-Phase GRPO Training ➔ Trained Model ➔ +327% reward improvement ➔ Anti-hacking safeguards explained.

---

🎮 2. Environment: What the Agent Sees, Does, and Gets Rewarded For

Live Demo & Model

🤗 Environment Space: huggingface.co/spaces/Leavin1611/logistics-hackathon-env
🧠 Trained Model Adapter: huggingface.co/Leavin1611/logistics-hackathon-model

Themes Covered

Hackathon Theme	How We Address It
Theme #1 — Multi-Agent	Routes have capacity limits; background agent traffic updates every turn, forcing strategic routing
Theme #2 — Long-Horizon Planning	5-7 turn episodes with cascading disruptions; early decisions determine final SLA outcome
Theme #3 — World Modeling	Partially observable; agent must call tools to query the live network state

Observation Space

Each step returns a LogisticsObservation with:

• shipments — cargo, route, carrier, SLA buffer, delay hours, status
• disruptions — active events (port strikes, accidents, carrier failures)
• route_load — real-time congestion per route (0.0 → 1.0), updated by simulated multi-agent traffic
• feedback — result of last action
• incremental_reward — immediate reward signal
• cumulative_reward — running episode total

Action Space

Action	Description
get_network_status	Query live network state
reroute_shipment	Move shipment to alternate route (blocked if route > 85% capacity)
set_priority	Fast-track up to 3 shipments
communicate_eta	Send NLP-graded ETA message to customer
escalate	Hand off to human (penalized — agent should solve it)
end_turn	Commit decisions and receive turn reward

Reward Function — Four Independent Signals (Anti-Hack Design)

Turn Reward = 0.40 × DelayScore + 0.30 × SLAScore + 0.20 × CommScore + 0.10 × EscScore

Signal	Weight	Guard Against Exploitation
Delay Reduction	40%	Bounded by realistic hours-saved map
SLA Compliance	30%	Based on live shipment state only
Communication Quality	20%	Duplicate message penalty: -0.5
Escalation Control	10%	-0.1 per human handoff

The -0.5 duplicate penalty directly prevents reward hacking: an agent spamming messages to inflate its communication score is penalized immediately.

Task Curriculum

Task	Name	Shipments	Turns	Challenge
TASK-EASY	Port Backlog Clearance	2	3	Single JNPT disruption
TASK-MEDIUM	Mumbai Crisis Coordination	4	5	Port + accident + carrier strike
TASK-HARD	Multi-Port Network Collapse	7	7	3 simultaneous failures + insolvency

---

📈 3. Results: Observable Evidence of Training Progress

We trained Qwen/Qwen2.5-1.5B-Instruct using GRPO against the live environment server:

Metric	Base Model (Untrained)	GRPO-Trained (Phase 3)
Cumulative Reward	0.18	0.7683 (+327%)
Valid JSON Actions	~60%	~98%
Strategic Reroutes per Episode	1	3+
Communication Quality	Bare / no apology	Empathetic + specific ETA
Escalation Rate	High	Near zero (self-solving)

Training Reward Curve

Reward progression across all 3 GRPO curriculum phases. Blue line = rolling average reward. Red dashed = untrained baseline (0.18). Green dashed = final trained average (0.7683). Each shaded region is one training phase.

Detailed Evaluation & Ablation Studies

Below are detailed episode-by-episode breakdowns proving the model's performance improvements across different curriculum stages and safeguards.

Curriculum Learning Progression

Episode-by-episode breakdown showing how curriculum learning (+34.2%) outperformed the untrained baseline consistently.

Reward Hacking Safeguards (Old vs New)

Comparison showing how the final model maintained high performance while successfully avoiding the reward-hacking penalties applied to the older model.

Final Logic Improvement

Final evaluation showing a massive +57.2% improvement in routing logic and SLA compliance.

Overall Efficiency Increase

Overall efficiency metrics comparing the pre-training baseline against the fully hardened GRPO model.

What the Trained Agent Learned:

1. Always call get_network_status first — it modeled the world before acting
2. Avoid overloaded routes — learned to check route load before rerouting
3. Write empathetic messages — discovered that apology + reason + ETA = maximum comm reward
4. Never escalate — learned that self-solving is always rewarded over hand-offs

---

🧠 4. Training Pipeline

Why GRPO?

GRPO (Group Relative Policy Optimization) — the algorithm behind DeepSeek-R1 — compares a group of rollouts against each other and rewards relatively better ones. No separate critic model needed. Ideal for verifiable, environment-driven rewards.

Stack

OpenEnv (live environment) → TRL + GRPO → Unsloth (T4 efficiency)

Run Training in One Click

1. Open the notebook in Google Colab (button above)
2. Set Runtime → T4 GPU
3. Click Run All
4. The notebook auto-generates reward plots and a full before/after comparison with baseline

---

🚀 5. Quick Start (Local)

# Clone the environment
git clone https://huggingface.co/spaces/Leavin1611/logistics-hackathon-env
cd logistics-hackathon-env

# Windows: double-click setup.bat, then start.bat
# OR manually:
python -m venv .venv
.venv\Scripts\pip install fastapi "uvicorn[standard]" pydantic openenv-core
.venv\Scripts\uvicorn server.app:app --host 0.0.0.0 --port 7860

# Open http://localhost:7860 in your browser

Run Inference (with any OpenAI-compatible API)

pip install openai python-dotenv
export OPENAI_API_KEY="your-groq-key"     # Free at console.groq.com
export API_BASE_URL="https://api.groq.com/openai/v1"
export MODEL_NAME="llama-3.1-8b-instant"
python inference.py

---

🗺️ Shipment State Machine

stateDiagram-v2
    [*] --> IN_TRANSIT : Episode starts
    IN_TRANSIT --> DELAYED : SLA buffer expires
    DELAYED --> IN_TRANSIT : reroute_shipment (saves hours)
    DELAYED --> CRITICAL : SLA buffer < -4h
    CRITICAL --> DELAYED : reroute + set_priority
    IN_TRANSIT --> RESOLVED : delay_h = 0
    RESOLVED --> [*] : Episode ends

---

📁 Project Structure

logistics-hackathon-env/
├── server/
│   ├── app.py              # FastAPI server (OpenEnv-compatible)
│   └── environment.py      # Core RL engine — reset/step/state/rewards
├── dashboard.html          # Live interactive UI (served at root)
├── inference.py            # Baseline agent runner
├── train_colab.ipynb       # GRPO training notebook (one-click Colab)
├── HF_BLOG_POST.md         # Full writeup / mini-blog
├── openenv.yaml            # Environment manifest
├── setup.bat               # Windows local setup script
├── start.bat               # Windows local server launcher
└── examples/
    └── train_grpo.py       # Training script

---

🔗 Links & Materials

Resource	Link
🤗 Live HF Space	huggingface.co/spaces/Leavin1611/logistics-hackathon-env
🧠 Trained Model	Leavin1611/logistics-hackathon-model
📓 Training Notebook	train_colab.ipynb (Colab)
🖥️ Slide Deck	Live Preview → /slides — 9-slide presentation (arrow keys)
📝 Mini-Blog	HF_BLOG_POST.md
📦 GitHub Repo	github.com/leavin1611/Logistics-hackathon-env
🔧 OpenEnv Framework	github.com/meta-pytorch/OpenEnv
🔑 Free Groq API Key	console.groq.com/keys

---

📚 Further Reading

• HF_BLOG_POST.md — Full narrative writeup for judges
• Slide Deck — 9-slide live presentation (arrow key navigation)
• DESIGN.md — Architecture decisions and reward anatomy
• CONTRIBUTING.md — How to add scenarios, routes, actions

---

Built for the Meta PyTorch OpenEnv Hackathon 2026 — India Round 2 Stack: OpenEnv · FastAPI · TRL · GRPO · Unsloth · Qwen2.5