Upload folder using huggingface_hub

README.md

@@ -1,255 +1,231 @@
 ---
-title: Traffic Control Environment Server
-emoji: 🎯
-colorFrom: gray
-colorTo: gray
+title: Autonomous Traffic Control Environment
+emoji: 🚦
+colorFrom: red
+colorTo: green
 sdk: docker
 pinned: false
 app_port: 7860
 base_path: /web
 tags:
   - openenv
+  - reinforcement-learning
+  - traffic-control
+  - emergency-vehicles
+  - autonomous-systems
 ---
 
-# Traffic Control Environment
+# 🚦 Autonomous Traffic Control — OpenEnv Environment
 
-A simple test environment that echoes back messages. Perfect for testing the env APIs as well as demonstrating environment usage patterns.
+An LLM-driven reinforcement learning environment for autonomous traffic signal control at a 4-way intersection. Built for the **Meta × PyTorch × SST × OpenEnv Hackathon**.
 
-## Quick Start
-
-The simplest way to use the Traffic Control environment is through the `TrafficControlEnv` class:
+- **HF Space:** [amanmurari/sst-hack](https://huggingface.co/spaces/amanmurari/sst-hack)
+- **GitHub:** [amanmurari/openenv-hc2](https://github.com/amanmurari/openenv-hc2)
 
-```python
-from traffic_control import TrafficControlAction, TrafficControlEnv
+---
 
-try:
-    # Create environment from Docker image
-    traffic_controlenv = TrafficControlEnv.from_docker_image("traffic_control-env:latest")
+## Overview
 
-    # Reset
-    result = traffic_controlenv.reset()
-    print(f"Reset: {result.observation.echoed_message}")
+An LLM agent controls traffic signals to maximise vehicle throughput while prioritising emergency vehicles. The environment features:
 
-    # Send multiple messages
-    messages = ["Hello, World!", "Testing echo", "Final message"]
+- Sinusoidal traffic wave patterns (realistic rush-hour simulation)
+- Emergency vehicles with escalating urgency (urgency^1.5 penalty per waiting step)
+- Yellow-light transition state machine
+- Traffic surge events in hard tasks
+- Multi-objective grading aligned with real traffic KPIs
 
-    for msg in messages:
-        result = traffic_controlenv.step(TrafficControlAction(message=msg))
-        print(f"Sent: '{msg}'")
-        print(f"  → Echoed: '{result.observation.echoed_message}'")
-        print(f"  → Length: {result.observation.message_length}")
-        print(f"  → Reward: {result.reward}")
+---
 
-finally:
-    # Always clean up
-    traffic_controlenv.close()
-```
+## Tasks
 
-That's it! The `TrafficControlEnv.from_docker_image()` method handles:
-- Starting the Docker container
-- Waiting for the server to be ready
-- Connecting to the environment
-- Container cleanup when you call `close()`
+| Task | Difficulty | Steps | Key Challenge |
+|---|---|---|---|
+| `basic_flow` | Easy | 200 | Maximise throughput (target 1.8 veh/step) |
+| `emergency_priority` | Medium | 300 | Clear emergencies fast (avg delay < 3 steps) |
+| `dynamic_scenarios` | Hard | 400 | Surge traffic + simultaneous emergencies, no collisions |
 
-## Building the Docker Image
+---
 
-Before using the environment, you need to build the Docker image:
+## Action & Observation Space
 
-```bash
-# From project root
-docker build -t traffic_control-env:latest -f server/Dockerfile .
+### Action
+```python
+TrafficAction(light_phase: int)
+# 0 = NS_GREEN  (North-South green, East-West red)
+# 1 = EW_GREEN  (East-West green, North-South red)
+# 2 = ALL_RED   (All red — emergency clearance)
 ```
 
-## Deploying to Hugging Face Spaces
-
-You can easily deploy your OpenEnv environment to Hugging Face Spaces using the `openenv push` command:
-
-```bash
-# From the environment directory (where openenv.yaml is located)
-openenv push
-
-# Or specify options
-openenv push --namespace my-org --private
+### Observation
+```python
+TrafficObservation(
+    current_phase: int,          # Active phase (0-4, incl. yellow transitions)
+    time_in_phase: int,          # Steps held in current phase
+    queue_lengths: List[int],    # Regular vehicle queue [N, S, E, W]
+    emergency_queue: List[int],  # Emergency vehicle count [N, S, E, W]
+    emergency_urgency: List[int],# Max urgency 0-10 per approach
+    vehicles_passed: int,        # Regular vehicles cleared this step
+    emergency_passed: int,       # Emergency vehicles cleared this step
+    avg_wait_time: float,        # Avg waiting time across all queued vehicles
+    queue_trend: List[int],      # Queue growth since last step [N, S, E, W]
+    collision: bool,             # Gridlock-induced collision flag
+    done: bool,
+    reward: float,
+)
 ```
 
-The `openenv push` command will:
-1. Validate that the directory is an OpenEnv environment (checks for `openenv.yaml`)
-2. Prepare a custom build for Hugging Face Docker space (enables web interface)
-3. Upload to Hugging Face (ensuring you're logged in)
-
-### Prerequisites
-
-- Authenticate with Hugging Face: The command will prompt for login if not already authenticated
-
-### Options
-
-- `--directory`, `-d`: Directory containing the OpenEnv environment (defaults to current directory)
-- `--repo-id`, `-r`: Repository ID in format 'username/repo-name' (defaults to 'username/env-name' from openenv.yaml)
-- `--base-image`, `-b`: Base Docker image to use (overrides Dockerfile FROM)
-- `--private`: Deploy the space as private (default: public)
+---
 
-### Examples
+## Reward Function
 
-```bash
-# Push to your personal namespace (defaults to username/env-name from openenv.yaml)
-openenv push
+```
++0.30  × regular vehicles cleared per step
++12.0  × emergency vehicles cleared per step
+-0.08  × total vehicles waiting (queue pressure)
+-(urgency^1.5) × 0.5  per waiting emergency vehicle (every step!)
+-0.50 to -2.0  for unnecessary phase switch (proportional to empty-queue ratio)
++0.05  stability bonus when traffic flows without switching
+-200   gridlock collision (episode-ending catastrophe)
+```
 
-# Push to a specific repository
-openenv push --repo-id my-org/my-env
+---
 
-# Push with a custom base image
-openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest
+## Grading Weights
 
-# Push as a private space
-openenv push --private
+### basic_flow
+```
+score = throughput_score × 0.60 + efficiency_score × 0.40 + stability_bonus
+throughput_score = min(vehicles_per_step / 1.8, 1.0)
+efficiency_score = 1 / (1 + avg_waiting × 0.1)
+stability_bonus  = max(0, 0.05 × (1 − min(switch_rate × 4, 1.0)))
+```
 
-# Combine options
-openenv push --repo-id my-org/my-env --base-image custom-base:latest --private
+### emergency_priority
+```
+score = throughput × 0.30 + em_rate × 0.35 + delay × 0.20 + efficiency × 0.15
+em_rate = min(emergency_cleared_per_step / (1/20), 1.0)
+delay   = max(0, 1 − avg_em_delay / 12)
 ```
 
-After deployment, your space will be available at:
-`https://huggingface.co/spaces/<repo-id>`
+### dynamic_scenarios
+```
+score = throughput × 0.25 + em_rate × 0.30 + delay × 0.20
+      + efficiency × 0.15 + adaptability × 0.10
+adaptability = 1 / (1 + phase_changes_per_step × 0.5)
+```
 
-The deployed space includes:
-- **Web Interface** at `/web` - Interactive UI for exploring the environment
-- **API Documentation** at `/docs` - Full OpenAPI/Swagger interface
-- **Health Check** at `/health` - Container health monitoring
-- **WebSocket** at `/ws` - Persistent session endpoint for low-latency interactions
+---
 
-## Environment Details
+## Agent Architecture
 
-### Action
-**TrafficControlAction**: Contains a single field
-- `message` (str) - The message to echo back
+The inference agent uses a **hybrid heuristic + LLM** architecture:
 
-### Observation
-**TrafficControlObservation**: Contains the echo response and metadata
-- `echoed_message` (str) - The message echoed back
-- `message_length` (int) - Length of the message
-- `reward` (float) - Reward based on message length (length × 0.1)
-- `done` (bool) - Always False for echo environment
-- `metadata` (dict) - Additional info like step count
+1. **Heuristic recommender** — computes directional pressure using the actual reward formula (`urgency^1.5 × 0.5`), applies 5 priority rules (critical emergency, moderate emergency, hysteresis, pressure-based switch, default hold).
 
-### Reward
-The reward is calculated as: `message_length × 0.1`
-- "Hi" → reward: 0.2
-- "Hello, World!" → reward: 1.3
-- Empty message → reward: 0.0
+2. **Live score projection** — computes current projected grading scores (throughput, emergency rate, delay, efficiency, adaptability) from `env.state()` and includes them in every LLM prompt.
 
-## Advanced Usage
+3. **Chain-of-thought LLM** — the model reasons through scoring implications then outputs `{"light_phase": N}` on the final line.
 
-### Connecting to an Existing Server
+4. **Heuristic fallback** — if LLM output is unparseable, silently falls back to the heuristic. No crashes, no missed steps.
 
-If you already have a Traffic Control environment server running, you can connect directly:
+---
 
-```python
-from traffic_control import TrafficControlEnv
+## Quick Start
 
-# Connect to existing server
-traffic_controlenv = TrafficControlEnv(base_url="<ENV_HTTP_URL_HERE>")
+### Connect to the live HF Space
 
-# Use as normal
-result = traffic_controlenv.reset()
-result = traffic_controlenv.step(TrafficControlAction(message="Hello!"))
+```python
+from traffic_control.client import TrafficControlEnv
+from traffic_control.models import TrafficAction
+
+with TrafficControlEnv(base_url="https://amanmurari-sst-hack.hf.space").sync() as env:
+    result = env.reset(task_id="basic_flow", seed=42)
+    obs = result.observation
+
+    while not result.done:
+        action = TrafficAction(light_phase=0)   # replace with your agent
+        result = env.step(action)
+        obs = result.observation
+        print(f"Cleared: {obs.vehicles_passed} regular, {obs.emergency_passed} emergency | reward={result.reward:.2f}")
 ```
 
-Note: When connecting to an existing server, `traffic_controlenv.close()` will NOT stop the server.
-
-### Using the Context Manager
-
-The client supports context manager usage for automatic connection management:
+### Run inference locally
 
-```python
-from traffic_control import TrafficControlAction, TrafficControlEnv
+```bash
+# Set required env vars
+export API_BASE_URL="https://router.huggingface.co/v1"
+export HF_TOKEN="hf_..."
+export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"
+export SERVER_URL="http://localhost:7860"
 
-# Connect with context manager (auto-connects and closes)
-with TrafficControlEnv(base_url="http://localhost:8000") as env:
-    result = env.reset()
-    print(f"Reset: {result.observation.echoed_message}")
-    # Multiple steps with low latency
-    for msg in ["Hello", "World", "!"]:
-        result = env.step(TrafficControlAction(message=msg))
-        print(f"Echoed: {result.observation.echoed_message}")
+python inference.py
 ```
 
-The client uses WebSocket connections for:
-- **Lower latency**: No HTTP connection overhead per request
-- **Persistent session**: Server maintains your environment state
-- **Efficient for episodes**: Better for many sequential steps
-
-### Concurrent WebSocket Sessions
-
-The server supports multiple concurrent WebSocket connections. To enable this,
-modify `server/app.py` to use factory mode:
+### Build and run with Docker
 
-```python
-# In server/app.py - use factory mode for concurrent sessions
-app = create_app(
-    TrafficControlEnvironment,  # Pass class, not instance
-    TrafficControlAction,
-    TrafficControlObservation,
-    max_concurrent_envs=4,  # Allow 4 concurrent sessions
-)
+```bash
+docker build -t traffic-control-env .
+docker run -p 7860:7860 traffic-control-env
 ```
 
-Then multiple clients can connect simultaneously:
-
-```python
-from traffic_control import TrafficControlAction, TrafficControlEnv
-from concurrent.futures import ThreadPoolExecutor
+---
 
-def run_episode(client_id: int):
-    with TrafficControlEnv(base_url="http://localhost:8000") as env:
-        result = env.reset()
-        for i in range(10):
-            result = env.step(TrafficControlAction(message=f"Client {client_id}, step {i}"))
-        return client_id, result.observation.message_length
+## API Endpoints
 
-# Run 4 episodes concurrently
-with ThreadPoolExecutor(max_workers=4) as executor:
-    results = list(executor.map(run_episode, range(4)))
-```
+Once the server is running at `http://localhost:7860`:
 
-## Development & Testing
+| Endpoint | Description |
+|---|---|
+| `GET /health` | Health check — returns `{"status": "ok"}` |
+| `POST /reset` | Reset episode — body: `{"task_id": "basic_flow", "seed": 42}` |
+| `POST /step` | Execute action — body: `{"light_phase": 0}` |
+| `GET /state` | Cumulative episode state |
+| `WS /ws` | WebSocket endpoint for low-latency multi-step sessions |
+| `GET /web` | Interactive web dashboard |
+| `GET /docs` | OpenAPI / Swagger docs |
 
-### Direct Environment Testing
+---
 
-Test the environment logic directly without starting the HTTP server:
+## Project Structure
 
-```bash
-# From the server directory
-python3 server/traffic_control_environment.py
+```
+traffic_control/
+├── inference.py          # LLM agent (heuristic + chain-of-thought LLM)
+├── client.py             # TrafficControlEnv WebSocket client
+├── models.py             # TrafficAction / TrafficObservation / TrafficState
+├── environment.py        # Core simulation engine
+├── tasks.py              # Task graders (basic_flow, emergency_priority, dynamic_scenarios)
+├── dashboard.py          # Web UI dashboard
+├── analytics.py          # Episode analytics
+├── arena.py              # Multi-agent arena
+├── openenv.yaml          # OpenEnv manifest
+├── pyproject.toml        # Package metadata
+├── Dockerfile            # Container (port 7860)
+└── server/
+    ├── app.py            # FastAPI app (HTTP + WebSocket)
+    └── traffic_control_environment.py
 ```
 
-This verifies that:
-- Environment resets correctly
-- Step executes actions properly
-- State tracking works
-- Rewards are calculated correctly
+---
 
-### Running Locally
+## Environment Variables
 
-Run the server locally for development:
+| Variable | Required | Description |
+|---|---|---|
+| `API_BASE_URL` | Yes | LLM proxy endpoint (injected by validator) |
+| `API_KEY` | Yes | Proxy API key (injected by validator) |
+| `HF_TOKEN` | Alt | Hugging Face token (used if `API_KEY` not set) |
+| `MODEL_NAME` | No | LLM model (default: `Qwen/Qwen2.5-72B-Instruct`) |
+| `SERVER_URL` | No | Env server URL (default: `http://localhost:7860`) |
 
-```bash
-uvicorn server.app:app --reload
-```
+---
 
-## Project Structure
+## Stdout Format
 
 ```
-traffic_control/
-├── .dockerignore         # Docker build exclusions
-├── __init__.py            # Module exports
-├── README.md              # This file
-├── openenv.yaml           # OpenEnv manifest
-├── pyproject.toml         # Project metadata and dependencies
-├── uv.lock                # Locked dependencies (generated)
-├── client.py              # TrafficControlEnv client
-├── models.py              # Action and Observation models
-└── server/
-    ├── __init__.py        # Server module exports
-    ├── traffic_control_environment.py  # Core environment logic
-    ���── app.py             # FastAPI application (HTTP + WebSocket endpoints)
-    └── Dockerfile         # Container image definition
+[START] task=basic_flow env=traffic_control model=Qwen/Qwen2.5-72B-Instruct
+[STEP] step=1 action=light_phase=0 reward=0.65 done=false error=null
+[STEP] step=2 action=light_phase=0 reward=0.80 done=false error=null
+...
+[END] success=true steps=200 score=0.847 rewards=0.65,0.80,...
 ```