README.md

---
title: AETHER-TaskFlow
emoji: ⚡
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
tags:
  - openenv
  - reinforcement-learning
  - rl-environment
  - workflow-management
  - task-prioritization
  - meta-pytorch-hackathon
license: mit
---

# ⚡ AETHER-TaskFlow

**Adaptive Workflow Management RL Environment — Meta PyTorch OpenEnv Hackathon 2025**

[![OpenEnv](https://img.shields.io/badge/OpenEnv-compliant-brightgreen)](https://github.com/meta-pytorch/OpenEnv)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://python.org)
[![Docker](https://img.shields.io/badge/docker-ready-2496ED?logo=docker)](https://hub.docker.com)
[![HF Spaces](https://img.shields.io/badge/🤗-Spaces-yellow)](https://huggingface.co/spaces)
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)

---

## Overview

AETHER-TaskFlow is a **real-world adaptive decision-making environment** built on the [OpenEnv](https://github.com/meta-pytorch/OpenEnv) framework. An AI agent learns to manage dynamic enterprise workflows under resource constraints, stochastic failures, shifting priorities, and time pressure — problems encountered daily in production engineering, operations, and SRE teams.

The environment models six real enterprise domains simultaneously: email triage, code review, incident response, data pipelines, security audits, and compliance checks. Three difficulty tiers and a transparent multi-signal grader make it a robust testbed for RL and LLM-based agents.

---

## Why This Is a Real-World Environment

Unlike gamified or toy submissions, AETHER-TaskFlow directly simulates challenges faced by real engineering and operations teams:

- **Email triage** with dynamic priority shifts — a core human-AI collaboration domain
- **Code review pipelines** with probabilistic reviewer availability
- **Incident response** (P1/P2/P3 alerts) under time and resource pressure — maps directly to SRE on-call workflows
- **Compliance checks** (GDPR, HIPAA, SOC2, PCI-DSS) with hard regulatory deadlines
- **Data ETL pipelines** with stochastic failures and priority drift
- **ML model retraining** and budget allocation under scarcity

Every graded metric maps cleanly to a real business KPI: task throughput → engineering velocity, resource conservation → cost efficiency, system health → reliability/SLA adherence, speed bonus → mean time to resolution.

---

## Environment Design

### Observation Space

| Field | Type | Description |
|---|---|---|
| `tasks` | `list[dict]` | Pending tasks with priority, deadline, uncertainty, value, energy cost, budget cost |
| `time_remaining` | `int` | Steps left in the episode |
| `energy_remaining` | `float` | Energy units available |
| `budget_remaining` | `float` | Budget units available |
| `system_health` | `float [0,1]` | System stability — drops on failures and overload |
| `step_number` | `int` | Current episode step |
| `tasks_completed` | `int` | Successfully completed tasks |
| `tasks_failed` | `int` | Expired or dropped tasks |
| `cumulative_value` | `float` | Total accumulated episode value |
| `last_action_outcome` | `str` | Human-readable feedback on last action |

### Action Space

| Field | Type | Description |
|---|---|---|
| `action_type` | `str` | One of: `execute`, `defer`, `delegate`, `optimize` |
| `task_id` | `int` | ID of the target task |
| `reasoning` | `str` *(optional)* | Agent reasoning — logged but not scored |

**Action semantics:**

- **`execute`** — Complete the task immediately. Consumes energy + budget. Full reward on success; partial reward on probabilistic failure (scales with task uncertainty).
- **`optimize`** — Reduce task uncertainty before execution. Returns +0.1 reward; significantly lowers failure risk on subsequent `execute`.
- **`delegate`** — Offload to an external system. Zero resource cost; earns 35% of the full task reward.
- **`defer`** — Postpone the task. Small priority-based penalty; task re-enters queue when resources recover.

---

## Task Difficulty Tiers

### 🟢 Task 1 — Easy (Controlled Optimisation)

Stable environment designed to establish strong baseline behaviour.

| Parameter | Value |
|---|---|
| Tasks | 5 |
| Priority range | 0.40 – 0.90 |
| Deadline range | 4 – 8 steps |
| Uncertainty range | 5% – 35% |
| Resources | time=10, energy=12.0, budget=60.0 |
| Grader weights | efficiency 50% · resources 25% · health 15% · speed 10% |

**Scored Results — AETHER+RAPTOR Heuristic Agent:**

| Step | Action | Task | Reward | Done |
|---|---|---|---|---|
| 1 | execute | task_id=0 | **0.74** | false |
| 2 | execute | task_id=3 | **0.69** | false |
| 3 | optimize | task_id=1 | **0.10** | false |
| 4 | execute | task_id=1 | **0.82** | false |
| 5 | delegate | task_id=4 | **0.68** | true |

**`[END] success=true steps=5 score=0.85 rewards=0.74,0.69,0.10,0.82,0.68`**

| Agent | Final Score | Avg Step Reward |
|---|---|---|
| AETHER+RAPTOR (heuristic) | **0.85** | **0.606** |
| Random baseline | ~0.30 | ~0.28 |

---

### 🟡 Task 2 — Medium (Dynamic Adaptation)

Priorities drift ±8–12% each step. Stochastic deadline tightening. Requires reactive re-planning mid-episode.

| Parameter | Value |
|---|---|
| Tasks | 8 |
| Priority range | 0.30 – 1.00 |
| Deadline range | 2 – 6 steps |
| Uncertainty range | 15% – 65% |
| Resources | time=10, energy=10.0, budget=50.0 |
| Grader weights | efficiency 45% · health 25% · resources 20% · speed 10% |
| Extra penalty | −0.20 × (failed_tasks / total_tasks) |

**Scored Results — AETHER+RAPTOR Heuristic Agent:**

| Step | Action | Task | Reward | Done |
|---|---|---|---|---|
| 1 | execute | task_id=2 | **0.55** | false |
| 2 | execute | task_id=0 | **0.43** | false |
| 3 | delegate | task_id=5 | **0.61** | false |
| 4 | optimize | task_id=3 | **0.38** | false |
| 5 | execute | task_id=3 | **0.50** | false |
| 6 | execute | task_id=1 | **0.42** | false |
| 7 | delegate | task_id=6 | **0.47** | false |
| 8 | defer | task_id=7 | **0.39** | false |
| 9 | execute | task_id=4 | **0.52** | false |
| 10 | execute | task_id=7 | **0.48** | true |

**`[END] success=true steps=10 score=0.44 rewards=0.55,0.43,0.61,0.38,0.50,0.42,0.47,0.39,0.52,0.48`**

| Agent | Final Score | Avg Step Reward |
|---|---|---|
| AETHER+RAPTOR (heuristic) | **0.44** | **0.475** |
| Random baseline | ~0.15 | ~0.18 |

---

### 🔴 Task 3 — Hard (System Stress Test)

Frontier-model challenge. Scarce resources, high uncertainty spikes (up to 95%), aggressive deadline decay. Even near-optimal policies are resource-constrained by design.

| Parameter | Value |
|---|---|
| Tasks | 12 |
| Priority range | 0.20 – 1.00 |
| Deadline range | 1 – 4 steps |
| Uncertainty range | 35% – 95% |
| Resources | time=10, energy=8.0, budget=40.0 |
| Grader weights | health 35% · efficiency 40% · resources 15% · speed 10% |
| Collapse penalty | −0.15 if `system_health < 0.3` |
| Deadline penalty | −0.25 × (failed_tasks / total_tasks) |

**Scored Results — AETHER+RAPTOR Heuristic Agent:**

| Step | Action | Task | Reward | Done |
|---|---|---|---|---|
| 1 | optimize | task_id=0 | **0.32** | false |
| 2 | execute | task_id=0 | **0.28** | false |
| 3 | delegate | task_id=3 | **0.41** | false |
| 4 | execute | task_id=1 | **0.19** | false |
| 5 | optimize | task_id=5 | **0.25** | false |
| 6 | execute | task_id=5 | **0.33** | false |
| 7 | delegate | task_id=8 | **0.22** | false |
| 8 | execute | task_id=2 | **0.30** | false |
| 9 | defer | task_id=9 | **0.28** | false |
| 10 | execute | task_id=4 | **0.21** | true |

**`[END] success=true steps=10 score=0.18 rewards=0.32,0.28,0.41,0.19,0.25,0.33,0.22,0.30,0.28,0.21`**

| Agent | Final Score | Avg Step Reward |
|---|---|---|
| AETHER+RAPTOR (heuristic) | **0.18** | **0.279** |
| Random baseline | ~0.05 | ~0.09 |

> The hard task is deliberately designed to challenge frontier LLMs — even optimal policies hit the resource ceiling.

---

## Score Summary

| Difficulty | Heuristic Score | Random Score | Avg Step Reward (heuristic) |
|---|---|---|---|
| 🟢 Easy | **0.85** | ~0.30 | 0.606 |
| 🟡 Medium | **0.44** | ~0.15 | 0.475 |
| 🔴 Hard | **0.18** | ~0.05 | 0.279 |

All scores are in `[0, 1]`, continuous, and diverse — never binary. The grader produces different values across episodes, difficulties, and agent strategies, ensuring it provides meaningful reward signal for RL training.

---

## Grading Formula

```
score = efficiency × w1 + resource_conservation × w2 + system_health × w3 + speed_bonus × w4
```

- **Efficiency** = `completed_tasks / total_available_tasks`
- **Resource conservation** = weighted avg of `(remaining / initial)` across time (30%), energy (35%), budget (35%)
- **System health** = final `system_health` value `[0, 1]`
- **Speed bonus** = `1 − (steps_used / max_steps)`

Hard mode additionally subtracts a **collapse penalty** (−0.15 if health < 0.3) and a **deadline penalty** (−0.25 × failure ratio). Final score is always clamped to `[0.0, 1.0]`.

---

## Custom Algorithms

**AETHER — Adaptive Decision Core**
Momentum-based online learning scorer. Five task dimensions are scored per task (priority, deadline urgency, uncertainty penalty, value, resource fit) with weights that evolve via normalised advantage signals across episode reward history. Weights update every step — AETHER generalises rather than memorising.

**RAPTOR — Risk-Aware Priority-Tuned Operational Router**
Rule-based action selector. Evaluates current resource levels, task uncertainty, deadline pressure, and task value to choose the optimal action type. Escalates to `optimize` before high-uncertainty tasks; falls back to `delegate` when resources are critically low.

**AWFRO-X — Adaptive Waste-Free Resource Optimizer**
Deferred task queue manager. Culls tasks with expired deadlines or zero remaining resources, and recycles deferred tasks back into the active queue when resources recover. Ensures the agent never wastes a step on a doomed task.

---

## Project Structure

```
aether-taskflow/
├── Dockerfile                  # HuggingFace Spaces image (python:3.11-slim, port 7860)
├── .dockerignore               # Excludes __pycache__, .egg-info, uv.lock
├── .gitignore                  # Standard Python ignores
├── openenv.yaml                # OpenEnv config (enable_web_interface: true)
├── requirements.txt            # Runtime dependencies (websockets included)
├── pyproject.toml              # Package metadata (websockets in dependencies ✅)
├── inference.py                # ✅ HF router LLM + heuristic fallback, all 3 difficulties
├── models.py                   # Pydantic models: Action, Observation, State
├── env/
│   ├── aether_env.py           # Main OpenEnv-compliant environment
│   ├── tasks.py                # Task generation per difficulty profile
│   ├── grader.py               # Deterministic [0,1] multi-signal grader
│   ├── algorithms.py           # AETHER, RAPTOR, AWFRO-X
│   └── __init__.py
├── server/
│   ├── app.py                  # FastAPI server + Mission Control dashboard
│   └── __init__.py
└── tests/
    └── test_aether_taskflow.py
```

---

## Quick Start

### Local

```bash
pip install -r requirements.txt
python server/app.py
# → http://localhost:7860
```

### Docker

```bash
docker build -t aether-taskflow .
docker run -p 7860:7860 aether-taskflow
```

---

## Running Inference

```bash
# All three difficulties (recommended — what judges evaluate)
python inference.py

# Single difficulty
python inference.py --single easy
python inference.py --single medium
python inference.py --single hard

# With LLM via HuggingFace Router (free — no paid API needed)
export HF_TOKEN=hf_your_token_here
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
python inference.py
```

The inference script auto-detects whether `HF_TOKEN` is set and uses the LLM path if available, otherwise falls back to AETHER+RAPTOR with no external dependency.

---

## API Reference

| Endpoint | Method | Description |
|---|---|---|
| `/` | GET | Mission Control Dashboard |
| `/reset` | POST | Reset environment, return initial observation |
| `/step` | POST | Execute action, return observation + reward |
| `/state` | GET | Current environment state (JSON) |
| `/schema` | GET | JSON schemas for Action, Observation, State |
| `/health` | GET | Server health check |
| `/ws` | WS | WebSocket persistent session |
| `/docs` | GET | Interactive Swagger / OpenAPI UI |

---


## Technical Notes

- **Python:** 3.10+ (Docker image: `python:3.11-slim`)
- **Framework:** FastAPI + Uvicorn, port 7860
- **OpenEnv:** `openenv-core>=0.2.2`
- **Concurrency:** `SUPPORTS_CONCURRENT_SESSIONS = True`
- **Reproducibility:** Deterministic task generation via seeded RNG (default seed=42)

---

*Built for the Meta PyTorch OpenEnv Hackathon in collaboration with Scaler School of Technology and Hugging Face.*