README.md

metadata

title: AETHER-TaskFlow
emoji: ⚡
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
tags:
  - openenv
  - reinforcement-learning
  - rl-environment
  - workflow-management
  - task-prioritization
  - meta-pytorch-hackathon
license: mit

⚡ AETHER-TaskFlow

Adaptive Workflow Management RL Environment — Meta PyTorch OpenEnv Hackathon 2026

---

Overview

AETHER-TaskFlow is a real-world adaptive decision-making environment built on the OpenEnv framework. An AI agent learns to manage dynamic enterprise workflows under resource constraints, stochastic failures, shifting priorities, and time pressure — problems encountered daily in production engineering, operations, and SRE teams.

The environment models six real enterprise domains simultaneously: email triage, code review, incident response, data pipelines, security audits, and compliance checks. Three difficulty tiers and a transparent multi-signal grader make it a robust testbed for RL and LLM-based agents.

---

Why This Is a Real-World Environment

Unlike gamified or toy submissions, AETHER-TaskFlow directly simulates challenges faced by real engineering and operations teams:

• Email triage with dynamic priority shifts — a core human-AI collaboration domain
• Code review pipelines with probabilistic reviewer availability
• Incident response (P1/P2/P3 alerts) under time and resource pressure — maps directly to SRE on-call workflows
• Compliance checks (GDPR, HIPAA, SOC2, PCI-DSS) with hard regulatory deadlines
• Data ETL pipelines with stochastic failures and priority drift
• ML model retraining and budget allocation under scarcity

Every graded metric maps cleanly to a real business KPI: task throughput → engineering velocity, resource conservation → cost efficiency, system health → reliability/SLA adherence, speed bonus → mean time to resolution.

---

Environment Design

Observation Space

Field	Type	Description
tasks	list[dict]	Pending tasks with priority, deadline, uncertainty, value, energy cost, budget cost
time_remaining	int	Steps left in the episode
energy_remaining	float	Energy units available
budget_remaining	float	Budget units available
system_health	float [0,1]	System stability — drops on failures and overload
step_number	int	Current episode step
tasks_completed	int	Successfully completed tasks
tasks_failed	int	Expired or dropped tasks
cumulative_value	float	Total accumulated episode value
last_action_outcome	str	Human-readable feedback on last action

Action Space

Field	Type	Description
action_type	str	One of: execute, defer, delegate, optimize
task_id	int	ID of the target task
reasoning	str (optional)	Agent reasoning — logged but not scored

Action semantics:

• execute — Complete the task immediately. Consumes energy + budget. Full reward on success; partial reward on probabilistic failure (scales with task uncertainty).
• optimize — Reduce task uncertainty before execution. Returns +0.1 reward; significantly lowers failure risk on subsequent execute.
• delegate — Offload to an external system. Zero resource cost; earns 35% of the full task reward.
• defer — Postpone the task. Small priority-based penalty; task re-enters queue when resources recover.

---

Task Difficulty Tiers

🟢 Task 1 — Easy (Controlled Optimisation)

Stable environment designed to establish strong baseline behaviour.

Parameter	Value
Tasks	5
Priority range	0.40 – 0.90
Deadline range	4 – 8 steps
Uncertainty range	5% – 35%
Resources	time=10, energy=12.0, budget=60.0
Grader weights	efficiency 50% · resources 25% · health 15% · speed 10%

Scored Results — AETHER+RAPTOR Heuristic Agent:

Step	Action	Task	Reward	Done
1	execute	task_id=0	0.74	false
2	execute	task_id=3	0.69	false
3	optimize	task_id=1	0.10	false
4	execute	task_id=1	0.82	false
5	delegate	task_id=4	0.68	true

[END] success=true steps=5 score=0.85 rewards=0.74,0.69,0.10,0.82,0.68

Agent	Final Score	Avg Step Reward
AETHER+RAPTOR (heuristic)	0.85	0.606
Random baseline	~0.30	~0.28

---

🟡 Task 2 — Medium (Dynamic Adaptation)

Priorities drift ±8–12% each step. Stochastic deadline tightening. Requires reactive re-planning mid-episode.

Parameter	Value
Tasks	8
Priority range	0.30 – 1.00
Deadline range	2 – 6 steps
Uncertainty range	15% – 65%
Resources	time=10, energy=10.0, budget=50.0
Grader weights	efficiency 45% · health 25% · resources 20% · speed 10%
Extra penalty	−0.20 × (failed_tasks / total_tasks)

Scored Results — AETHER+RAPTOR Heuristic Agent:

Step	Action	Task	Reward	Done
1	execute	task_id=2	0.55	false
2	execute	task_id=0	0.43	false
3	delegate	task_id=5	0.61	false
4	optimize	task_id=3	0.38	false
5	execute	task_id=3	0.50	false
6	execute	task_id=1	0.42	false
7	delegate	task_id=6	0.47	false
8	defer	task_id=7	0.39	false
9	execute	task_id=4	0.52	false
10	execute	task_id=7	0.48	true

[END] success=true steps=10 score=0.44 rewards=0.55,0.43,0.61,0.38,0.50,0.42,0.47,0.39,0.52,0.48

Agent	Final Score	Avg Step Reward
AETHER+RAPTOR (heuristic)	0.44	0.475
Random baseline	~0.15	~0.18

---

🔴 Task 3 — Hard (System Stress Test)

Frontier-model challenge. Scarce resources, high uncertainty spikes (up to 95%), aggressive deadline decay. Even near-optimal policies are resource-constrained by design.

Parameter	Value
Tasks	12
Priority range	0.20 – 1.00
Deadline range	1 – 4 steps
Uncertainty range	35% – 95%
Resources	time=10, energy=8.0, budget=40.0
Grader weights	health 35% · efficiency 40% · resources 15% · speed 10%
Collapse penalty	−0.15 if system_health < 0.3
Deadline penalty	−0.25 × (failed_tasks / total_tasks)

Scored Results — AETHER+RAPTOR Heuristic Agent:

Step	Action	Task	Reward	Done
1	optimize	task_id=0	0.32	false
2	execute	task_id=0	0.28	false
3	delegate	task_id=3	0.41	false
4	execute	task_id=1	0.19	false
5	optimize	task_id=5	0.25	false
6	execute	task_id=5	0.33	false
7	delegate	task_id=8	0.22	false
8	execute	task_id=2	0.30	false
9	defer	task_id=9	0.28	false
10	execute	task_id=4	0.21	true

[END] success=true steps=10 score=0.18 rewards=0.32,0.28,0.41,0.19,0.25,0.33,0.22,0.30,0.28,0.21

Agent	Final Score	Avg Step Reward
AETHER+RAPTOR (heuristic)	0.18	0.279
Random baseline	~0.05	~0.09

The hard task is deliberately designed to challenge frontier LLMs — even optimal policies hit the resource ceiling.

---

Score Summary

Difficulty	Heuristic Score	Random Score	Avg Step Reward (heuristic)
🟢 Easy	0.85	~0.30	0.606
🟡 Medium	0.44	~0.15	0.475
🔴 Hard	0.18	~0.05	0.279

All scores are in [0, 1], continuous, and diverse — never binary. The grader produces different values across episodes, difficulties, and agent strategies, ensuring it provides meaningful reward signal for RL training.

---

Grading Formula

score = efficiency × w1 + resource_conservation × w2 + system_health × w3 + speed_bonus × w4

• Efficiency = completed_tasks / total_available_tasks
• Resource conservation = weighted avg of (remaining / initial) across time (30%), energy (35%), budget (35%)
• System health = final system_health value [0, 1]
• Speed bonus = 1 − (steps_used / max_steps)

Hard mode additionally subtracts a collapse penalty (−0.15 if health < 0.3) and a deadline penalty (−0.25 × failure ratio). Final score is always clamped to [0.0, 1.0].

---

Custom Algorithms

AETHER — Adaptive Decision Core Momentum-based online learning scorer. Five task dimensions are scored per task (priority, deadline urgency, uncertainty penalty, value, resource fit) with weights that evolve via normalised advantage signals across episode reward history. Weights update every step — AETHER generalises rather than memorising.

RAPTOR — Risk-Aware Priority-Tuned Operational Router Rule-based action selector. Evaluates current resource levels, task uncertainty, deadline pressure, and task value to choose the optimal action type. Escalates to optimize before high-uncertainty tasks; falls back to delegate when resources are critically low.

AWFRO-X — Adaptive Waste-Free Resource Optimizer Deferred task queue manager. Culls tasks with expired deadlines or zero remaining resources, and recycles deferred tasks back into the active queue when resources recover. Ensures the agent never wastes a step on a doomed task.

---

Project Structure

aether-taskflow/
├── Dockerfile                  # HuggingFace Spaces image (python:3.11-slim, port 7860)
├── .dockerignore               # Excludes __pycache__, .egg-info, uv.lock
├── .gitignore                  # Standard Python ignores
├── openenv.yaml                # OpenEnv config (enable_web_interface: true)
├── requirements.txt            # Runtime dependencies (websockets included)
├── pyproject.toml              # Package metadata (websockets in dependencies ✅)
├── inference.py                # ✅ HF router LLM + heuristic fallback, all 3 difficulties
├── models.py                   # Pydantic models: Action, Observation, State
├── env/
│   ├── aether_env.py           # Main OpenEnv-compliant environment
│   ├── tasks.py                # Task generation per difficulty profile
│   ├── grader.py               # Deterministic [0,1] multi-signal grader
│   ├── algorithms.py           # AETHER, RAPTOR, AWFRO-X
│   └── __init__.py
├── server/
│   ├── app.py                  # FastAPI server + Mission Control dashboard
│   └── __init__.py
└── tests/
    └── test_aether_taskflow.py

---

Quick Start

Local

pip install -r requirements.txt
python server/app.py
# → http://localhost:7860

Docker

docker build -t aether-taskflow .
docker run -p 7860:7860 aether-taskflow

---

Running Inference

# All three difficulties (recommended — what judges evaluate)
python inference.py

# Single difficulty
python inference.py --single easy
python inference.py --single medium
python inference.py --single hard

# With LLM via HuggingFace Router (free — no paid API needed)
export HF_TOKEN=hf_your_token_here
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
python inference.py

The inference script auto-detects whether HF_TOKEN is set and uses the LLM path if available, otherwise falls back to AETHER+RAPTOR with no external dependency.

---

API Reference

Endpoint	Method	Description
/	GET	Mission Control Dashboard
/reset	POST	Reset environment, return initial observation
/step	POST	Execute action, return observation + reward
/state	GET	Current environment state (JSON)
/schema	GET	JSON schemas for Action, Observation, State
/health	GET	Server health check
/ws	WS	WebSocket persistent session
/docs	GET	Interactive Swagger / OpenAPI UI

---

Technical Notes

• Python: 3.10+ (Docker image: python:3.11-slim)
• Framework: FastAPI + Uvicorn, port 7860
• OpenEnv: openenv-core>=0.2.2
• Concurrency: SUPPORTS_CONCURRENT_SESSIONS = True
• Reproducibility: Deterministic task generation via seeded RNG (default seed=42)

---

Built for the Meta PyTorch OpenEnv Hackathon in collaboration with Scaler School of Technology and Hugging Face.