SchedulingOptEnv
A Markov Decision Environment for Training Autonomous
Scheduling Optimisation Agents
Meta × Scaler — OpenEnv Hackathon Submission
---
## Abstract
We present **SchedulingOptEnv**, a real-world training environment for autonomous AI agents built upon the OpenEnv framework. The environment formalises combinatorial scheduling optimisation as a sequential decision problem, exposing agents to three progressively challenging sub-tasks: binary feasibility determination, multi-class constraint-violation classification, and full schedule repair. Each task is paired with a structured, differentiable reward function that provides dense, partial-progress signals rather than sparse binary outcomes. A 12-instance scheduling corpus covering five distinct constraint-violation classes, a FastAPI inference server, and a GPT-4o-mini baseline are included. The environment is deployable as a Docker container on Hugging Face Spaces with a single command.
---
## 1. Introduction
Combinatorial scheduling — the assignment of jobs to machines subject to resource, temporal, and precedence constraints — is a foundational problem in operations research, manufacturing, cloud computing, and logistics. Despite its industrial importance, existing benchmarks for evaluating AI agents on scheduling tasks are either purely offline (single-pass solution quality) or narrowly scoped to continuous optimisation rather than the constraint-satisfaction and repair workflow practised by human planners.
OpenEnv [1] provides an abstraction layer for building *interactive* environments where agents act, receive graded feedback, and improve across episodes. SchedulingOptEnv fills a gap by framing schedule analysis and repair as a Markov Decision Process (MDP) with:
- A well-defined **observation space** (JSON-encoded scheduling instance, task context, step counter)
- A structured **action space** (categorical labels or JSON repair schedules)
- A **multi-component reward function** that awards partial credit for structurally valid but suboptimal repairs
- Three **difficulty tiers** mirroring the cognitive complexity gradient faced by human schedulers
---
## 2. Environment Design
### 2.1 MDP Formulation
| Component | Definition |
|-----------|-----------|
| State *S* | Current scheduling instance, task type, step count, episode history |
| Observation *O* | `{schedule_instance: str (JSON), task_id, context, step_number}` |
| Action *A* | `{response: str, task_id: str}` |
| Reward *R* | Float ∈ [0.0, 1.0] from task-specific grader |
| Horizon *T* | Task-dependent: 3 / 5 / 8 steps |
| Terminal | *done* = True when *T* reached or *R* ≥ 0.95 |
### 2.2 Scheduling Instance Corpus
The environment ships with **12 curated scheduling instances** spanning five constraint-violation classes plus two fully feasible baselines. Instances are drawn from a task-aware pool: feasibility-check episodes see all 12, while classification and repair episodes see only the 10 infeasible instances.
| # | Feasible | Violation Class | Description |
|---|----------|----------------|-------------|
| 0 | No | `resource_overload` | J1 and J2 overlap on single-capacity machine M1 |
| 1 | No | `deadline_violation` | J1 starts late and finishes after hard deadline |
| 2 | No | `precedence_violation` | J2 starts before its predecessor J1 finishes |
| 3 | No | `availability_conflict` | J1 scheduled outside machine operating hours |
| 4 | No | `capacity_exceeded` | 3 concurrent jobs on capacity-2 machine |
| 5 | No | `resource_overload` | Pairwise overlap of J1 and J2 on capacity-1 machine |
| 6 | No | `deadline_violation` | Precedence chain forces J3 past hard deadline |
| 7 | No | `precedence_violation` | J3 starts before both predecessors complete |
| 8 | No | `availability_conflict` | J1 extends into machine maintenance window |
| 9 | No | `capacity_exceeded` | 4 concurrent jobs on capacity-3 machine |
| 10 | Yes | — | Fully feasible 3-job, 2-machine schedule |
| 11 | Yes | — | Fully feasible 5-job, 3-machine schedule with precedence |
---
## 3. Tasks
### Task 1 — Feasibility Check *(Easy)*
**Objective:** Given a JSON-encoded scheduling instance (jobs, machines, proposed assignments), determine whether the schedule satisfies all constraints.
**Action space:** `{"feasible", "infeasible"}`
**Grading function:**
```
R(a, g) = 1.0 if normalise(a) == ground_truth
0.1 if a is non-empty but incorrect
0.0 if a is empty
```
**Episode horizon:** 3 steps. **Target agent accuracy:** ~90%.
---
### Task 2 — Conflict Classification *(Medium)*
**Objective:** Identify the constraint violation present in an infeasible schedule from the closed vocabulary:
`{resource_overload, deadline_violation, precedence_violation, availability_conflict, capacity_exceeded}`
**Grading function:**
```
R(a, g) = 1.0 if a == ground_truth (exact)
0.5 if a ∈ related_group(ground_truth) (partial)
0.1 if a ∈ valid_categories \ related_group(g) (wrong family)
0.0 if a ∉ valid_categories (unparseable)
```
where `related_groups = [{resource_overload, capacity_exceeded}, {deadline_violation, precedence_violation}]`.
**Episode horizon:** 5 steps. **Target agent accuracy:** ~60%.
---
### Task 3 — Schedule Repair *(Hard)*
**Objective:** Return a corrected schedule as a JSON object that resolves all constraint violations and minimises total makespan.
**Required JSON format:**
```json
{
"assignments": [
{"job_id": "J1", "machine_id": "M1", "start_time": 0},
{"job_id": "J2", "machine_id": "M1", "start_time": 4}
]
}
```
**Grading function (additive, max 1.0):**
```
R(a, g) = 0.2 × parseable_json(a)
+ 0.2 × valid_schema(a, g)
+ 0.4 × constraint_satisfaction_ratio(a, g)
+ 0.2 × optimality_score(makespan(a), makespan*(g))
```
where:
- `parseable_json(a)` — 1 if the response parses as valid JSON, else 0
- `valid_schema(a, g)` — 1 if all required fields are present and all jobs are assigned, else 0
- `constraint_satisfaction_ratio(a, g)` — fraction of four constraint categories satisfied:
capacity, deadlines, precedence, availability (each worth 0.25)
- `optimality_score(m, m*)` — 1.0 if *m* ≤ 1.30·*m** ; 0.5 if *m* ≤ 1.60·*m** ; 0 otherwise
**Episode horizon:** 8 steps. **Target agent accuracy:** ~30%.
---
## 4. Server API
The environment is exposed over HTTP via a FastAPI server on port **7860** (Hugging Face Spaces default).
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/health` | Liveness probe — returns `{"status": "ok"}` |
| `POST` | `/reset` | Begin new episode: `{"task_id": "feasibility_check"}` |
| `POST` | `/step` | Submit action: `{"response": "infeasible", "task_id": "feasibility_check"}` |
| `GET` | `/state` | Full internal state snapshot |
| `GET` | `/tasks` | Task catalogue with action schemas |
| `POST` | `/grader` | Direct grader invocation for offline evaluation |
| `GET` | `/baseline` | Trigger baseline inference; returns per-task scores |
---
## 5. Baseline
A standalone inference script (`baseline.py`) evaluates GPT-4o-mini on all three tasks. When `OPENAI_API_KEY` is not set, the script falls back to oracle mock responses, enabling offline verification of the grading pipeline without API access.
### 5.1 Baseline Scores (Mock / Oracle)
| Task | Instances | Average Score |
|------|-----------|--------------|
| Feasibility Check | 12 | 1.000 |
| Conflict Classification | 10 | 1.000 |
| Schedule Repair | 10 | 1.000 |
| **Overall** | | **1.000** |
---
## 6. Setup and Deployment
### 6.1 Prerequisites
| Requirement | Version |
|-------------|---------|
| Python | ≥ 3.11 |
| pip | ≥ 22.0 |
| Docker *(optional)* | ≥ 20.10 |
| Git | ≥ 2.30 |
### 6.2 Local Installation
```bash
# 1. Clone the repository
git clone https://github.com/Vittal-Mukunda/OpenEnv-Hackathon-Meta-x-Scaler.git
cd OpenEnv-Hackathon-Meta-x-Scaler
# 2. Create and activate a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate # Linux / macOS
# .venv\Scripts\activate # Windows
# 3. Install dependencies
pip install -r requirements.txt
# 4. Launch the server
uvicorn server:app --host 0.0.0.0 --port 7860
# 5. Verify the server is running
curl http://localhost:7860/health
# Expected: {"status":"ok"}
```
### 6.3 Docker Deployment
```bash
# Build the image
docker build -t scheduling-opt-env .
# Run the container
docker run -p 7860:7860 scheduling-opt-env
# Verify
curl http://localhost:7860/health
```
### 6.4 Hugging Face Spaces
Push this repository to a Hugging Face Space configured with the **Docker** SDK. The server listens on port 7860, which Spaces exposes automatically. No additional configuration is required.
### 6.5 Running the Baseline
```bash
# Without API key (uses oracle mock responses — scores 1.0 on all tasks)
python baseline.py
# With OpenAI API key (evaluates GPT-4o-mini)
export OPENAI_API_KEY=sk-...
python baseline.py
```
---
## 7. Example Interaction
```bash
# 1. Health check
curl http://localhost:7860/health
# 2. Start a feasibility-check episode
curl -X POST http://localhost:7860/reset \
-H "Content-Type: application/json" \
-d '{"task_id": "feasibility_check"}'
# 3. Submit a feasibility answer
curl -X POST http://localhost:7860/step \
-H "Content-Type: application/json" \
-d '{"response": "infeasible", "task_id": "feasibility_check"}'
# 4. Start a conflict-classification episode
curl -X POST http://localhost:7860/reset \
-H "Content-Type: application/json" \
-d '{"task_id": "conflict_classification"}'
# 5. Classify the violation
curl -X POST http://localhost:7860/step \
-H "Content-Type: application/json" \
-d '{"response": "resource_overload", "task_id": "conflict_classification"}'
# 6. Start a schedule-repair episode
curl -X POST http://localhost:7860/reset \
-H "Content-Type: application/json" \
-d '{"task_id": "schedule_repair"}'
# 7. Submit a repaired schedule
curl -X POST http://localhost:7860/step \
-H "Content-Type: application/json" \
-d '{
"response": "{\"assignments\": [{\"job_id\": \"J1\", \"machine_id\": \"M1\", \"start_time\": 0}]}",
"task_id": "schedule_repair"
}'
# 8. Inspect environment state
curl http://localhost:7860/state
# 9. Invoke a grader directly
curl -X POST http://localhost:7860/grader \
-H "Content-Type: application/json" \
-d '{
"action": {"response": "deadline_violation", "task_id": "conflict_classification"},
"ground_truth": {"violation_type": "deadline_violation"}
}'
```
---
## 8. Project Structure
```
.
├── openenv.yaml # OpenEnv metadata manifest
├── models.py # Pydantic v2 data models (Observation, Action, Reward)
├── environment.py # SchedulingOptEnv core (reset / step / state + instance bank)
├── server.py # FastAPI HTTP server (7 endpoints)
├── baseline.py # GPT-4o-mini baseline with oracle fallback
├── Dockerfile # Container definition (python:3.11-slim, port 7860)
├── requirements.txt # Python dependencies
├── tasks/
│ ├── __init__.py # Task module exports
│ ├── task1_easy.py # Feasibility check — episode runner + instance accessor
│ ├── task2_medium.py # Conflict classification — episode runner + instance accessor
│ └── task3_hard.py # Schedule repair — episode runner + instance accessor
└── graders/
├── __init__.py # Grader exports (FeasibilityGrader, ConflictGrader, RepairGrader)
├── grader_detection.py # Grader: feasibility (binary, synonym-aware)
├── grader_classification.py # Grader: conflict classification (family-aware partial credit)
└── grader_fix.py # Grader: schedule repair (4-component additive reward)
```
---
## 9. Dependencies
| Package | Version | Purpose |
|---------|---------|---------|
| `fastapi` | ≥ 0.104 | HTTP server framework |
| `uvicorn` | ≥ 0.24 | ASGI server |
| `pydantic` | ≥ 2.5 | Data validation and serialisation |
| `openai` | ≥ 1.6 | LLM baseline inference |
| `pyyaml` | ≥ 6.0 | YAML manifest parsing |
| `httpx` | ≥ 0.25 | Async HTTP client |
---
## 10. References
[1] OpenEnv Framework. *Building Real-World AI Agent Training Environments*. Meta × Scaler Hackathon, 2026.
[2] Pinedo, M. L. *Scheduling: Theory, Algorithms, and Systems* (5th ed.). Springer, 2016.
[3] Garey, M. R., & Johnson, D. S. *Computers and Intractability: A Guide to the Theory of NP-Completeness*. W. H. Freeman, 1979.
[4] Zhang, C. et al. *Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learning*. NeurIPS 2020.
[5] Kwon, Y.-D. et al. *POMO: Policy Optimization with Multiple Optima for Reinforcement Learning*. NeurIPS 2020.