SchedulingOptEnv
A Markov Decision Environment for Training Autonomous Scheduling Optimisation Agents
Meta × Scaler — OpenEnv Hackathon Submission
Team Vector — Vittal Mukunda · Nikhilesh Nuthalapati · Stavan Rahul Khobare (Lead)
Table of Contents
- Abstract
- Introduction
- Background and Motivation
- System Architecture
- Methodology
- The Scheduling Instance Corpus
- Task Definitions and Grading Functions
- Reward Design Philosophy
- Data Models
- API Specification
- Inference and Baseline
- Setup and Installation
- How to Get Results
- End-to-End Walkthrough
- Evaluation and Scoring
- Project Structure
- Dependencies
- Glossary
1. Abstract
We present SchedulingOptEnv, a real-world AI agent training environment built on the OpenEnv framework for the Meta × Scaler OpenEnv Hackathon. The environment frames combinatorial scheduling optimisation as a sequential decision problem — a Markov Decision Process (MDP) — exposing agents to three progressively harder sub-tasks: binary feasibility determination, multi-class constraint-violation classification, and full schedule repair.
Each task is paired with a structured, differentiable reward function that provides dense, partial-progress signals rather than sparse binary outcomes. This design choice ensures that an AI agent always has a meaningful learning signal at every step, even when its answer is wrong — accelerating convergence during training.
The environment ships with:
- A 12-instance scheduling corpus covering five distinct constraint-violation classes and two fully feasible baseline schedules
- A FastAPI HTTP inference server with 7 endpoints, exposing the full OpenEnv API contract
- A standalone inference script (
inference.py) using the OpenAI client with configurableAPI_BASE_URL,MODEL_NAME, andHF_TOKENenvironment variables - A GPT-4o-mini baseline with oracle mock fallback for offline verification
- A Docker container deployable to Hugging Face Spaces with a single command
Oracle baseline achieves perfect scores (1.0) on all three tasks. The environment is fully spec-compliant with the OpenEnv standard and passes all automated pre-submission validation checks.
2. Introduction
2.1 What Is This Project?
In plain terms: SchedulingOptEnv is a training gym where AI learns to detect, classify, and fix broken work schedules.
Imagine a factory that needs to schedule 5 machines and 20 jobs. Some jobs can only run after other jobs finish. Some machines can only handle 2 jobs at once. Some machines are offline for maintenance from 3pm to 6pm. If someone hands the factory manager a schedule that violates any of these rules, the schedule is broken and production will fail.
This project creates a structured training playground where an AI agent can:
- Look at a proposed schedule
- Decide whether it is valid or broken
- If broken, identify which type of rule was broken
- Produce a corrected schedule that satisfies all constraints and minimises total time
The agent learns by trial and error: it receives a graded reward when it responds, and over thousands of practice rounds, it improves.
2.2 Why Scheduling?
Scheduling is one of the most practically important and computationally hard problems in computer science. It appears across nearly every industry:
| Industry | Scheduling Problem |
|---|---|
| Manufacturing | Assigning jobs to machines, managing shift constraints |
| Healthcare | Booking operating rooms, allocating staff and equipment |
| Aviation | Scheduling flights, crew rotations, gate assignments |
| Cloud computing | Allocating compute tasks to servers with capacity limits |
| Construction | Sequencing tasks with dependencies and resource limits |
| Logistics | Routing vehicles with time-window constraints |
Despite its industrial importance, prior benchmarks for evaluating AI agents on scheduling tasks were either:
- Offline only — agents see a problem once and produce a single answer, with no iterative improvement
- Narrowly scoped — focused purely on optimisation, not on the constraint-satisfaction and repair workflow that human planners actually use day-to-day
SchedulingOptEnv fills this gap. It creates an interactive, multi-step environment where an agent can reason, receive feedback, and refine its answers — mirroring the real cognitive workflow of a scheduling expert.
2.3 What Is OpenEnv?
OpenEnv is a framework specification by Meta and Hugging Face for building standardised, interactive AI agent training environments. It defines a common API contract — step(), reset(), state() — that any AI training system can connect to. By building SchedulingOptEnv on top of OpenEnv, any compatible AI agent or training loop can immediately start learning from our environment without any custom integration work.
3. Background and Motivation
3.1 The Scheduling Problem Formally
A scheduling instance consists of:
A set of jobs
J = {J1, J2, ..., Jn}, each with:duration— how long the job takes to rundeadline— the latest time by which the job must be completeddependencies— a list of job IDs that must complete before this job startsresource_req— how much capacity this job consumes on its assigned machine
A set of machines
M = {M1, M2, ..., Mm}, each with:capacity— how many jobs can run concurrentlyavailable_start— the earliest time the machine is operationalavailable_end— the latest time the machine is operational
A proposed schedule — a set of assignments
{(job_id, machine_id, start_time)}specifying when and where each job runs
A schedule is feasible if and only if it satisfies all four constraint categories simultaneously:
| Constraint | Formal Definition |
|---|---|
| Capacity | At every time t, the number of jobs concurrently running on machine m does not exceed m.capacity |
| Deadline | For every job j, start_time(j) + duration(j) ≤ deadline(j) |
| Precedence | For every job j and every predecessor p ∈ dependencies(j): start_time(j) ≥ start_time(p) + duration(p) |
| Availability | For every job j on machine m: start_time(j) ≥ m.available_start and start_time(j) + duration(j) ≤ m.available_end |
A schedule is infeasible if it violates any one of these.
3.2 Why This Is Hard
Even checking whether a schedule is feasible is non-trivial for large instances. Repairing a broken schedule to be both feasible and optimal is NP-hard in general. Human planners develop intuition over years of experience. This project asks: can we create an environment rich enough for an AI to develop similar intuition through reinforcement learning?
3.3 The Gap in Existing Benchmarks
Existing scheduling benchmarks (e.g., JSP, FJSP) focus on finding optimal schedules from scratch. They do not model:
- The iterative detect → classify → repair workflow
- Partial-credit rewards for nearly-correct repairs
- Multi-step episodes where agents can refine answers
- Diverse constraint violation types as a classification target
SchedulingOptEnv addresses all of these gaps.
4. System Architecture
The system is organised into five logical layers, each building on the one below it.
┌─────────────────────────────────────────────────────────────────────────┐
│ LAYER 5 — EXTERNAL CLIENTS │
│ AI agents, training loops, researchers, the inference script │
│ Communicate via HTTP JSON requests to port 7860 │
└────────────────────────────────┬────────────────────────────────────────┘
│ HTTP / JSON
┌────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 4 — API SERVER (server.py) │
│ FastAPI application with 7 REST endpoints │
│ Validates requests, routes to environment, serialises responses │
│ Raises HTTP 400/422 for malformed input │
└────────────────────────────────┬────────────────────────────────────────┘
│ Python function calls
┌────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 3 — ENVIRONMENT (environment.py) │
│ SchedulingOptEnv class — the core "game engine" │
│ Manages: episode lifecycle, instance selection, step counting, │
│ termination logic, cumulative reward tracking, action history │
│ Holds: INSTANCE_BANK (12 scheduling problems), grader singletons │
└──────────────────┬──────────────────────────────────┬───────────────────┘
│ │
┌────────────▼──────────┐ ┌────────────▼──────────────────┐
│ LAYER 2a — TASKS │ │ LAYER 2b — GRADERS │
│ (tasks/ folder) │ │ (graders/ folder) │
│ │ │ │
│ task1_easy.py │ │ grader_detection.py │
│ task2_medium.py │ │ FeasibilityGrader │
│ task3_hard.py │ │ (binary, synonym-aware) │
│ │ │ │
│ Each task exposes: │ │ grader_classification.py │
│ - episode runner │ │ ConflictGrader │
│ - instance accessor │ │ (family-aware partial) │
│ - task metadata │ │ │
└────────────┬──────────┘ │ grader_fix.py │
│ │ RepairGrader │
│ │ (4-component additive) │
│ └────────────┬──────────────────┘
│ │
┌──────────────────▼──────────────────────────────────▼──────────────────┐
│ LAYER 1 — DATA MODELS (models.py) │
│ Pydantic v2 schemas: Observation, Action, Reward │
│ The shared type contract all layers use to communicate │
│ Enforces field types, constraints (score ∈ [0.0, 1.0]), and docs │
└─────────────────────────────────────────────────────────────────────────┘
Additional components (alongside Layer 4):
inference.py — OpenAI-client inference script (API_BASE_URL / MODEL_NAME / HF_TOKEN)
baseline.py — GPT-4o-mini evaluation with oracle mock fallback
openenv.yaml — OpenEnv metadata manifest (machine-readable spec declaration)
Dockerfile — Container definition (python:3.11-slim, port 7860)
4.1 Component Responsibilities
models.py — Defines the three Pydantic v2 schemas that every other component imports:
Observation— what the agent sees at each step (the schedule, task ID, instructions, step number)Action— what the agent submits (its text response + task ID)Reward— what the grader returns (float score + human-readable feedback string)
environment.py — The central engine. Contains:
INSTANCE_BANK— the list of 12 scheduling instances with ground-truth answersSchedulingOptEnvclass withreset(),step(), andstate()methods- Task-aware instance routing (Tasks 2 and 3 only see infeasible instances)
- Round-robin instance cycling to ensure all instances are covered during training
- Per-step contextual hints embedded in the
contextfield of each Observation
server.py — FastAPI web application. Creates one shared SchedulingOptEnv instance and exposes it over 7 HTTP endpoints. Handles input validation and error responses.
graders/ — Three independent grader classes, each implementing a grade(action, ground_truth) → float method. Graders are stateless between calls but store the last grading breakdown in last_breakdown for the environment to surface in the info dict.
tasks/ — Thin wrappers that expose episode-running logic and instance pools for each task. These are used both by the environment and directly in the inference script.
inference.py — Standalone evaluation script. Reads API_BASE_URL, MODEL_NAME, and HF_TOKEN from environment variables. Creates an OpenAI client pointed at the configured endpoint. Runs all 32 episodes (12 + 10 + 10) and emits [START], [STEP], and [END] structured JSON logs to stdout.
5. Methodology
5.1 MDP Formulation
The environment is formalised as a Markov Decision Process with the following components:
| MDP Component | Definition |
|---|---|
State S |
Current scheduling instance + task type + step count + episode history |
Observation O |
{schedule_instance: str (JSON), task_id: str, context: str, step_number: int} |
Action A |
{response: str, task_id: str} |
Reward R |
Float ∈ [0.0, 1.0] from the task-specific grader |
Horizon T |
3 steps (Task 1) / 5 steps (Task 2) / 8 steps (Task 3) |
| Terminal | done = True when T is reached OR reward ≥ 0.95 |
The observation is always a JSON-encoded scheduling instance paired with a context string that instructs the agent on what to do. The action is always a text string — the agent's natural-language or structured-JSON response.
5.2 Episode Lifecycle
Every episode follows this exact sequence:
1. reset(task_id) is called
├── Validate task_id ∈ {feasibility_check, conflict_classification, schedule_repair}
├── Select instance pool for this task
├── Pick next instance via round-robin counter
├── Set step = 0, done = False, cumulative_reward = 0.0
├── Build initial context string (task instructions)
└── Return Observation
2. Agent reads the Observation and formulates a response
3. step(action) is called
├── Increment step counter
├── Select correct grader based on task_id
├── Call grader.grade(action, instance_ground_truth)
├── Clamp reward to [0.0, 1.0]
├── Accumulate cumulative_reward
├── Log (step, action, reward) to history
├── Check termination: done = (step ≥ max_steps OR reward ≥ 0.95)
└── Return (Observation, reward, done, info)
4. If done = False → repeat from step 2
If done = True → episode ends; call reset() to start the next episode
5.3 Task-Aware Instance Routing
Not all 12 instances are appropriate for every task:
- Task 1 (Feasibility Check) — Uses all 12 instances (10 infeasible + 2 feasible), because the agent must learn to recognise both valid and invalid schedules
- Task 2 (Conflict Classification) — Uses only the 10 infeasible instances, because the task presupposes the schedule is broken
- Task 3 (Schedule Repair) — Uses only the 10 infeasible instances with known optimal repairs
5.4 Progressive Difficulty Design
The three tasks form a deliberate cognitive ladder:
Task 1 — Feasibility Check (EASY)
↓ Binary yes/no decision
↓ No structural reasoning required beyond spotting a single violation
↓ Target accuracy: ~90%
Task 2 — Conflict Classification (MEDIUM)
↓ Must identify WHICH of 5 violation categories is present
↓ Requires understanding of all four constraint types
↓ Target accuracy: ~60%
Task 3 — Schedule Repair (HARD)
↓ Must produce a syntactically valid, structurally correct, constraint-satisfying,
near-optimal repaired schedule as JSON
↓ Requires full combinatorial reasoning
↓ Target accuracy: ~30%
This design ensures that agents can make measurable progress through the curriculum, and that the hardest task is genuinely challenging for frontier models.
6. The Scheduling Instance Corpus
The environment ships with 12 hand-crafted scheduling instances. Each instance is a Python dictionary with the following structure:
{
"instance": { # exposed to the agent
"problem_id": "P01",
"jobs": [...],
"machines": [...],
"proposed_schedule": {"assignments": [...]}
},
"is_feasible": False, # ground truth for Task 1
"violation_type": "resource_overload", # ground truth for Task 2
"optimal_schedule": {"assignments": [...]}, # ground truth for Task 3
"optimal_makespan": 7, # used for optimality scoring in Task 3
"description": "..." # human-readable summary
}
6.1 Full Instance Catalogue
| # | Problem ID | Feasible | Violation Class | Key Constraint Broken |
|---|---|---|---|---|
| 0 | P01 | No | resource_overload |
J1[0,4) and J2[2,5) overlap on M1 (capacity=1) |
| 1 | P02 | No | deadline_violation |
J1 starts at t=5, finishes at t=10 > deadline=8 |
| 2 | P03 | No | precedence_violation |
J2 starts at t=0 but depends on J1 which ends at t=8 |
| 3 | P04 | No | availability_conflict |
J1 starts at t=5, before M1's window opens at t=8 |
| 4 | P05 | No | capacity_exceeded |
3 concurrent jobs on M1 with capacity=2 |
| 5 | P06 | No | resource_overload |
J1 and J2 overlap on capacity-1 M1 (second instance) |
| 6 | P07 | No | deadline_violation |
Precedence chain forces J3 past its hard deadline |
| 7 | P08 | No | precedence_violation |
J3 starts before both J1 and J2 complete |
| 8 | P09 | No | availability_conflict |
J1 extends into M1's maintenance window |
| 9 | P10 | No | capacity_exceeded |
4 concurrent jobs on M1 with capacity=3 |
| 10 | P11 | Yes | — | Fully feasible 3-job, 2-machine schedule |
| 11 | P12 | Yes | — | Fully feasible 5-job, 3-machine schedule with precedence |
6.2 Violation Class Distribution
| Class | Count | Description |
|---|---|---|
resource_overload |
2 | Two jobs time-overlap on a single-capacity machine |
deadline_violation |
2 | A job finishes after its required deadline |
precedence_violation |
2 | A job starts before its predecessor completes |
availability_conflict |
2 | A job is scheduled outside a machine's availability window |
capacity_exceeded |
2 | Concurrent jobs exceed a machine's capacity limit |
| Feasible (no violation) | 2 | Used only in Task 1 |
Each violation class appears exactly twice, ensuring balanced representation and preventing class imbalance during training.
7. Task Definitions and Grading Functions
7.1 Task 1 — Feasibility Check (Easy)
Objective: Determine whether a proposed schedule satisfies all four constraint categories.
Action space: {"feasible", "infeasible"}
Episode horizon: 3 steps
Grader: FeasibilityGrader
The grader normalises common synonyms before scoring, so agents that reply with natural language equivalents still receive full credit:
| Synonyms treated as "feasible" | Synonyms treated as "infeasible" |
|---|---|
| feasible, valid, correct, satisfiable, yes, ok, pass | infeasible, invalid, incorrect, unsatisfiable, no, violated, conflict, fail, impossible, broken |
Scoring function:
R(action, ground_truth) =
1.0 if normalise(action.response) == ground_truth.is_feasible
0.1 if response is non-empty but cannot be parsed OR is wrong
0.0 if response is empty
The 0.1 score for a wrong-but-present answer is deliberate. It ensures the agent always has a gradient signal — even a completely wrong answer tells the training system "the agent tried; adjust its policy."
7.2 Task 2 — Conflict Classification (Medium)
Objective: Identify the constraint violation type present in an infeasible schedule from the closed vocabulary of five classes.
Action space: {resource_overload, deadline_violation, precedence_violation, availability_conflict, capacity_exceeded}
Episode horizon: 5 steps
Grader: ConflictGrader
The grader is aware of semantic "constraint families" — groups of violation types that are conceptually related. This enables partial credit for answers that are wrong but not completely off-base.
Constraint families:
- Resource-limit family:
resource_overloadandcapacity_exceeded— both concern concurrent job count on a machine - Temporal-ordering family:
deadline_violationandprecedence_violation— both concern job timing and sequencing - Standalone:
availability_conflict— concerns machine operational windows
Scoring function:
R(action, ground_truth) =
1.0 if action.response == ground_truth.violation_type (exact match)
0.5 if action.response is in the same family as ground_truth (partial credit)
0.1 if action.response is a valid category but wrong family (attempted)
0.0 if action.response is not a valid category (unparseable)
The grader also normalises spacing and dashes to underscores, so "deadline violation" and "deadline-violation" both map to "deadline_violation" before scoring.
7.3 Task 3 — Schedule Repair (Hard)
Objective: Return a corrected schedule as a JSON object that resolves all constraint violations and minimises total makespan.
Required output format:
{
"assignments": [
{"job_id": "J1", "machine_id": "M1", "start_time": 0},
{"job_id": "J2", "machine_id": "M1", "start_time": 4},
{"job_id": "J3", "machine_id": "M2", "start_time": 0}
]
}
Episode horizon: 8 steps
Grader: RepairGrader
The grader evaluates four independent components and sums them additively to a maximum of 1.0:
Component 1 — JSON Parseability (20% of score)
0.20 if the response parses as a valid JSON object
0.00 if the response is not valid JSON (no partial credit at this level)
The parser uses three strategies in sequence:
- Direct
json.loads()— handles pure JSON responses - Strip markdown code fences (
```) then parse — handles LLM wrapping - Brace-counting to extract the outermost
{...}block — handles prose-wrapped JSON
Component 2 — Schema Validity (20% of score)
0.20 if the JSON contains "assignments" list, every assignment has
{job_id, machine_id, start_time}, start_time ≥ 0, and every job
from the instance appears exactly once
0.00 otherwise
Component 3 — Constraint Satisfaction (40% of score)
Checks all four constraint categories independently. Each is worth 10% of the total score (0.10 points):
capacity_ok — no machine has more concurrent jobs than its capacity
deadlines_ok — every job finishes at or before its deadline
precedence_ok — every job starts after all its predecessors finish
availability_ok — every job runs within its machine's operational window
score += 0.40 × (number of satisfied categories / 4)
Component 4 — Makespan Optimality (20% of score)
0.20 if makespan(response) ≤ optimal_makespan × 1.30 (within 30% of optimal)
0.10 if makespan(response) ≤ optimal_makespan × 1.60 (within 60% of optimal)
0.00 if makespan(response) > optimal_makespan × 1.60 (too slow)
Where makespan is defined as the maximum finish time across all jobs: max(start_time(j) + duration(j)) ∀j.
Full scoring formula:
R = 0.20 × parseable_json(response)
+ 0.20 × valid_schema(response)
+ 0.40 × (satisfied_constraints / 4)
+ 0.20 × optimality_score(makespan, optimal_makespan)
8. Reward Design Philosophy
8.1 Dense vs Sparse Rewards
A key design principle in this environment is the use of dense rewards — reward signals that give partial credit throughout the episode, not just at the end.
Sparse reward (bad for learning):
score = 1.0 if the answer is perfect
score = 0.0 otherwise
This gives the agent almost no information. If the answer is wrong, the agent doesn't know if it was slightly wrong or completely wrong.
Dense reward (what this environment uses):
score = sum of partial credits for each correct sub-component
This tells the agent exactly which parts of its answer were right and which were wrong, enabling it to make targeted improvements in subsequent steps.
8.2 Why Partial Credit Matters
Consider a Schedule Repair attempt that produces syntactically valid JSON, covers all jobs, and fixes 3 out of 4 constraints, but is 50% slower than optimal:
score = 0.20 (JSON valid)
+ 0.20 (schema valid)
+ 0.30 (3/4 constraints: 0.40 × 0.75)
+ 0.10 (makespan within 60% of optimal)
= 0.80
The agent knows it is close. It knows it missed one constraint and its solution is a bit slow. It can target those specific issues in the next step.
8.3 The 0.1 Floor for Task 1 and Task 2
Even an incorrect answer receives a non-zero score (0.1) as long as the response is non-empty and parseable. This is intentional: it prevents the agent from learning to produce empty responses as a strategy to avoid penalties, and it keeps the gradient non-zero for wrong-but-present answers.
9. Data Models
All inter-component communication uses three Pydantic v2 schemas defined in models.py:
Observation
class Observation(BaseModel):
schedule_instance: str # JSON-encoded scheduling problem
task_id: str # "feasibility_check" | "conflict_classification" | "schedule_repair"
context: str # Natural-language instructions for the current step
step_number: int # Current step (0-indexed, ge=0)
Action
class Action(BaseModel):
response: str # Agent's text answer (e.g., "infeasible", "resource_overload", or JSON)
task_id: str # Must match the current episode's task_id
Reward
class Reward(BaseModel):
score: float # Grading result, enforced ∈ [0.0, 1.0]
feedback: str # Human-readable explanation of the score
10. API Specification
The environment is exposed as a RESTful HTTP API via FastAPI, running on port 7860 (the Hugging Face Spaces default).
Endpoints
| Method | Path | Description | Request Body | Response |
|---|---|---|---|---|
GET |
/health |
Liveness probe | None | {"status": "ok"} |
POST |
/reset |
Start new episode | {"task_id": "..."} |
Observation |
POST |
/step |
Submit action | {"response": "...", "task_id": "..."} |
StepResponse |
GET |
/state |
Internal state snapshot | None | Full state dict |
GET |
/tasks |
Task catalogue | None | Array of task descriptions |
POST |
/grader |
Direct grader invocation | {action, ground_truth} |
{"score": float} |
GET |
/baseline |
Run GPT-4o-mini baseline | None | Per-task scores |
StepResponse Schema
{
"observation": { "schedule_instance": "...", "task_id": "...", "context": "...", "step_number": 1 },
"reward": 1.0,
"done": true,
"info": {
"step_reward": 1.0,
"cumulative_reward": 1.0,
"steps_remaining": 2,
"instance_description": "...",
"grading_breakdown": { ... }
}
}
The info.grading_breakdown dict exposes the full internal grading decision — predicted vs expected, per-constraint pass/fail flags, makespan ratio — enabling training loops to inspect the decision without parsing the float reward.
11. Inference and Baseline
11.1 inference.py
The primary evaluation script. Uses three environment variables:
| Variable | Purpose | Example |
|---|---|---|
API_BASE_URL |
Base URL for the OpenAI-compatible API | https://api.openai.com/v1 |
MODEL_NAME |
Model identifier | gpt-4o-mini |
HF_TOKEN |
API key or Hugging Face token | hf_... or sk-... |
Log format:
Every episode emits three structured JSON log lines to stdout:
[START] {"task_id": "feasibility_check", "instance_id": 0}
[STEP] {"task_id": "feasibility_check", "instance_id": 0, "step": 1, "action": "infeasible", "reward": 1.0, "done": true, "feedback": "Correct."}
[END] {"task_id": "feasibility_check", "instance_id": 0, "final_reward": 1.0}
Final summary line:
{"event": "eval_end", "summary": {"feasibility_check": {"average_score": 1.0, "num_instances": 12}, "conflict_classification": {"average_score": 1.0, "num_instances": 10}, "schedule_repair": {"average_score": 1.0, "num_instances": 10}, "overall_average": 1.0}}
Oracle fallback: When HF_TOKEN is not set, the script falls back to deterministic mock responses (the ground-truth answers) and scores 1.0 on all tasks. This enables offline verification of the grading pipeline without any API access.
11.2 baseline.py
An alternative evaluation script (baseline.py) that calls the graders directly (without HTTP) and supports the same GPT-4o-mini / oracle-mock pattern. Useful for rapid local testing.
11.3 Baseline Scores
| Mode | Task 1 | Task 2 | Task 3 | Overall |
|---|---|---|---|---|
| Oracle mock (no API key) | 1.000 | 1.000 | 1.000 | 1.000 |
| GPT-4o-mini (with API key) | ~0.90 | ~0.60 | ~0.30 | ~0.60 |
The mock oracle achieves perfect scores by design — it is used to verify the grading pipeline, not to claim AI performance. The GPT-4o-mini scores reflect realistic expectations based on the designed difficulty levels.
12. Setup and Installation
12.1 Prerequisites
| Requirement | Version |
|---|---|
| Python | ≥ 3.11 |
| pip | ≥ 22.0 |
| Docker Desktop (for container testing) | ≥ 20.10 |
| Git | ≥ 2.30 |
12.2 Local Installation
# Step 1 — Clone the repository
git clone https://github.com/Vittal-Mukunda/OpenEnv-Hackathon-Meta-x-Scaler.git
cd OpenEnv-Hackathon-Meta-x-Scaler
# Step 2 — Create a virtual environment (strongly recommended)
python -m venv .venv
# Activate it:
# On macOS / Linux:
source .venv/bin/activate
# On Windows (PowerShell):
.venv\Scripts\Activate.ps1
# On Windows (CMD):
.venv\Scripts\activate.bat
# Step 3 — Install dependencies
pip install -r requirements.txt
# Step 4 — Start the server
uvicorn server:app --host 0.0.0.0 --port 7860
# Step 5 — Verify the server is alive (new terminal)
curl http://localhost:7860/health
# Expected: {"status":"ok"}
12.3 Docker Deployment
# Step 1 — Open Docker Desktop and wait for it to fully start
# Step 2 — Build the image (takes 1-2 minutes first time)
docker build -t scheduling-env .
# Step 3 — Run the container
docker run -p 7860:7860 scheduling-env
# Step 4 — Verify in a new terminal
curl http://localhost:7860/health
# Expected: {"status":"ok"}
# Step 5 — Stop the container
# Press Ctrl+C in the terminal running Docker
12.4 Hugging Face Spaces Deployment
- Create a new Hugging Face Space at huggingface.co/new-space
- Select Docker as the SDK
- Push this repository to the Space:
git remote add space https://huggingface.co/spaces/<your-username>/<space-name> git push space master - The Space will auto-detect the Dockerfile, build, and deploy. Port 7860 is exposed automatically.
13. How to Get Results
13.1 Quick Test — Oracle Mock (No API Key Needed)
This runs the full evaluation pipeline with deterministic oracle answers. Use this to verify everything is working correctly.
python inference.py
Expected output:
{"event": "eval_start", "mode": "oracle mock", "model": "gpt-4o-mini"}
[START] {"task_id": "feasibility_check", "instance_id": 0}
[STEP] {"task_id": "feasibility_check", "instance_id": 0, "step": 1, "action": "infeasible", "reward": 1.0, "done": true, "feedback": "Correct."}
[END] {"task_id": "feasibility_check", "instance_id": 0, "final_reward": 1.0}
... (32 episodes total) ...
{"event": "eval_end", "summary": {"feasibility_check": {"average_score": 1.0, "num_instances": 12}, "conflict_classification": {"average_score": 1.0, "num_instances": 10}, "schedule_repair": {"average_score": 1.0, "num_instances": 10}, "overall_average": 1.0}}
What the output means:
- Each
[START]line: a new episode begins on a specific instance - Each
[STEP]line: the agent submitted an answer; shows what it said and what score it received - Each
[END]line: the episode is over; shows the final reward - The last line: aggregated scores per task and overall
13.2 Real LLM Evaluation (With API Key)
# Using OpenAI
export API_BASE_URL=https://api.openai.com/v1
export MODEL_NAME=gpt-4o-mini
export HF_TOKEN=sk-your-openai-key-here
python inference.py
# Using Hugging Face Inference API
export API_BASE_URL=https://api-inference.huggingface.co/v1
export MODEL_NAME=meta-llama/Meta-Llama-3-8B-Instruct
export HF_TOKEN=hf_your-token-here
python inference.py
13.3 Testing Individual Endpoints
With the server running (uvicorn server:app --port 7860):
# Health check
curl http://localhost:7860/health
# Start a feasibility episode
curl -X POST http://localhost:7860/reset \
-H "Content-Type: application/json" \
-d '{"task_id": "feasibility_check"}'
# Submit an answer
curl -X POST http://localhost:7860/step \
-H "Content-Type: application/json" \
-d '{"response": "infeasible", "task_id": "feasibility_check"}'
# Start a classification episode
curl -X POST http://localhost:7860/reset \
-H "Content-Type: application/json" \
-d '{"task_id": "conflict_classification"}'
# Submit classification
curl -X POST http://localhost:7860/step \
-H "Content-Type: application/json" \
-d '{"response": "resource_overload", "task_id": "conflict_classification"}'
# Start a repair episode
curl -X POST http://localhost:7860/reset \
-H "Content-Type: application/json" \
-d '{"task_id": "schedule_repair"}'
# Submit a repaired schedule
curl -X POST http://localhost:7860/step \
-H "Content-Type: application/json" \
-d '{"response": "{\"assignments\": [{\"job_id\": \"J1\", \"machine_id\": \"M1\", \"start_time\": 0}, {\"job_id\": \"J2\", \"machine_id\": \"M1\", \"start_time\": 4}]}", "task_id": "schedule_repair"}'
# Check full internal state
curl http://localhost:7860/state
# Grade an action directly (without going through an episode)
curl -X POST http://localhost:7860/grader \
-H "Content-Type: application/json" \
-d '{"action": {"response": "deadline_violation", "task_id": "conflict_classification"}, "ground_truth": {"violation_type": "deadline_violation"}}'
13.4 Interpreting the Score
| Score Range | Meaning |
|---|---|
| 1.0 | Perfect answer |
| 0.8 – 0.99 | Excellent — nearly correct, minor issue |
| 0.5 – 0.79 | Good — correct constraint family or partial repair |
| 0.2 – 0.49 | Partial — JSON parsed, some constraints fixed |
| 0.1 – 0.19 | Attempted — something was submitted, but mostly wrong |
| 0.0 | Empty response or completely unparseable |
14. End-to-End Walkthrough
Full Example: Task 3 Schedule Repair
This walkthrough shows exactly what happens when an agent attempts to repair instance P01 (resource_overload: J1[0,4) and J2[2,5) overlap on M1 capacity=1).
Step 1 — Reset
POST /reset
{"task_id": "schedule_repair"}
The environment:
- Validates task_id = "schedule_repair"
- Selects the infeasible instance pool (10 instances)
- Picks instance 0 (P01) via round-robin
- Sets step=0, done=False
- Returns Observation:
{
"schedule_instance": "{\"problem_id\": \"P01\", \"jobs\": [{\"id\": \"J1\", \"duration\": 4, ...}, ...], \"proposed_schedule\": {\"assignments\": [{\"job_id\": \"J1\", \"start_time\": 0}, {\"job_id\": \"J2\", \"start_time\": 2}]}}",
"task_id": "schedule_repair",
"context": "The proposed_schedule is infeasible. Return ONLY a JSON object with key \"assignments\"...",
"step_number": 0
}
Step 2 — Agent submits broken JSON
POST /step
{"response": "Here is my repair: {bad json", "task_id": "schedule_repair"}
RepairGrader tries all 3 parse strategies. All fail. Returns:
json_parseable: false→ score = 0.0done: false(step 1/8)
Step 3 — Agent submits valid JSON, wrong schema
POST /step
{"response": "{\"jobs\": [...]}", "task_id": "schedule_repair"}
RepairGrader: JSON parses (0.20), but no "assignments" key (schema fails). Returns:
- score = 0.20
done: false(step 2/8)
Step 4 — Agent submits correct schema, partial constraints
{"assignments": [{"job_id": "J1", "machine_id": "M1", "start_time": 0}, {"job_id": "J2", "machine_id": "M1", "start_time": 2}, {"job_id": "J3", "machine_id": "M2", "start_time": 0}]}
Grader checks:
- JSON: ✅ (0.20)
- Schema: ✅ all jobs present (0.20)
- Capacity: ❌ J1[0,4) and J2[2,5) still overlap on M1
- Deadlines: ✅
- Precedence: ✅
- Availability: ✅
- Constraints: 3/4 = 0.30
- Makespan: 7 vs optimal 7 → ratio 1.0 → 0.20
Score = 0.20 + 0.20 + 0.30 + 0.20 = 0.90
Step 5 — Agent submits optimal repair
{"assignments": [{"job_id": "J1", "machine_id": "M1", "start_time": 0}, {"job_id": "J2", "machine_id": "M1", "start_time": 4}, {"job_id": "J3", "machine_id": "M2", "start_time": 0}]}
Grader checks:
- JSON: ✅ (0.20)
- Schema: ✅ (0.20)
- Capacity: ✅ J1[0,4), J2[4,7) no overlap
- Deadlines: ✅
- Precedence: ✅
- Availability: ✅
- Constraints: 4/4 = 0.40
- Makespan: 7 vs optimal 7 → ratio 1.0 → 0.20
Score = 1.00. done: true. Episode ends.
15. Evaluation and Scoring
15.1 Pre-Submission Checklist
| Check | Pass Condition |
|---|---|
| HF Space deploys | GET /health returns 200 and {"status":"ok"} |
| OpenEnv spec compliance | openenv.yaml valid; typed models; step()/reset()/state() respond correctly |
| Dockerfile builds | docker build completes without errors |
| Baseline reproduces | python inference.py completes without error and produces scores |
| 3+ tasks with graders | All three tasks return scores ∈ [0.0, 1.0] |
15.2 Hackathon Scoring Breakdown
| Criterion | Weight | This Project |
|---|---|---|
| Real-world utility | 30% | Industrial scheduling (manufacturing, cloud, healthcare) |
| Task & grader quality | 25% | 3 tasks, difficulty range easy→hard, deterministic graders |
| Environment design | 20% | Dense rewards, clean state, sensible episode boundaries |
| Code quality & spec compliance | 15% | Typed models, documented, Docker works, spec compliant |
| Creativity & novelty | 10% | Scheduling is rare in OpenEnv; multi-component repair grader is novel |
16. Project Structure
.
├── inference.py # Primary inference script (API_BASE_URL/MODEL_NAME/HF_TOKEN)
├── baseline.py # GPT-4o-mini baseline with oracle fallback
├── server.py # FastAPI HTTP server (7 endpoints, port 7860)
├── environment.py # SchedulingOptEnv core + INSTANCE_BANK (12 problems)
├── models.py # Pydantic v2 data models (Observation, Action, Reward)
├── openenv.yaml # OpenEnv metadata manifest
├── Dockerfile # Container definition (python:3.11-slim, port 7860)
├── requirements.txt # Python dependencies
├── tasks/
│ ├── __init__.py # Task module exports
│ ├── task1_easy.py # Feasibility check — episode runner + instance accessor
│ ├── task2_medium.py # Conflict classification — episode runner + instance accessor
│ └── task3_hard.py # Schedule repair — episode runner + instance accessor
└── graders/
├── __init__.py # Grader exports
├── grader_detection.py # FeasibilityGrader (binary, synonym-aware)
├── grader_classification.py # ConflictGrader (family-aware partial credit)
└── grader_fix.py # RepairGrader (4-component additive reward)
17. Dependencies
| Package | Version | Purpose |
|---|---|---|
fastapi |
≥ 0.104 | HTTP server framework |
uvicorn |
≥ 0.24 | ASGI server (runs FastAPI) |
pydantic |
≥ 2.5 | Data validation and serialisation |
openai |
≥ 1.6 | LLM inference (used in inference.py and baseline.py) |
pyyaml |
≥ 6.0 | YAML manifest parsing (openenv.yaml) |
httpx |
≥ 0.25 | Async HTTP client (used internally by FastAPI) |
All dependencies are intentionally lightweight and ship in python:3.11-slim with no additional system packages required.
18. Glossary
| Term | Definition |
|---|---|
| MDP | Markov Decision Process — a formal framework for sequential decision problems |
| Episode | One complete round from reset() to terminal state |
| Horizon | Maximum number of steps allowed per episode |
| Terminal | State where done = True — episode must end |
| Dense reward | A reward signal that provides partial credit at every step, not just at the end |
| Sparse reward | A reward signal that is non-zero only at the final step (bad for learning) |
| Feasible | A schedule that satisfies all four constraint categories |
| Infeasible | A schedule that violates at least one constraint |
| Makespan | The total time from the start of the first job to the finish of the last job |
| Optimal makespan | The minimum achievable makespan for a given set of jobs and machines |
| Instance | One specific scheduling problem (jobs + machines + proposed schedule + ground truth) |
| Grader | A Python class that scores an agent's action against ground truth |
| OpenEnv | The framework specification by Meta/HuggingFace that this environment implements |
| Constraint family | A group of semantically related violation types used for partial-credit scoring |
| Oracle | A mock agent that always produces the ground-truth answer — used to verify the grading pipeline |
Built for the Meta × Scaler OpenEnv Hackathon · April 2026 · MIT License