GRADER_CONFIGURATION.md

Grader Configuration Guide

Overview

This environment includes 5 task-specific graders for the Energy & Memory RAM Optimization tasks.

Available Graders

All graders are:

• ✅ Properly exported from task_graders module
• ✅ Callable functions that take EnergyOptimizationObservation and return float score (0.0-1.0)
• ✅ Registered in TASK_GRADERS dictionary
• ✅ Configured in openenv.yaml

Grader List

Task	Function	Difficulty	Min/Max Steps	Target RAM	Target Energy
Basic RAM Reduction	task_1_basic_ram_reduction_grader	1	10	< 70%	< 7.5 kWh
Energy Optimization	task_2_energy_optimization_grader	2	15	< 75%	< 6.0 kWh
Balanced Optimization	task_3_balanced_optimization_grader	3	20	< 60%	< 5.0 kWh
Advanced Efficiency	task_4_advanced_efficiency_grader	4	25	< 50%	< 4.0 kWh
Expert Optimization	task_5_expert_optimization_grader	5	30	< 40%	< 3.0 kWh

How to Access Graders

Method 1: Direct Import

from task_graders import (
    task_1_basic_ram_reduction_grader,
    task_2_energy_optimization_grader,
    task_3_balanced_optimization_grader,
    task_4_advanced_efficiency_grader,
    task_5_expert_optimization_grader,
)

# Use grader
obs = ...
score = task_1_basic_ram_reduction_grader(obs)

Method 2: Via Registry

from task_graders import get_grader, TASK_GRADERS

# Get specific grader
grader = get_grader("basic_ram_reduction")
score = grader(observation)

# Get all graders
all_graders = TASK_GRADERS  # Dict with grader functions

Method 3: Via Task Registry

from task_registry import get_task_grader, get_tasks_count

print(f"Total tasks: {get_tasks_count()}")  # 5
grader = get_task_grader("basic_ram_reduction")

Method 4: String-based Import (OpenEnV Format)

# Pattern used in openenv.yaml: module:function
module_name, func_name = "task_graders", "task_1_basic_ram_reduction_grader"
module = __import__(module_name)
grader = getattr(module, func_name)
score = grader(observation)

Method 5: Via Server Endpoints (FastAPI)

# Get all graders metadata
GET http://localhost:8000/graders

# Get specific grader info
GET http://localhost:8000/graders/{task_name}

# Get grader manifest
GET http://localhost:8000/graders/manifest

# Get grader discovery info
GET http://localhost:8000/graders/discovery

Validation

All graders are validated to:

• ✅ Return float scores in valid range (0.001 - 0.999)
• ✅ Accept EnergyOptimizationObservation objects
• ✅ Implement scoring logic based on RAM/energy targets
• ✅ Be callable and importable
• ✅ Meet hackathon requirement (≥3 graders required, we have 5)

openenv.yaml Configuration

Tasks are configured with grader paths as:

tasks:
  - name: basic_ram_reduction
    grader: task_graders:task_1_basic_ram_reduction_grader
    ...

The format module:function_name allows OpenEnV validator to:

1. Parse the reference
2. Import task_graders module
3. Get the function by name using getattr()
4. Verify it's callable
5. Register it as the grader for that task

Verification Commands

# Test grader imports
python -c "from task_graders import *; print('✅ All imports OK')"

# Test grader callable
python -c "import task_graders; print(callable(task_graders.task_1_basic_ram_reduction_grader))"

# Test grader execution
python -c "
from task_graders import task_1_basic_ram_reduction_grader
from models import EnergyOptimizationObservation
obs = EnergyOptimizationObservation(ram_usage=80, energy_consumption=8, current_task='test', step=1, done=False)
score = task_1_basic_ram_reduction_grader(obs)
print(f'Score: {score}')
"

# Run full validation
python validate.py

Requirements Met ✅

• Minimum 3 graders: ✅ 5 graders
• Grader score range: ✅ 0.001-0.999
• OpenEnV compatibility: ✅ Verified
• Multi-mode deployment: ✅ Ready