docs/fix.md

Exact error meaning

The validator is failing at:

Task Validation Not enough tasks with graders

This means the system could not detect at least 3 registered tasks that each have a grader.

The hackathon explicitly requires:

minimum 3 tasks with agent graders easy → medium → hard score in [0,1]

So your environment is likely exposing either:

• only 1 task
• 3 tasks but not registered
• tasks exist but grader missing / disabled
• openenv.yaml only lists one task

This is now not an app error — it’s a spec / task registration issue.

---

Most likely reason #1 (90% probability)

Your openenv.yaml probably has only one task.

Example of the problem:

tasks:
  - easy

This fails.

You need 3 tasks minimum.

---

Exact fix

Your openenv.yaml should look like this:

name: trace
version: 1.0.0
description: AI-native incident response runtime

entrypoint: trace.env:TraceEnv

tasks:
  - name: easy_incident
    difficulty: easy
    grader: trace.graders:grade_easy

  - name: medium_cascade
    difficulty: medium
    grader: trace.graders:grade_medium

  - name: hard_adversarial
    difficulty: hard
    grader: trace.graders:grade_hard

This is the most important fix.

---

Most likely reason #2

You may have 3 scenarios in code, but only one is exposed to OpenEnv.

Example:

TASKS = {
    "easy": ...,
    "medium": ...,
    "hard": ...
}

This alone is not enough.

They must be enumerable by the validator.

Usually via openenv.yaml or task registry.

---

Most likely reason #3

Graders missing.

The validator explicitly says:

tasks with graders

That means each task needs a scoring function.

Example:

def grade_easy(history):
    return min(max(score, 0.0), 1.0)

def grade_medium(history):
    return min(max(score, 0.0), 1.0)

def grade_hard(history):
    return min(max(score, 0.0), 1.0)

Three separate graders is safest.

---

What I strongly recommend

Make this exact structure.

---

tasks/

tasks/
├── easy_incident.yaml
├── medium_cascade.yaml
└── hard_adversarial.yaml

---

graders.py

def grade_easy(state):
    ...

def grade_medium(state):
    ...

def grade_hard(state):
    ...

---

openenv.yaml

Explicitly map them.

This is what the validator likely parses.

---

Quick emergency fix (best for resubmission)

Even if all 3 tasks use same environment class, expose them as separate tasks.

Example:

tasks:
  - name: easy
    grader: trace.graders:grade_easy
  - name: medium
    grader: trace.graders:grade_medium
  - name: hard
    grader: trace.graders:grade_hard

This should pass validation.

---

My blunt diagnosis

This is now very close to passing.

Your system is working.

The validator simply cannot “see” 3 tasks.

This is a metadata + grader registration issue, not a code failure.

Honestly, this is a good place to be.

If you want, paste your current openenv.yaml, and I’ll tell you exactly what needs to be changed to pass this check.