I766446 commited on
Commit
71b519d
·
1 Parent(s): 1d3b3a7

simplify tasks for validator

Browse files
Files changed (1) hide show
  1. openenv.yaml +0 -30
openenv.yaml CHANGED
@@ -142,43 +142,13 @@ reward:
142
  # ---------------------------------------------------------------------------
143
  tasks:
144
  - id: easy
145
- difficulty: easy
146
- description: >
147
- Clear-cut claims with overwhelming contradicting evidence from authoritative
148
- sources (WHO, CDC, NASA, official government). Decision should be
149
- straightforward for a competent agent.
150
- dataset: tasks/easy/scenarios.json
151
- n_scenarios: 5
152
  grader: tasks.easy.grader.grade
153
- weights: { decision: 0.50, evidence: 0.25, reasoning: 0.15, efficiency: 0.10 }
154
- ideal_max_steps: 8
155
- ground_truth_distribution: { allow: 1, flag: 1, remove: 3 }
156
 
157
  - id: medium
158
- difficulty: medium
159
- description: >
160
- Partially true or misleading content. Mixed evidence with some supporting
161
- and some contradicting sources. Requires careful claim-by-claim verification
162
- and evidence weighing.
163
- dataset: tasks/medium/scenarios.json
164
- n_scenarios: 5
165
  grader: tasks.medium.grader.grade
166
- weights: { decision: 0.35, evidence: 0.35, reasoning: 0.20, efficiency: 0.10 }
167
- ideal_max_steps: 12
168
- ground_truth_distribution: { allow: 1, flag: 4, remove: 0 }
169
 
170
  - id: hard
171
- difficulty: hard
172
- description: >
173
- Conflicting evidence from credible sources; genuinely contested ground truth.
174
- Requires nuanced multi-claim reasoning, evidence reliability weighting, and
175
- tolerance for ambiguity. All four scoring dimensions count equally.
176
- dataset: tasks/hard/scenarios.json
177
- n_scenarios: 5
178
  grader: tasks.hard.grader.grade
179
- weights: { decision: 0.30, evidence: 0.30, reasoning: 0.30, efficiency: 0.10 }
180
- ideal_max_steps: 18
181
- ground_truth_distribution: { allow: 0, flag: 5, remove: 0 }
182
 
183
  # ---------------------------------------------------------------------------
184
  # Grader
 
142
  # ---------------------------------------------------------------------------
143
  tasks:
144
  - id: easy
 
 
 
 
 
 
 
145
  grader: tasks.easy.grader.grade
 
 
 
146
 
147
  - id: medium
 
 
 
 
 
 
 
148
  grader: tasks.medium.grader.grade
 
 
 
149
 
150
  - id: hard
 
 
 
 
 
 
 
151
  grader: tasks.hard.grader.grade
 
 
 
152
 
153
  # ---------------------------------------------------------------------------
154
  # Grader