openhands openhands commited on
Commit
eff7f34
·
1 Parent(s): da96293

Fix: Treat zero scores as valid results, not missing categories

Browse files

A score of 0.0 is a legitimate evaluation result (e.g., qwen-3-coder's
commit0 score). Previously, entries with any 0.0 score were incorrectly
marked as 'incomplete' because the code treated 0.0 the same as missing.

This fix ensures that only truly missing values (NA/None) are considered
incomplete, while 0.0 scores are counted as attempted categories.

Co-authored-by: openhands <openhands@all-hands.dev>

Files changed (1) hide show
  1. leaderboard_transformer.py +2 -2
leaderboard_transformer.py CHANGED
@@ -345,8 +345,8 @@ class DataTransformer:
345
  count = 0
346
  for category in main_categories:
347
  value = row.get(f"{category} Score")
348
- # Handle pd.NA, None, and 0.0 properly
349
- if pd.notna(value) and value != 0.0:
350
  count += 1
351
  return f"{count}/5"
352
 
 
345
  count = 0
346
  for category in main_categories:
347
  value = row.get(f"{category} Score")
348
+ # A score of 0.0 is a valid result - only exclude truly missing values
349
+ if pd.notna(value):
350
  count += 1
351
  return f"{count}/5"
352