clean the code

CODE_VERIFICATION_REPORT.md

@@ -1,405 +0,0 @@
-# Code Verification Report - Real-Time Log Streaming
-
-**Date**: January 13, 2026
-**Status**: ✅ ALL CHECKS PASSED
-
-## Summary
-
-All code changes have been verified for correctness. The implementation is ready for deployment and should provide real-time log streaming with progress indicators.
-
----
-
-## File 1: app.py
-
-**Location**: `/root/code/MedVidBench-Leaderboard/app.py`
-
-### ✅ Syntax Check
-```bash
-python -m py_compile app.py
-# Result: SUCCESS (no errors)
-```
-
-### ✅ Unbuffered Subprocess Configuration (Lines 768-784)
-
-**Command Construction**:
-```python
-cmd = [
-    sys.executable,
-    "-u",  # ✅ Unbuffered output flag present
-    str(eval_wrapper),
-    str(input_file),
-    "--grouping", "overall",
-    "--ground-truth", "data/ground_truth.json"
-]
-```
-
-**Process Configuration**:
-```python
-process = subprocess.Popen(
-    cmd,
-    stdout=subprocess.PIPE,        # ✅ Pipe stdout for reading
-    stderr=subprocess.STDOUT,       # ✅ Merge stderr into stdout
-    text=True,                      # ✅ Text mode (not bytes)
-    bufsize=1,                      # ✅ Line-buffered
-    env={**os.environ, "PYTHONUNBUFFERED": "1"}  # ✅ Force unbuffered
-)
-```
-
-**Verification**: ✅ Both `-u` flag AND `PYTHONUNBUFFERED=1` are present
-
-### ✅ Non-Blocking Read Implementation (Line 810)
-
-```python
-ready, _, _ = select.select([process.stdout], [], [], 0.5)
-```
-
-**Verification**: ✅ Using `select.select()` with 0.5s timeout for non-blocking reads
-
-### ✅ Progress Bar Implementation (Lines 847-850)
-
-```python
-# Increment progress gradually from 25% to 75%
-progress_increment = min(0.75, 0.25 + (line_count / 500) * 0.50)
-progress(progress_increment, desc="Running evaluation...")
-```
-
-**Verification**: ✅ Progressive increment from 25% → 75% based on log lines
-
-### ✅ Heartbeat Messages (Lines 832-836)
-
-```python
-if not log_buffer:
-    elapsed = int(time.time() - start_time)
-    log_text = f"⚙️ **Step 3/6**: Running evaluation...\n\n```\nWaiting for evaluation output... ({elapsed}s elapsed)\n```"
-    yield log_text
-```
-
-**Verification**: ✅ Shows elapsed time when no logs appear
-
-### ✅ Generator Function (Line 720)
-
-```python
-def submit_model(file, model_name: str, organization: str, contact: str = "", progress=gr.Progress()):
-    """
-    Process model submission: validate, evaluate, and add to leaderboard.
-    Yields progress updates during evaluation.
-    """
-```
-
-**Verification**: ✅ Function uses `yield` for streaming updates
-
----
-
-## File 2: evaluation/evaluate_predictions.py
-
-**Location**: `/root/code/MedVidBench-Leaderboard/evaluation/evaluate_predictions.py`
-
-### ✅ Syntax Check
-```bash
-python -m py_compile evaluation/evaluate_predictions.py
-# Result: SUCCESS (no errors)
-```
-
-### ✅ Flush Statements (10 occurrences found)
-
-**Line 186**: Loading message
-```python
-print(f"[EvaluationWrapper] Loading predictions from {args.predictions_file}", flush=True)
-```
-
-**Lines 194-195**: Merged format detection
-```python
-print("[EvaluationWrapper] ✓ Detected: Predictions already contain ground-truth", flush=True)
-print("[EvaluationWrapper] Using predictions file directly for evaluation", flush=True)
-```
-
-**Lines 198-199**: Prediction-only format detection
-```python
-print("[EvaluationWrapper] ✓ Detected: Prediction-only format (id, qa_type, prediction)", flush=True)
-print("[EvaluationWrapper] Merging with ground-truth...", flush=True)
-```
-
-**Line 215**: Merge completion
-```python
-print(f"[EvaluationWrapper] ✓ Merged data saved to temporary file: {eval_file}", flush=True)
-```
-
-**Lines 218-220**: Handoff to evaluate_all_pai
-```python
-print(f"\n[EvaluationWrapper] {'='*80}", flush=True)
-print(f"[EvaluationWrapper] Starting evaluation with evaluate_all_pai.py", flush=True)
-print(f"[EvaluationWrapper] {'='*80}\n", flush=True)
-```
-
-**Verification**: ✅ All critical print statements have `flush=True`
-
----
-
-## File 3: evaluation/evaluate_all_pai.py
-
-**Location**: `/root/code/MedVidBench-Leaderboard/evaluation/evaluate_all_pai.py`
-
-### ✅ Syntax Check
-```bash
-python -m py_compile evaluation/evaluate_all_pai.py
-# Result: SUCCESS (no errors)
-```
-
-### ✅ Flush Statements (15 occurrences found)
-
-**Lines 58-64**: Dataset analysis output
-```python
-print(f"\nFound QA types:", flush=True)
-for qa_type, count in qa_type_counts.items():
-    print(f"  {qa_type}: {count} records", flush=True)
-
-print(f"\nFound datasets:", flush=True)
-for dataset, count in dataset_counts.items():
-    print(f"  {dataset}: {count} records", flush=True)
-```
-
-**Lines 770-771**: Task list and total count
-```python
-print(f"\nRunning evaluation for tasks: {tasks}", flush=True)
-print(f"Total tasks to evaluate: {len(tasks)}", flush=True)
-```
-
-**Line 786**: Task progress counter (⭐ KEY FEATURE)
-```python
-print(f"\n[Progress] Task {task_idx}/{len(tasks)}: {task.upper()}", flush=True)
-```
-
-**Lines 790-792**: Skip message for pre-computed LLM scores
-```python
-print(f"\n{'='*80}", flush=True)
-print(f"SKIPPING {task.upper()} EVALUATION (LLM judge pre-computed)", flush=True)
-print(f"{'='*80}", flush=True)
-```
-
-**Lines 798-800**: Task evaluation banner
-```python
-print(f"\n{'='*80}", flush=True)
-print(f"RUNNING {task.upper()} EVALUATION", flush=True)
-print(f"{'='*80}", flush=True)
-```
-
-**Line 803**: Silent mode progress
-```python
-print(f"Evaluating {task.upper()}...", flush=True)
-```
-
-**Line 820**: Task completion message (⭐ KEY FEATURE)
-```python
-print(f"[Progress] ✓ Completed {task.upper()} evaluation (Task {task_idx}/{len(tasks)})", flush=True)
-```
-
-**Verification**: ✅ All progress messages have `flush=True`
-
----
-
-## Key Features Verification
-
-### ✅ Feature 1: Format Auto-Detection
-**Location**: `evaluate_predictions.py` lines 191-216
-**Status**: ✅ Working correctly
-- Detects merged format → skips merging
-- Detects prediction-only → merges with ground-truth
-- Prints clear messages with `flush=True`
-
-### ✅ Feature 2: Real-Time Log Streaming
-**Location**: `app.py` lines 768-858
-**Status**: ✅ Fully implemented
-- Unbuffered subprocess (`-u` + `PYTHONUNBUFFERED=1`)
-- Non-blocking read with `select.select()`
-- 0.5s update frequency
-- Shows last 25 lines of logs
-
-### ✅ Feature 3: Heartbeat Feedback
-**Location**: `app.py` lines 832-836
-**Status**: ✅ Working
-- Shows "Waiting for output... (Xs elapsed)"
-- Updates every 0.5s even when no logs
-
-### ✅ Feature 4: Progressive Progress Bar
-**Location**: `app.py` lines 847-850
-**Status**: ✅ Working
-- Starts at 25% (beginning of evaluation)
-- Advances based on log lines
-- Caps at 75% (end of evaluation)
-
-### ✅ Feature 5: Task Progress Counters
-**Location**: `evaluate_all_pai.py` lines 770-820
-**Status**: ✅ Fully implemented
-- Shows "Total tasks to evaluate: 8"
-- Shows "[Progress] Task 1/8: TAL"
-- Shows "[Progress] ✓ Completed TAL (Task 1/8)"
-
----
-
-## Expected User Experience
-
-### Phase 1: Initialization (5% → 15%)
-```
-🔍 Step 1/6: Checking if model name is available...
-✓ Model name available
-
-📋 Step 2/6: Validating predictions file format...
-✓ Valid format detected
-```
-
-### Phase 2: Format Detection (15% → 25%)
-```
-⚙️ Step 3/6: Running evaluation (streaming logs)...
-
-[EvaluationWrapper] Loading predictions from input.json
-[EvaluationWrapper] ✓ Detected: Predictions already contain ground-truth
-[EvaluationWrapper] Using predictions file directly for evaluation
-```
-
-### Phase 3: Dataset Analysis (25% → 30%)
-```
-Found QA types:
-  tal: 1637 records
-  stg: 780 records
-  next_action: 1280 records
-  dvc: 3000 records
-  vs: 1500 records
-  rc: 2522 records
-  skill_assessment: 390 records
-  cvs_assessment: 390 records
-
-Found datasets:
-  jigsaws: 780 records
-  ...
-```
-
-### Phase 4: Task Evaluations (30% → 75%)
-```
-Running evaluation for tasks: ['tal', 'stg', 'next_action', 'dvc', 'vs', 'rc', 'skill_assessment', 'cvs_assessment']
-Total tasks to evaluate: 8
-
-[Progress] Task 1/8: TAL
-================================================================================
-RUNNING TAL EVALUATION
-================================================================================
-[Progress] ✓ Completed TAL evaluation (Task 1/8)          [Progress: 35%]
-
-[Progress] Task 2/8: STG
-================================================================================
-RUNNING STG EVALUATION
-================================================================================
-[Progress] ✓ Completed STG evaluation (Task 2/8)          [Progress: 40%]
-
-...
-
-[Progress] Task 8/8: CVS_ASSESSMENT
-[Progress] ✓ Completed CVS_ASSESSMENT evaluation (Task 8/8) [Progress: 75%]
-```
-
-### Phase 5: Validation (75% → 90%)
-```
-✓ Evaluation completed!
-🔍 Step 4/6: Validating extracted metrics...
-✓ All 10 metrics successfully computed
-
-📊 Step 5/6: Adding model to leaderboard...
-✓ Leaderboard updated!
-```
-
-### Phase 6: Complete (90% → 100%)
-```
-✅ Step 6/6: Submission complete!
-
----
-
-## ✅ Submission Successful!
-
-**Model**: MyModel
-**Organization**: MyOrg
-
-### 📈 Metric Scores
-- **CVS Assessment Accuracy**: 0.8234
-- **Skill Assessment Accuracy**: 0.7891
-...
-
-### 🏆 Ranking
-**Rank**: #3 out of 15 models
-```
-
----
-
-## Deployment Checklist
-
-### ✅ Code Changes
-- [x] app.py modified (unbuffered subprocess + non-blocking read)
-- [x] evaluate_predictions.py modified (flush=True added)
-- [x] evaluate_all_pai.py modified (task progress counters)
-
-### ✅ Syntax Validation
-- [x] app.py compiles without errors
-- [x] evaluate_predictions.py compiles without errors
-- [x] evaluate_all_pai.py compiles without errors
-
-### ✅ Feature Verification
-- [x] Unbuffered subprocess configuration
-- [x] Non-blocking read with select.select()
-- [x] Heartbeat messages
-- [x] Progressive progress bar
-- [x] Task progress counters
-- [x] All flush=True statements present
-
-### 📦 Ready for Deployment
-The code is **production-ready** and should work correctly when deployed to HF Spaces.
-
----
-
-## Troubleshooting (If Issues Persist on HF Spaces)
-
-If the progress messages still don't appear after deployment:
-
-1. **Verify Files on HF Spaces**:
-   - Go to Files tab on HF Space
-   - Check that `app.py`, `evaluation/evaluate_predictions.py`, and `evaluation/evaluate_all_pai.py` contain the new code
-   - Search for `[Progress]` and `flush=True` in the files
-
-2. **Check Build Logs**:
-   - Go to Settings → "View Logs"
-   - Verify the space rebuilt after your push
-   - Look for "Building..." and "Running..." messages
-
-3. **Force Rebuild**:
-   - Settings → Factory reboot
-   - Wait 2-3 minutes for rebuild
-
-4. **Test Locally First**:
-   - Run the test script: `python test_streaming.py`
-   - Verify logs stream in real-time locally
-   - If local works but HF doesn't, it's a deployment issue
-
-5. **Browser Cache**:
-   - Clear browser cache (Ctrl+Shift+Delete)
-   - Try incognito/private browsing mode
-   - Try different browser
-
----
-
-## Conclusion
-
-✅ **Code verification: PASSED**
-
-All three files have been verified:
-- ✅ No syntax errors
-- ✅ All critical features implemented
-- ✅ All flush=True statements present
-- ✅ Unbuffered subprocess configuration correct
-- ✅ Non-blocking I/O implemented correctly
-- ✅ Progress tracking fully functional
-
-**The code is ready for production deployment on HF Spaces.**
-
-If logs still don't appear on HF Spaces, the issue is likely:
-1. HF Spaces hasn't rebuilt with the new code yet
-2. Browser cache showing old version
-3. Network delays in SSE streaming (HF Spaces infrastructure)
-
-The code itself is **correct and production-ready**.

LEADERBOARD_FORMATS.md

@@ -1,228 +0,0 @@
-# MedVidBench Leaderboard Supported Formats
-
-## Overview
-
-The leaderboard web app now accepts **two submission formats**:
-
-1. **Prediction-only** (preferred for users)
-2. **Merged format** (for testing/debugging)
-
-Both formats are automatically detected and handled by the evaluation system.
-
-## Format 1: Prediction-Only (User Submission)
-
-**Recommended for**: Public user submissions
-
-**Structure**:
-```json
-[
-  {
-    "id": "video_id&&start_frame&&end_frame&&fps",
-    "qa_type": "tal",
-    "prediction": "0.0-10.0 seconds."
-  },
-  {
-    "id": "another_video&&0&&100&&1.0",
-    "qa_type": "video_summary",
-    "prediction": "The surgeon performs cholecystectomy..."
-  }
-]
-```
-
-**Required fields**:
-- `id`: Sample identifier (video_id&&start&&end&&fps)
-- `qa_type`: Task type
-- `prediction`: Model's answer text
-
-**What happens**:
-1. Server validates format
-2. Server merges with private ground-truth
-3. Runs evaluation
-4. Adds to leaderboard
-
-## Format 2: Merged (Internal/Testing)
-
-**Recommended for**: Internal testing, debugging
-
-**Structure**:
-```json
-{
-  "0": {
-    "metadata": {
-      "video_id": "kcOqlifSukA",
-      "fps": "1.0",
-      "input_video_start_frame": "22425",
-      "input_video_end_frame": "25124"
-    },
-    "qa_type": "tal",
-    "struc_info": [
-      {
-        "action": "cutting",
-        "spans": [{"start": 0.0, "end": 10.0}]
-      }
-    ],
-    "question": "When does cutting happen?",
-    "gnd": "0.0-10.0 seconds.",
-    "answer": "0.0-10.0 seconds.",
-    "data_source": "AVOS"
-  }
-}
-```
-
-**Required fields**:
-- `metadata`: Video metadata
-- `qa_type`: Task type
-- `struc_info`: Structured ground-truth
-- `question`: Question text
-- `gnd`: Ground-truth answer
-- `answer`: Model prediction
-- `data_source`: Dataset name
-
-**What happens**:
-1. Server validates format
-2. Skips ground-truth merging (already has it)
-3. Runs evaluation directly
-4. Adds to leaderboard
-
-## How It Works
-
-### Validation (`app.py::validate_results_file`)
-
-The validator auto-detects format by checking fields:
-
-```python
-# Format 1: Prediction-only
-is_prediction_only = "id" in sample and "prediction" in sample
-
-# Format 2: Merged
-is_merged = "metadata" in sample and "question" in sample and "answer" in sample
-```
-
-Both formats pass validation if they have:
-- Valid structure
-- Required fields
-- ≥5000 samples
-- Valid qa_types
-
-### Evaluation (`app.py::run_evaluation`)
-
-Uses `evaluation/evaluate_predictions.py` wrapper which:
-
-1. **Auto-detects format**:
-   - Checks for `id` + `prediction` → Prediction-only
-   - Checks for `question` + `gnd` + `struc_info` → Merged
-
-2. **Handles accordingly**:
-   - Prediction-only → Merge with ground-truth first
-   - Merged → Use directly
-
-3. **Runs evaluation**: Calls `evaluate_all_pai.py`
-
-4. **Returns metrics**: 10 metrics across 8 tasks
-
-## Examples
-
-### Example 1: User Submits Predictions
-
-```bash
-# User downloads test set from HuggingFace
-# User runs inference on their model
-# User formats predictions as prediction-only JSON
-# User uploads to leaderboard
-
-# Result: Server merges with private GT → evaluates → adds to board
-```
-
-### Example 2: Internal Testing with Merged File
-
-```bash
-# Developer has complete results.json (with GT)
-# Developer uploads to leaderboard for testing
-
-# Result: Server detects merged format → skips merging → evaluates → adds to board
-```
-
-## File Size Requirements
-
-- **Minimum samples**: 5,000
-- **Full test set**: 6,245 samples
-- Files with <5,000 samples are rejected
-
-## Valid QA Types
-
-- `tal` - Temporal Action Localization
-- `stg` - Spatiotemporal Grounding
-- `next_action` - Next Action Prediction
-- `dense_captioning` / `dense_captioning_gpt` / `dense_captioning_gemini`
-- `video_summary` / `video_summary_gpt` / `video_summary_gemini`
-- `region_caption` / `region_caption_gpt` / `region_caption_gemini`
-- `skill_assessment` - Skill Assessment
-- `cvs_assessment` - CVS Assessment
-
-## Testing
-
-### Test with Prediction-Only Format
-
-```bash
-# Create sample predictions
-python -c "
-import json
-with open('data/sample_predictions.json') as f:
-    data = json.load(f)
-print(f'Format: prediction-only')
-print(f'Samples: {len(data)}')
-print(f'Fields: {list(data[0].keys())}')
-"
-
-# Upload to leaderboard (web interface)
-# Should show: "✓ Valid predictions file (prediction-only format) with 100 samples"
-```
-
-### Test with Merged Format
-
-```bash
-# Check merged format
-python -c "
-import json
-with open('data/results.json') as f:
-    data = json.load(f)
-records = list(data.values())
-print(f'Format: merged')
-print(f'Samples: {len(records)}')
-print(f'Fields: {list(records[0].keys())}')
-"
-
-# Upload to leaderboard (web interface)
-# Should show: "✓ Valid predictions file (merged format) with 6245 samples"
-```
-
-## Error Messages
-
-| Error | Cause | Solution |
-|-------|-------|----------|
-| Missing required field: 'id' | Wrong format | Check if using merged format, should pass now |
-| Missing required field: 'prediction' | Wrong format | Ensure prediction-only has 'prediction' field |
-| Invalid format: Must be either... | Unrecognized structure | Check file structure matches one of two formats |
-| Too few samples (X) | Incomplete file | Should have ~6245 samples for full test set |
-| Invalid qa_type | Wrong task name | Use valid qa_types listed above |
-
-## Implementation Files
-
-- `app.py::validate_results_file()` - Format detection and validation
-- `app.py::run_evaluation()` - Uses wrapper for evaluation
-- `evaluation/evaluate_predictions.py` - Auto-detection wrapper
-- `evaluation/evaluate_all_pai.py` - Core evaluation engine
-
-## Security Notes
-
-- Ground-truth data stored privately in `data/ground_truth.json`
-- Never exposed to users
-- Server-side merging ensures GT integrity
-- Users only submit predictions
-
-## Updates (2026-01-13)
-
-- ✅ Added support for merged format in leaderboard
-- ✅ Auto-detection for both formats
-- ✅ Unified validation and evaluation
-- ✅ Both formats now accepted on web interface

README.md

@@ -69,7 +69,9 @@ Run your model on the MedVidBench test set (6,245 samples) to generate predictio
 
 ### 2. Expected Results Format
 
-Your results file should be a JSON with this structure:
+The leaderboard supports **two formats** for submission:
+
+#### Format 1: Full Format (with Ground Truth)
 
 ```json
 [
@@ -90,6 +92,41 @@ Your results file should be a JSON with this structure:
 ]
 ```
 
+#### Format 2: Prediction-Only Format
+
+```json
+[
+  {
+    "id": "video_id&&start_frame&&end_frame&&fps",
+    "qa_type": "tal",
+    "prediction": "Your model's answer"
+  },
+  ...
+]
+```
+
+**Example**:
+```json
+[
+  {
+    "id": "kcOqlifSukA&&22425&&25124&&1.0",
+    "qa_type": "tal",
+    "prediction": "22.0-78.0, 89.0-94.0 seconds."
+  },
+  {
+    "id": "VsKw5d-4rq8&&13561&&16184&&1.0",
+    "qa_type": "stg",
+    "prediction": "[10, 20, 30, 40] 5.0-10.0 seconds."
+  }
+]
+```
+
+**Key differences**:
+- Format 1: Uses `response` + `ground_truth` fields with full metadata (dictionary format indexed by string keys "0", "1", etc.)
+- Format 2: Uses `id` + `prediction` fields only (list format, GT merged automatically by **index position**)
+- The `id` field format: `{video_id}&&{start_frame}&&{end_frame}&&{fps}` is included for reference but **matching is done by array index**
+- **Important**: Predictions in Format 2 must be in the same order as the test set
+
 **Valid qa_types**:
 - `tal` - Temporal Action Localization
 - `stg` - Spatiotemporal Grounding