docs/timeout_ground_truth.md

# Timeout Ground Truth

## Video: OSU vs Tenn 12.21.24.mkv

### Timeout Events (Chronological)

| Timestamp | Seconds | Team | Notes |
|-----------|---------|------|-------|
| 4:25 | 265 | HOME | First timeout of the game |
| 1:07:30 | 4050 | AWAY | |
| 1:09:40 | 4180 | AWAY | |
| 1:14:07 | 4447 | HOME | |
| 1:16:06 | 4566 | HOME | |
| 1:17:32 | - | - | **Halftime** - All timeouts hidden |
| 1:20:53 | - | - | Scorebug reappears, timeouts reset to 3 each |
| 1:44:54 | 6294 | AWAY | |
| 2:22:30 | - | - | **Game Over** - All timeouts hidden |

### Summary by Half

**First Half:**
- HOME: 3 timeouts used (4:25, 1:14:07, 1:16:06)
- AWAY: 2 timeouts used (1:07:30, 1:09:40)

**Second Half:**
- AWAY: 1 timeout used (1:44:54)

### Total Timeouts to Detect: 6

---

## v4 Baseline Timeout Tracker Performance

### Detected Timeouts (17 total from v4_baseline.json)

| Play # | Timestamp | Seconds | Team | Ground Truth Match |
|--------|-----------|---------|------|-------------------|
| 4 | 4:26 | 266 | HOME | ✓ Matches 4:25 |
| 10 | 9:19 | 559 | HOME | ✗ False positive |
| 15 | 12:58 | 778 | HOME | ✗ False positive |
| 21 | 17:37 | 1057 | HOME | ✗ False positive |
| 35 | 29:05 | 1745 | HOME | ✗ False positive |
| 60 | 48:21 | 2901 | HOME | ✗ False positive |
| 68 | 55:04 | 3304 | HOME | ✗ False positive |
| 79 | 1:02:24 | 3744 | HOME | ✗ False positive |
| 91 | 1:11:54 | 4314 | HOME | ✗ False positive |
| 102 | 1:23:48 | 5028 | AWAY | ✗ False positive |
| 104 | 1:25:40 | 5140 | HOME | ✗ False positive |
| 111 | 1:30:35 | 5435 | HOME | ✗ False positive |
| 131 | 1:44:48 | 6288 | AWAY | ✓ Matches 1:44:54 |
| 146 | 1:57:54 | 7074 | HOME | ✗ False positive |
| 151 | 2:01:50 | 7310 | HOME | ✗ False positive |
| 155 | 2:04:52 | 7492 | HOME | ✗ False positive |
| 175 | 2:18:51 | 8331 | AWAY | ✗ False positive |

### Ground Truth Comparison

| Ground Truth | Detected? | Notes |
|--------------|-----------|-------|
| 4:25 (HOME) | ✓ Play 4 at 4:26 | 1 second off |
| 1:07:30 (AWAY) | ✗ MISSED | No detection near this time |
| 1:09:40 (AWAY) | ✗ MISSED | No detection near this time |
| 1:14:07 (HOME) | ✗ MISSED | No detection near this time |
| 1:16:06 (HOME) | ✗ MISSED | No detection near this time |
| 1:44:54 (AWAY) | ✓ Play 131 at 1:44:48 | 6 seconds off |

### Performance Summary

- **True Positives**: 2 (detected correctly)
- **False Negatives**: 4 (missed real timeouts)
- **False Positives**: 15 (incorrectly flagged as timeout)
- **Recall**: 2/6 = **33%**
- **Precision**: 2/17 = **12%**

### Key Observations

1. **Most first-half timeouts missed**: 4 out of 5 first-half timeouts not detected
2. **High false positive rate**: 15 false positives, mostly flagged as HOME
3. **Timeout indicator region may be misconfigured**: The detector appears to trigger on 40->25 play clock transitions that aren't actual timeouts
4. **Second half better**: The one second-half timeout (1:44:54) was detected

---

## Root Cause Analysis

### How Timeout Detection Works

1. **Trigger**: Timeout detection is only triggered when a 40→25 play clock transition is detected
2. **Classification**: When 40→25 occurs, `classify_40_to_25_reset()` is called which:
   - Compares current timeout counts with last known values via `check_timeout_change()`
   - If `timeout_info.home_timeouts < last_home_timeouts` → HOME timeout
   - If `timeout_info.away_timeouts < last_away_timeouts` → AWAY timeout
   - Otherwise, classified as "special play" (punt/FG/XP)

### Why False Positives Occur

The `DetectTimeouts` class reads timeout indicators via bright pixel analysis in configured regions:
- **Config file**: `data/config/timeout_tracker_region.json`
- **Home region**: (1231, 972, 30, 49) - 30x49 pixel box
- **Away region**: (661, 973, 31, 47) - 31x47 pixel box

**Problems observed in isolation testing:**
- At 0:30 (start): Reads [False, False, False] for both teams (0 timeouts) - should be 3 each
- At 4:30: Reads [True, True, True] for both teams (3 timeouts) - wildly inconsistent
- Many "resets" and spurious transitions detected

**Likely causes:**
1. Region coordinates may be slightly off or need adjustment for this video
2. Brightness threshold (200) or ratio threshold (10%) may not be optimal
3. The timeout indicator ovals may have a different appearance than expected

---

## Updated Analysis (2026-01-09)

### Test Methodology Improvement

The initial v4 baseline test was reading timeout indicators **immediately at the 40→25 transition**. However, the timeout indicator on the scorebug updates with a **delay of 4-6 seconds** after the clock resets.

Updated test methodology:
1. Cached all 17,555 play clock readings at 2 fps
2. Identified 55 total 40→25 transitions
3. Compared timeout readings from 2s BEFORE to 2-6s AFTER each transition

### Validation Rules Implemented

A valid timeout must satisfy:
- **Exactly one team** decreases by **exactly 1**
- **Other team stays the same**
- **Confidence threshold**: Both before and after readings must have confidence >= 0.5

This filters out scorebug visibility issues where both teams' readings change:
- #5 at 9:19: (2,3)→(1,1) - both changed → REJECTED
- #11 at 29:04: (2,3)→(0,0) - both changed → REJECTED
- #35 at 1:30:35: (3,3)→(3,1) - away changed by 2 → REJECTED
- #45 at 1:57:55: (3,2)→(0,2) - home changed by 3 → REJECTED
- #32 at 1:23:48: (3,3)→(3,2) with low confidence (0.487) → REJECTED

---

## ✅ IMPLEMENTED FIX (2026-01-09)

### Final Results (Pipeline Implementation)

| Metric | Value |
|--------|-------|
| **True Positives** | 6 |
| **False Positives** | 0 |
| **False Negatives** | 0 |
| **Recall** | **100%** |
| **Precision** | **100%** |

### Detected Timeouts (All Correct)

| Timestamp | Seconds | Team | Ground Truth |
|-----------|---------|------|--------------|
| 4:24 | 265 | HOME | ✓ Matches 4:25 |
| 67:23 | 4044 | AWAY | ✓ Matches 67:30 |
| 69:38 | 4178 | AWAY | ✓ Matches 69:40 |
| 74:04 | 4444 | HOME | ✓ Matches 74:07 |
| 76:03 | 4563 | HOME | ✓ Matches 76:06 |
| 104:45 | 6285 | AWAY | ✓ Matches 104:54 |

### Implementation Details

#### Key Changes Made:

1. **Delayed timeout check** (`TIMEOUT_CHECK_DELAY = 5.5s`):
   - Store timeout reading when 40→25 transition detected
   - Schedule check 5.5 seconds later
   - Compare readings to detect change

2. **Validation logic**:
   - Require exactly ONE team to decrease by exactly 1
   - Other team's count must stay the same
   - Both readings must have confidence >= 0.5

3. **Multiple entry points**:
   - `classify_40_to_25_reset()` - handles 40→25 in PRE_SNAP state
   - `check_possession_change()` - handles 40→25 during PLAY_IN_PROGRESS state
   - Both now schedule delayed timeout checks

4. **Merger priority fix**:
   - Changed from `normal > special > timeout` to `normal > timeout > special`
   - Timeout plays now have higher priority than special plays

5. **Quiet time filter fix**:
   - Only filter "special" plays in quiet time after normal plays
   - Timeout plays can occur immediately after normal plays end

#### Files Modified:

- `src/tracking/play_identification_checks.py`
- `src/tracking/state_handlers.py`
- `src/tracking/play_state.py`
- `src/tracking/models.py`
- `src/tracking/play_merger.py`
- `src/tracking/clock_reset_identifier.py`

### Key Learnings

1. **Scorebug timeout indicator delay**: The indicator updates 4-6 seconds AFTER the play clock resets from 40→25, not immediately
2. **Confidence thresholds matter**: Low-confidence readings can cause false positives
3. **Validation rules essential**: Must verify exactly one team changes by exactly 1
4. **Multiple code paths**: 40→25 transitions can occur in both PRE_SNAP and PLAY_IN_PROGRESS states