some decent progress generalizing

docs/texas_video_vfr_issue.md

@@ -32,36 +32,53 @@ This means when we request sequential timestamps, we get frames that are **out o
 
 ## Solution Options
 
-### Option 1: Re-encode to CFR (Recommended for simplicity)
+### Option 1: Re-encode to CFR
 ```bash
 ffmpeg -i "OSU vs Texas 01.10.25.mkv" -vsync cfr -r 29.97 -c:v libx264 -preset fast -crf 18 -c:a copy "OSU_vs_Texas_CFR.mkv"
 ```
 - Pros: One-time fix, no code changes needed
-- Cons: Requires re-encoding (takes time, slight quality loss)
+- Cons: Requires re-encoding (takes time, slight quality loss), not video-agnostic
 
-### Option 2: Use FFmpeg for frame extraction
-Replace OpenCV seeking with ffmpeg-based extraction for accurate timestamps.
-- Pros: Works with original file, accurate timestamps
-- Cons: Requires code changes, potentially slower
+### Option 2: Use FFmpeg Pipe for frame extraction (RECOMMENDED)
+Replace OpenCV seeking with ffmpeg piping raw frames to OpenCV.
+- Pros: Works with any video (VFR or CFR), accurate timestamps, **36x faster than current method!**
+- Cons: Requires code changes in pipeline
 
 ### Option 3: Use OpenCV's timestamp-based API with compensation
 Use `CAP_PROP_POS_MSEC` instead of `CAP_PROP_POS_FRAMES`, and track actual timestamps rather than calculated ones.
 - Pros: Minimal code changes
 - Cons: Still unreliable for this video (both methods showed similar errors)
 
-## Recommended Approach
+## Performance Benchmark Results
+
+Tested extracting 300 frames from a 60-second segment:
+
+| Method | Time per Frame | vs Current |
+|--------|----------------|------------|
+| **FFmpeg Pipe to OpenCV** | 0.0023s | **36.55x FASTER** |
+| OpenCV Sequential Read | 0.0037s | 22.49x faster |
+| OpenCV Frame Seeking (current) | 0.0840s | baseline |
+| OpenCV Time Seeking | 0.0851s | 0.99x |
+| FFmpeg Single Frame | 0.2156s | 0.39x (slower) |
 
-For the Texas video specifically, **Option 1 (re-encoding to CFR)** is the cleanest solution since:
-1. The video only needs to be processed once
-2. All existing code continues to work correctly
-3. No risk of introducing new bugs in frame extraction
+The FFmpeg pipe method is both:
+1. **Accurate** - properly handles VFR timestamps
+2. **Fast** - 36x faster than current OpenCV seeking
+
+## Recommended Approach
 
-For a more general solution (to handle any VFR video), implement Option 2 in the frame extraction pipeline.
+**Use FFmpeg Pipe to OpenCV** (Option 2) because:
+1. Works with any video format (video-agnostic)
+2. Correctly handles VFR videos
+3. Significantly faster than current implementation
+4. No need to re-encode videos
 
 ## Diagnostic Scripts
 
 - `scripts/diagnose_video_timestamps.py` - Basic timestamp analysis
 - `scripts/diagnose_vfr_issue.py` - Detailed VFR investigation
+- `scripts/test_ffmpeg_frame_extraction.py` - Verify ffmpeg produces correct frame order
+- `scripts/benchmark_extraction_methods.py` - Performance comparison of extraction methods
 
 ## Debug Output
 

scripts/benchmark_extraction_methods.py

@@ -0,0 +1,561 @@
+"""
+Benchmark different frame extraction methods to assess performance impact.
+
+Compares:
+1. OpenCV frame-based seeking (CAP_PROP_POS_FRAMES) - current method
+2. OpenCV time-based seeking (CAP_PROP_POS_MSEC)
+3. FFmpeg single-frame extraction (one call per frame)
+4. FFmpeg batch extraction (one call for multiple frames)
+5. OpenCV sequential read with skip
+
+Usage:
+    python scripts/benchmark_extraction_methods.py
+"""
+
+import json
+import logging
+import os
+import subprocess
+import sys
+import tempfile
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+import cv2
+import numpy as np
+
+logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
+logger = logging.getLogger(__name__)
+
+
+def load_texas_config() -> Dict[str, Any]:
+    """Load the saved config for Texas video."""
+    config_path = Path("output/OSU_vs_Texas_01_10_25_config.json")
+    with open(config_path, "r") as f:
+        return json.load(f)
+
+
+# =============================================================================
+# Method 1: OpenCV Frame-Based Seeking (Current Method)
+# =============================================================================
+
+
+def benchmark_opencv_frame_seeking(video_path: str, timestamps: List[float]) -> Dict[str, Any]:
+    """
+    Benchmark OpenCV's CAP_PROP_POS_FRAMES seeking.
+    This is the current method used in the pipeline.
+    """
+    cap = cv2.VideoCapture(video_path)
+    if not cap.isOpened():
+        return {"error": "Failed to open video"}
+
+    fps = cap.get(cv2.CAP_PROP_FPS)
+    frames_extracted = 0
+
+    t_start = time.perf_counter()
+
+    for ts in timestamps:
+        frame_num = int(ts * fps)
+        cap.set(cv2.CAP_PROP_POS_FRAMES, frame_num)
+        ret, frame = cap.read()
+        if ret:
+            frames_extracted += 1
+
+    t_elapsed = time.perf_counter() - t_start
+    cap.release()
+
+    return {
+        "method": "OpenCV Frame Seeking",
+        "frames_requested": len(timestamps),
+        "frames_extracted": frames_extracted,
+        "total_time": t_elapsed,
+        "time_per_frame": t_elapsed / len(timestamps),
+        "fps": len(timestamps) / t_elapsed,
+    }
+
+
+# =============================================================================
+# Method 2: OpenCV Time-Based Seeking
+# =============================================================================
+
+
+def benchmark_opencv_time_seeking(video_path: str, timestamps: List[float]) -> Dict[str, Any]:
+    """
+    Benchmark OpenCV's CAP_PROP_POS_MSEC seeking.
+    """
+    cap = cv2.VideoCapture(video_path)
+    if not cap.isOpened():
+        return {"error": "Failed to open video"}
+
+    frames_extracted = 0
+
+    t_start = time.perf_counter()
+
+    for ts in timestamps:
+        cap.set(cv2.CAP_PROP_POS_MSEC, ts * 1000.0)
+        ret, frame = cap.read()
+        if ret:
+            frames_extracted += 1
+
+    t_elapsed = time.perf_counter() - t_start
+    cap.release()
+
+    return {
+        "method": "OpenCV Time Seeking",
+        "frames_requested": len(timestamps),
+        "frames_extracted": frames_extracted,
+        "total_time": t_elapsed,
+        "time_per_frame": t_elapsed / len(timestamps),
+        "fps": len(timestamps) / t_elapsed,
+    }
+
+
+# =============================================================================
+# Method 3: FFmpeg Single Frame Extraction
+# =============================================================================
+
+
+def benchmark_ffmpeg_single_frame(video_path: str, timestamps: List[float]) -> Dict[str, Any]:
+    """
+    Benchmark FFmpeg extraction, one frame at a time.
+    This is the slowest FFmpeg approach but most straightforward.
+    """
+    frames_extracted = 0
+
+    t_start = time.perf_counter()
+
+    for ts in timestamps:
+        with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
+            tmp_path = tmp.name
+
+        try:
+            cmd = [
+                "ffmpeg",
+                "-ss",
+                str(ts),
+                "-i",
+                str(video_path),
+                "-frames:v",
+                "1",
+                "-q:v",
+                "2",
+                "-loglevel",
+                "error",
+                tmp_path,
+                "-y",
+            ]
+
+            result = subprocess.run(cmd, capture_output=True, timeout=30)
+            if result.returncode == 0:
+                frame = cv2.imread(tmp_path)
+                if frame is not None:
+                    frames_extracted += 1
+        finally:
+            if os.path.exists(tmp_path):
+                os.remove(tmp_path)
+
+    t_elapsed = time.perf_counter() - t_start
+
+    return {
+        "method": "FFmpeg Single Frame",
+        "frames_requested": len(timestamps),
+        "frames_extracted": frames_extracted,
+        "total_time": t_elapsed,
+        "time_per_frame": t_elapsed / len(timestamps),
+        "fps": len(timestamps) / t_elapsed,
+    }
+
+
+# =============================================================================
+# Method 4: FFmpeg Batch Extraction (select filter)
+# =============================================================================
+
+
+def benchmark_ffmpeg_batch_select(video_path: str, timestamps: List[float]) -> Dict[str, Any]:
+    """
+    Benchmark FFmpeg batch extraction using select filter.
+    Extracts all frames in a single ffmpeg call using timestamp expressions.
+    """
+    with tempfile.TemporaryDirectory() as tmp_dir:
+        t_start = time.perf_counter()
+
+        # Build select filter expression for all timestamps
+        # Use 'between' to select frames near each timestamp (within 0.02s = ~1 frame at 60fps)
+        tolerance = 0.02
+        conditions = [f"between(t,{ts-tolerance},{ts+tolerance})" for ts in timestamps]
+        select_expr = "+".join(conditions)
+
+        cmd = [
+            "ffmpeg",
+            "-i",
+            str(video_path),
+            "-vf",
+            f"select='{select_expr}',setpts=N/TB",
+            "-vsync",
+            "vfr",
+            "-q:v",
+            "2",
+            "-loglevel",
+            "error",
+            f"{tmp_dir}/frame_%04d.png",
+            "-y",
+        ]
+
+        result = subprocess.run(cmd, capture_output=True, timeout=120)
+
+        t_elapsed = time.perf_counter() - t_start
+
+        # Count extracted frames
+        frames_extracted = len(list(Path(tmp_dir).glob("frame_*.png")))
+
+    return {
+        "method": "FFmpeg Batch Select",
+        "frames_requested": len(timestamps),
+        "frames_extracted": frames_extracted,
+        "total_time": t_elapsed,
+        "time_per_frame": t_elapsed / len(timestamps),
+        "fps": len(timestamps) / t_elapsed,
+        "note": "Single ffmpeg call with select filter",
+    }
+
+
+# =============================================================================
+# Method 5: FFmpeg Segment + Sequential Read
+# =============================================================================
+
+
+def benchmark_ffmpeg_segment_opencv_read(video_path: str, timestamps: List[float], interval: float) -> Dict[str, Any]:
+    """
+    Benchmark: Extract a video segment with ffmpeg, then read sequentially with OpenCV.
+    This is a hybrid approach that might give best accuracy with good speed.
+    """
+    if not timestamps:
+        return {"error": "No timestamps provided"}
+
+    start_ts = min(timestamps) - 1.0  # 1 second buffer
+    end_ts = max(timestamps) + 1.0
+
+    with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as tmp:
+        tmp_path = tmp.name
+
+    try:
+        t_start = time.perf_counter()
+
+        # Extract segment with ffmpeg (accurate seeking)
+        cmd = [
+            "ffmpeg",
+            "-ss",
+            str(start_ts),
+            "-i",
+            str(video_path),
+            "-t",
+            str(end_ts - start_ts),
+            "-c:v",
+            "libx264",
+            "-preset",
+            "ultrafast",
+            "-crf",
+            "18",
+            "-an",  # No audio
+            "-loglevel",
+            "error",
+            tmp_path,
+            "-y",
+        ]
+
+        result = subprocess.run(cmd, capture_output=True, timeout=120)
+        if result.returncode != 0:
+            return {"error": "FFmpeg segment extraction failed"}
+
+        t_extract = time.perf_counter() - t_start
+
+        # Now read sequentially from the segment
+        cap = cv2.VideoCapture(tmp_path)
+        if not cap.isOpened():
+            return {"error": "Failed to open extracted segment"}
+
+        fps = cap.get(cv2.CAP_PROP_FPS)
+        frames_extracted = 0
+
+        # Read frames at the target interval
+        t_read_start = time.perf_counter()
+        frame_skip = max(1, int(interval * fps))
+
+        current_time = 0.0
+        frame_idx = 0
+        while current_time < (end_ts - start_ts):
+            ret, frame = cap.read()
+            if not ret:
+                break
+
+            # Check if this frame is near any of our target timestamps
+            actual_video_time = start_ts + current_time
+            for ts in timestamps:
+                if abs(actual_video_time - ts) < interval / 2:
+                    frames_extracted += 1
+                    break
+
+            # Skip frames
+            for _ in range(frame_skip - 1):
+                cap.grab()
+
+            current_time += interval
+            frame_idx += 1
+
+        cap.release()
+        t_read = time.perf_counter() - t_read_start
+
+        t_elapsed = time.perf_counter() - t_start
+
+    finally:
+        if os.path.exists(tmp_path):
+            os.remove(tmp_path)
+
+    return {
+        "method": "FFmpeg Segment + OpenCV Read",
+        "frames_requested": len(timestamps),
+        "frames_extracted": frames_extracted,
+        "total_time": t_elapsed,
+        "extraction_time": t_extract,
+        "read_time": t_read,
+        "time_per_frame": t_elapsed / len(timestamps),
+        "fps": len(timestamps) / t_elapsed,
+    }
+
+
+# =============================================================================
+# Method 6: OpenCV Sequential Read with Skip (Baseline)
+# =============================================================================
+
+
+def benchmark_opencv_sequential(video_path: str, start_time: float, num_frames: int, interval: float) -> Dict[str, Any]:
+    """
+    Benchmark OpenCV sequential reading with frame skipping.
+    This avoids seeking entirely but requires reading from the start of a range.
+    """
+    cap = cv2.VideoCapture(video_path)
+    if not cap.isOpened():
+        return {"error": "Failed to open video"}
+
+    fps = cap.get(cv2.CAP_PROP_FPS)
+    frame_skip = max(1, int(interval * fps))
+
+    t_start = time.perf_counter()
+
+    # Seek to start position once
+    cap.set(cv2.CAP_PROP_POS_MSEC, start_time * 1000.0)
+
+    frames_extracted = 0
+    for _ in range(num_frames):
+        ret, frame = cap.read()
+        if not ret:
+            break
+        frames_extracted += 1
+
+        # Skip frames
+        for _ in range(frame_skip - 1):
+            cap.grab()
+
+    t_elapsed = time.perf_counter() - t_start
+    cap.release()
+
+    return {
+        "method": "OpenCV Sequential Read",
+        "frames_requested": num_frames,
+        "frames_extracted": frames_extracted,
+        "total_time": t_elapsed,
+        "time_per_frame": t_elapsed / num_frames,
+        "fps": num_frames / t_elapsed,
+        "note": "Single seek + sequential read with skip",
+    }
+
+
+# =============================================================================
+# Method 7: FFmpeg pipe to OpenCV (no temp files)
+# =============================================================================
+
+
+def benchmark_ffmpeg_pipe(video_path: str, start_time: float, duration: float, interval: float) -> Dict[str, Any]:
+    """
+    Benchmark FFmpeg piping raw frames to OpenCV.
+    This avoids temp files and gives accurate timestamps.
+    """
+    # Get video dimensions first
+    cap = cv2.VideoCapture(video_path)
+    if not cap.isOpened():
+        return {"error": "Failed to open video"}
+    width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+    cap.release()
+
+    # Calculate output fps based on interval
+    output_fps = 1.0 / interval
+
+    t_start = time.perf_counter()
+
+    cmd = [
+        "ffmpeg",
+        "-ss",
+        str(start_time),
+        "-i",
+        str(video_path),
+        "-t",
+        str(duration),
+        "-vf",
+        f"fps={output_fps}",
+        "-f",
+        "rawvideo",
+        "-pix_fmt",
+        "bgr24",
+        "-loglevel",
+        "error",
+        "-",
+    ]
+
+    process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+    frame_size = width * height * 3
+    frames_extracted = 0
+
+    while True:
+        raw_frame = process.stdout.read(frame_size)
+        if len(raw_frame) != frame_size:
+            break
+        frame = np.frombuffer(raw_frame, dtype=np.uint8).reshape((height, width, 3))
+        frames_extracted += 1
+
+    process.wait()
+    t_elapsed = time.perf_counter() - t_start
+
+    expected_frames = int(duration / interval)
+
+    return {
+        "method": "FFmpeg Pipe to OpenCV",
+        "frames_requested": expected_frames,
+        "frames_extracted": frames_extracted,
+        "total_time": t_elapsed,
+        "time_per_frame": t_elapsed / max(1, frames_extracted),
+        "fps": frames_extracted / t_elapsed if t_elapsed > 0 else 0,
+        "note": "FFmpeg pipes raw frames, no temp files",
+    }
+
+
+def main():
+    """Run all benchmarks and compare."""
+    config = load_texas_config()
+    video_path = config["video_path"]
+
+    logger.info("=" * 80)
+    logger.info("FRAME EXTRACTION METHOD BENCHMARK")
+    logger.info("=" * 80)
+    logger.info("Video: %s", video_path)
+    logger.info("")
+
+    # Test parameters
+    # Simulate typical pipeline: extract frames every 0.2s over a 60-second segment
+    interval = 0.2  # seconds between frames
+    segment_duration = 60.0  # seconds
+    start_time = 5900.0  # Start in the problem area
+
+    num_frames = int(segment_duration / interval)
+    timestamps = [start_time + (i * interval) for i in range(num_frames)]
+
+    logger.info("Test parameters:")
+    logger.info("  Segment: %.1fs to %.1fs (%.1fs duration)", start_time, start_time + segment_duration, segment_duration)
+    logger.info("  Interval: %.2fs", interval)
+    logger.info("  Frames to extract: %d", num_frames)
+    logger.info("")
+
+    results = []
+
+    # Benchmark each method
+    logger.info("Running benchmarks...")
+    logger.info("-" * 40)
+
+    # 1. Current method: OpenCV frame seeking
+    logger.info("  Testing OpenCV Frame Seeking...")
+    r1 = benchmark_opencv_frame_seeking(video_path, timestamps)
+    results.append(r1)
+    logger.info("    Done: %.2fs total, %.3fs/frame", r1["total_time"], r1["time_per_frame"])
+
+    # 2. OpenCV time seeking
+    logger.info("  Testing OpenCV Time Seeking...")
+    r2 = benchmark_opencv_time_seeking(video_path, timestamps)
+    results.append(r2)
+    logger.info("    Done: %.2fs total, %.3fs/frame", r2["total_time"], r2["time_per_frame"])
+
+    # 3. FFmpeg single frame (only test subset - it's slow)
+    subset_timestamps = timestamps[:20]  # Only test 20 frames
+    logger.info("  Testing FFmpeg Single Frame (20 frames only)...")
+    r3 = benchmark_ffmpeg_single_frame(video_path, subset_timestamps)
+    results.append(r3)
+    logger.info("    Done: %.2fs total, %.3fs/frame", r3["total_time"], r3["time_per_frame"])
+
+    # 4. OpenCV sequential read
+    logger.info("  Testing OpenCV Sequential Read...")
+    r4 = benchmark_opencv_sequential(video_path, start_time, num_frames, interval)
+    results.append(r4)
+    logger.info("    Done: %.2fs total, %.3fs/frame", r4["total_time"], r4["time_per_frame"])
+
+    # 5. FFmpeg pipe
+    logger.info("  Testing FFmpeg Pipe to OpenCV...")
+    r5 = benchmark_ffmpeg_pipe(video_path, start_time, segment_duration, interval)
+    results.append(r5)
+    logger.info("    Done: %.2fs total, %.3fs/frame", r5["total_time"], r5["time_per_frame"])
+
+    logger.info("")
+    logger.info("=" * 80)
+    logger.info("RESULTS SUMMARY")
+    logger.info("=" * 80)
+    logger.info("")
+
+    # Sort by time per frame
+    results_sorted = sorted(results, key=lambda x: x.get("time_per_frame", float("inf")))
+
+    # Find baseline (current method)
+    baseline_time = r1["time_per_frame"]
+
+    logger.info("%-30s %10s %10s %10s %10s", "Method", "Total(s)", "Per Frame", "FPS", "vs Current")
+    logger.info("-" * 80)
+
+    for r in results_sorted:
+        if "error" in r:
+            logger.info("%-30s ERROR: %s", r.get("method", "Unknown"), r["error"])
+            continue
+
+        speedup = baseline_time / r["time_per_frame"] if r["time_per_frame"] > 0 else 0
+        speedup_str = f"{speedup:.2f}x" if speedup != 1.0 else "baseline"
+
+        logger.info(
+            "%-30s %10.2f %10.4f %10.1f %10s",
+            r["method"],
+            r["total_time"],
+            r["time_per_frame"],
+            r["fps"],
+            speedup_str,
+        )
+
+    logger.info("")
+    logger.info("NOTES:")
+    logger.info("  - 'FFmpeg Single Frame' tested with only 20 frames (would be %.1fs for %d frames)", r3["time_per_frame"] * num_frames, num_frames)
+    logger.info("  - 'FFmpeg Pipe' gives accurate timestamps AND good performance")
+    logger.info("  - 'OpenCV Sequential Read' is fastest but requires contiguous segments")
+    logger.info("")
+
+    # Recommendation
+    fastest_accurate = None
+    for r in results_sorted:
+        if r["method"] in ["FFmpeg Pipe to OpenCV", "FFmpeg Segment + OpenCV Read"]:
+            fastest_accurate = r
+            break
+
+    if fastest_accurate:
+        speedup = baseline_time / fastest_accurate["time_per_frame"]
+        logger.info("RECOMMENDATION:")
+        logger.info("  Use '%s' for accurate VFR handling", fastest_accurate["method"])
+        logger.info("  Performance: %.2fx %s than current method", speedup, "faster" if speedup > 1 else "slower")
+
+
+if __name__ == "__main__":
+    main()

scripts/compare_tennessee_plays.py

@@ -0,0 +1,126 @@
+#!/usr/bin/env python3
+"""Compare Tennessee benchmark vs current FFmpeg output to identify missing plays."""
+
+import json
+import logging
+from pathlib import Path
+
+logging.basicConfig(level=logging.INFO, format="%(message)s")
+logger = logging.getLogger(__name__)
+
+
+def load_json(path: str) -> dict:
+    """Load a JSON file."""
+    with open(path, "r") as f:
+        return json.load(f)
+
+
+def find_matching_play(play: dict, play_list: list, tolerance: float = 2.0) -> dict | None:
+    """Find a matching play in the list based on start_time proximity."""
+    for p in play_list:
+        if abs(p["start_time"] - play["start_time"]) <= tolerance:
+            return p
+    return None
+
+
+def main():
+    # Load both outputs
+    benchmark_path = Path("output/benchmarks/v6_baseline.json")
+    current_path = Path("output/OSU_vs_Tenn_12_21_24_plays.json")
+
+    benchmark = load_json(benchmark_path)
+    current = load_json(current_path)
+
+    benchmark_plays = benchmark["plays"]
+    current_plays = current["plays"]
+
+    logger.info("=" * 80)
+    logger.info("TENNESSEE PLAY COMPARISON: Benchmark vs FFmpeg")
+    logger.info("=" * 80)
+    logger.info(f"\nBenchmark plays: {len(benchmark_plays)}")
+    logger.info(f"Current plays:   {len(current_plays)}")
+    logger.info(f"Difference:      {len(benchmark_plays) - len(current_plays)}")
+
+    # Compare play type distributions
+    logger.info("\n" + "-" * 40)
+    logger.info("PLAY TYPE COMPARISON")
+    logger.info("-" * 40)
+
+    benchmark_types = benchmark["stats"]["play_types"]
+    current_types = current["stats"]["play_types"]
+
+    logger.info(f"{'Type':<15} {'Benchmark':<12} {'Current':<12} {'Diff':<10}")
+    for ptype in set(list(benchmark_types.keys()) + list(current_types.keys())):
+        b_count = benchmark_types.get(ptype, 0)
+        c_count = current_types.get(ptype, 0)
+        diff = b_count - c_count
+        logger.info(f"{ptype:<15} {b_count:<12} {c_count:<12} {diff:+d}")
+
+    # Find plays in benchmark that are missing from current
+    logger.info("\n" + "-" * 40)
+    logger.info("PLAYS IN BENCHMARK BUT MISSING FROM CURRENT")
+    logger.info("-" * 40)
+
+    missing_plays = []
+    for bp in benchmark_plays:
+        match = find_matching_play(bp, current_plays, tolerance=3.0)
+        if match is None:
+            missing_plays.append(bp)
+
+    logger.info(f"\nFound {len(missing_plays)} missing plays:\n")
+    for p in missing_plays:
+        logger.info(f"  Play #{p['play_number']:3d}: {p['start_time']:7.1f}s - {p['end_time']:7.1f}s")
+        logger.info(f"            Type: {p['play_type']}, Method: {p['start_method']}")
+        logger.info(f"            Duration: {p['duration']:.1f}s, Clock: {p.get('start_clock_value')} -> {p.get('end_clock_value')}")
+        logger.info("")
+
+    # Group missing plays by type
+    logger.info("-" * 40)
+    logger.info("MISSING PLAYS BY TYPE")
+    logger.info("-" * 40)
+    missing_by_type = {}
+    for p in missing_plays:
+        ptype = p["play_type"]
+        if ptype not in missing_by_type:
+            missing_by_type[ptype] = []
+        missing_by_type[ptype].append(p)
+
+    for ptype, plays in sorted(missing_by_type.items()):
+        logger.info(f"\n{ptype.upper()} plays missing: {len(plays)}")
+        for p in plays:
+            logger.info(f"  - #{p['play_number']}: {p['start_time']:.1f}s (method: {p['start_method']})")
+
+    # Find plays in current that don't exist in benchmark (extra plays)
+    logger.info("\n" + "-" * 40)
+    logger.info("PLAYS IN CURRENT BUT NOT IN BENCHMARK (extras)")
+    logger.info("-" * 40)
+
+    extra_plays = []
+    for cp in current_plays:
+        match = find_matching_play(cp, benchmark_plays, tolerance=3.0)
+        if match is None:
+            extra_plays.append(cp)
+
+    if extra_plays:
+        logger.info(f"\nFound {len(extra_plays)} extra plays:\n")
+        for p in extra_plays:
+            logger.info(f"  Play #{p['play_number']:3d}: {p['start_time']:7.1f}s - {p['end_time']:7.1f}s")
+            logger.info(f"            Type: {p['play_type']}, Method: {p['start_method']}")
+    else:
+        logger.info("\nNo extra plays found (current is a subset of benchmark)")
+
+    # Check first play difference - benchmark starts at ~121s, current starts at ~180s
+    logger.info("\n" + "-" * 40)
+    logger.info("FIRST FEW PLAYS COMPARISON")
+    logger.info("-" * 40)
+    logger.info("\nBenchmark first 5 plays:")
+    for p in benchmark_plays[:5]:
+        logger.info(f"  #{p['play_number']}: {p['start_time']:.1f}s - {p['play_type']} ({p['start_method']})")
+
+    logger.info("\nCurrent first 5 plays:")
+    for p in current_plays[:5]:
+        logger.info(f"  #{p['play_number']}: {p['start_time']:.1f}s - {p['play_type']} ({p['start_method']})")
+
+
+if __name__ == "__main__":
+    main()

scripts/diagnose_tennessee_regression.py

@@ -0,0 +1,120 @@
+#!/usr/bin/env python3
+"""Diagnose Tennessee regression - check what's happening with frames at key timestamps."""
+
+import sys
+import logging
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+import cv2
+import numpy as np
+from src.video.ffmpeg_reader import FFmpegFrameReader
+from src.readers.flags import FlagReader
+
+logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
+logger = logging.getLogger(__name__)
+
+
+def check_frames_at_timestamp(video_path: str, start_time: float, duration: float = 10.0):
+    """Check what frames are extracted at a given timestamp range."""
+    logger.info(f"\n{'='*60}")
+    logger.info(f"Checking frames from {start_time}s to {start_time + duration}s")
+    logger.info("=" * 60)
+
+    frames = []
+    with FFmpegFrameReader(video_path, start_time, start_time + duration, 0.5) as reader:
+        for timestamp, frame in reader:
+            frames.append((timestamp, frame.copy()))
+            logger.info(f"  Frame at t={timestamp:.2f}s, shape={frame.shape}")
+
+    logger.info(f"  Total frames extracted: {len(frames)}")
+    return frames
+
+
+def check_flag_detection(video_path: str, start_time: float, duration: float = 30.0):
+    """Check flag detection at a timestamp range using direct yellow detection."""
+    import json
+
+    # Load saved session config (this is what main.py uses)
+    session_config_path = Path("output/OSU_vs_Tenn_12_21_24_config.json")
+    with open(session_config_path, "r") as f:
+        session_config = json.load(f)
+
+    # Extract flag region from session config (what main.py actually uses)
+    flag_x_offset = session_config["flag_x_offset"]
+    flag_y_offset = session_config["flag_y_offset"]
+    flag_width = session_config["flag_width"]
+    flag_height = session_config["flag_height"]
+
+    # Scorebug location from session config
+    scorebug_x = session_config["scorebug_x"]
+    scorebug_y = session_config["scorebug_y"]
+
+    # Calculate absolute flag region
+    x = scorebug_x + flag_x_offset
+    y = scorebug_y + flag_y_offset
+    w = flag_width
+    h = flag_height
+
+    logger.info(f"Flag region from SESSION config: x={x}, y={y}, w={w}, h={h}")
+    logger.info(f"  (offsets: {flag_x_offset}, {flag_y_offset}, size: {flag_width}x{flag_height})")
+
+    # Use the FlagReader with actual session config values
+    flag_reader = FlagReader(
+        flag_x_offset=flag_x_offset,
+        flag_y_offset=flag_y_offset,
+        flag_width=flag_width,
+        flag_height=flag_height,
+    )
+
+    logger.info(f"\n{'='*60}")
+    logger.info(f"Flag detection from {start_time}s to {start_time + duration}s")
+    logger.info("=" * 60)
+
+    flag_events = []
+    with FFmpegFrameReader(video_path, start_time, start_time + duration, 0.5) as reader:
+        for timestamp, frame in reader:
+            # Check for flag using fixed location
+            result = flag_reader.read_from_fixed_location(frame, (x, y, w, h))
+            yellow_pct = result.yellow_ratio * 100
+            if result.detected:
+                logger.info(f"  t={timestamp:7.1f}s: FLAG! yellow={yellow_pct:.1f}%, hue={result.mean_hue:.1f}")
+                flag_events.append(timestamp)
+            elif yellow_pct > 10:
+                logger.info(f"  t={timestamp:7.1f}s: some yellow={yellow_pct:.1f}%, hue={result.mean_hue:.1f}")
+
+    logger.info(f"\nTotal FLAG frames detected: {len(flag_events)}")
+
+
+def main():
+    video_path = "/Users/andytaylor/Documents/Personal/cfb40/full_videos/OSU vs Tenn 12.21.24.mkv"
+
+    # Check the very start of the video - the missing first play starts at ~121.9s
+    logger.info("\n" + "=" * 80)
+    logger.info("INVESTIGATING MISSING FIRST PLAY (~121.9s)")
+    logger.info("=" * 80)
+    # Just check what frames we get around the first play
+    check_frames_at_timestamp(video_path, 115.0, 20.0)
+
+    # Check one of the missing flag plays - the first one at ~340.6s
+    logger.info("\n" + "=" * 80)
+    logger.info("INVESTIGATING MISSING FLAG PLAY (~340.6s)")
+    logger.info("=" * 80)
+    check_flag_detection(video_path, 335.0, 40.0)
+
+    # Check another missing flag play at ~422.9s
+    logger.info("\n" + "=" * 80)
+    logger.info("INVESTIGATING MISSING FLAG PLAY (~422.9s)")
+    logger.info("=" * 80)
+    check_flag_detection(video_path, 418.0, 30.0)
+
+    # Check the ONE flag play that WAS detected (at 3552.5s)
+    logger.info("\n" + "=" * 80)
+    logger.info("INVESTIGATING DETECTED FLAG PLAY (~3552.5s)")
+    logger.info("=" * 80)
+    check_flag_detection(video_path, 3548.0, 20.0)
+
+
+if __name__ == "__main__":
+    main()

scripts/find_flag_ground_truth.py

@@ -178,28 +178,82 @@ class ScanResult:
         }
 
 
-def load_flag_region_config() -> Optional[Dict[str, Any]]:
-    """Load the FLAG region configuration."""
-    config_path = DATA_CONFIG_DIR / "flag_region.json"
-    if not config_path.exists():
-        logger.error("FLAG region config not found: %s", config_path)
-        logger.error("Please run test_flag_region_selection.py first")
-        return None
+def get_video_basename(video_path: str) -> str:
+    """Get a clean basename from video path for config naming."""
+    basename = Path(video_path).stem
+    for char in [" ", ".", "-"]:
+        basename = basename.replace(char, "_")
+    while "__" in basename:
+        basename = basename.replace("__", "_")
+    return basename.strip("_")
 
-    with open(config_path, "r", encoding="utf-8") as f:
-        return json.load(f)
 
+def load_flag_region_config(video_path: str) -> Optional[Dict[str, Any]]:
+    """
+    Load the FLAG region configuration for a specific video.
+
+    Tries video-specific config first, then falls back to generic.
+    """
+    video_basename = get_video_basename(video_path)
+
+    # Try video-specific flag config first
+    video_flag_config_path = OUTPUT_DIR / f"{video_basename}_flag_config.json"
+    if video_flag_config_path.exists():
+        logger.info("Loading flag config from: %s", video_flag_config_path)
+        with open(video_flag_config_path, "r", encoding="utf-8") as f:
+            return json.load(f)
+
+    # Try session config (has flag_x_offset, etc.)
+    session_config_path = OUTPUT_DIR / f"{video_basename}_config.json"
+    if session_config_path.exists():
+        logger.info("Loading flag config from session: %s", session_config_path)
+        with open(session_config_path, "r", encoding="utf-8") as f:
+            session_config = json.load(f)
+
+        # Convert session config format to flag config format
+        if session_config.get("flag_x_offset") is not None:
+            return {
+                "flag_region": {
+                    "x_offset": session_config["flag_x_offset"],
+                    "y_offset": session_config["flag_y_offset"],
+                    "width": session_config["flag_width"],
+                    "height": session_config["flag_height"],
+                },
+                "source_video": Path(video_path).name,
+            }
+
+    # Fall back to generic config (legacy)
+    generic_config_path = DATA_CONFIG_DIR / "flag_region.json"
+    if generic_config_path.exists():
+        logger.warning("Using generic flag config (may not match video): %s", generic_config_path)
+        with open(generic_config_path, "r", encoding="utf-8") as f:
+            return json.load(f)
+
+    logger.error("No FLAG region config found for video: %s", video_basename)
+    logger.error("Please run test_flag_region_selection.py --video '%s' first", video_path)
+    return None
+
+
+def load_scorebug_config(video_path: str) -> Optional[Tuple[BBox, str]]:
+    """Load the scorebug region from config for a specific video."""
+    video_basename = get_video_basename(video_path)
+
+    # Try video-specific config first
+    config_path = OUTPUT_DIR / f"{video_basename}_config.json"
 
-def load_scorebug_config() -> Optional[Tuple[BBox, str]]:
-    """Load the scorebug region from config."""
-    config_files = list(OUTPUT_DIR.glob("*_config.json"))
-    main_configs = [f for f in config_files if "playclock" not in f.name and "timeout" not in f.name]
+    if not config_path.exists():
+        # Fall back to most recently modified config (legacy behavior)
+        config_files = list(OUTPUT_DIR.glob("*_config.json"))
+        main_configs = [f for f in config_files if "playclock" not in f.name and "timeout" not in f.name and "flag" not in f.name]
 
-    if not main_configs:
-        logger.error("No scorebug config found in output/")
-        return None
+        if not main_configs:
+            logger.error("No scorebug config found in output/")
+            return None
 
-    config_path = max(main_configs, key=lambda p: p.stat().st_mtime)
+        config_path = max(main_configs, key=lambda p: p.stat().st_mtime)
+        logger.warning("Using fallback config (may not match video): %s", config_path)
+
+    logger.info("Loading scorebug config from: %s", config_path)
 
     with open(config_path, "r", encoding="utf-8") as f:
         config = json.load(f)
@@ -706,23 +760,27 @@ def main() -> int:
     print("   FLAG Ground Truth Scanner")
     print("=" * 60)
 
-    # Load configs
-    flag_config = load_flag_region_config()
+    # Determine video path first (needed for config loading)
+    video_path = args.video if args.video else str(DEFAULT_VIDEO)
+    if not Path(video_path).exists():
+        logger.error("Video not found: %s", video_path)
+        return 1
+
+    video_basename = get_video_basename(video_path)
+    print(f"\nVideo: {Path(video_path).name}")
+    print(f"Config basename: {video_basename}")
+
+    # Load configs for specific video
+    flag_config = load_flag_region_config(video_path)
     if flag_config is None:
         return 1
 
-    scorebug_result = load_scorebug_config()
+    scorebug_result = load_scorebug_config(video_path)
     if scorebug_result is None:
         return 1
 
     scorebug_bbox, _ = scorebug_result
 
-    # Determine video path
-    video_path = args.video if args.video else str(DEFAULT_VIDEO)
-    if not Path(video_path).exists():
-        logger.error("Video not found: %s", video_path)
-        return 1
-
     # Run scan
     result = scan_video_for_flags(
         video_path=video_path,
@@ -737,9 +795,9 @@ def main() -> int:
     # Print results
     print_results(result)
 
-    # Save results
+    # Save results to video-specific file
     OUTPUT_CACHE_DIR.mkdir(parents=True, exist_ok=True)
-    output_path = OUTPUT_CACHE_DIR / "flag_candidates.json"
+    output_path = OUTPUT_CACHE_DIR / f"{video_basename}_flag_candidates.json"
 
     with open(output_path, "w", encoding="utf-8") as f:
         json.dump(result.to_dict(), f, indent=2)

scripts/test_flag_region_selection.py

@@ -6,14 +6,19 @@ This script allows interactive selection of the FLAG indicator region on the sco
 The FLAG region is where "1st & 10" / "FLAG" text appears on the scorebug.
 
 Usage:
+    # Use default Tennessee video
     python scripts/test_flag_region_selection.py
 
+    # Use a specific video
+    python scripts/test_flag_region_selection.py --video "full_videos/OSU vs Texas 01.10.25.mkv"
+
 The script will:
-1. Load sample frames from the test video
+1. Load sample frames from the specified video
 2. Display the frame with the existing scorebug region highlighted
 3. Allow user to click/drag to select the FLAG region
-4. Save the selected region to data/config/flag_region.json
-5. Display a cropped preview of the selected region
+4. Save the selected region to output/{video_basename}_flag_config.json
+5. Optionally update the session config if it exists
+6. Display a cropped preview of the selected region
 """
 
 import json
@@ -365,18 +370,33 @@ def show_preview(frame: np.ndarray[Any, Any], flag_region_bbox: BBox, scorebug_b
     cv2.destroyAllWindows()
 
 
-def save_flag_region(flag_bbox: BBox, source_video: str, scorebug_template: str) -> Path:
+def get_video_basename(video_path: str) -> str:
+    """Get a clean basename from video path for config naming."""
+    basename = Path(video_path).stem
+    for char in [" ", ".", "-"]:
+        basename = basename.replace(char, "_")
+    while "__" in basename:
+        basename = basename.replace("__", "_")
+    return basename.strip("_")
+
+
+def save_flag_region(flag_bbox: BBox, source_video: str, scorebug_template: str, video_path: str) -> Path:
     """
-    Save the FLAG region configuration to JSON.
+    Save the FLAG region configuration to video-specific JSON file.
+
+    Also updates the session config if it exists.
 
     Args:
         flag_bbox: FLAG region bounding box (relative to scorebug)
         source_video: Name of the source video
         scorebug_template: Name of the scorebug template file
+        video_path: Full path to the video (used for naming)
 
     Returns:
         Path to the saved config file
     """
+    video_basename = get_video_basename(video_path)
+
     config = {
         "flag_region": {
             "x_offset": flag_bbox.x,
@@ -388,13 +408,34 @@ def save_flag_region(flag_bbox: BBox, source_video: str, scorebug_template: str)
         "scorebug_template": scorebug_template,
     }
 
-    output_path = DATA_CONFIG_DIR / "flag_region.json"
-    DATA_CONFIG_DIR.mkdir(parents=True, exist_ok=True)
+    # Save to video-specific file in output directory
+    output_path = OUTPUT_DIR / f"{video_basename}_flag_config.json"
+    OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
 
     with open(output_path, "w", encoding="utf-8") as f:
         json.dump(config, f, indent=2)
 
     logger.info("Saved FLAG region config to: %s", output_path)
+
+    # Also update session config if it exists
+    session_config_path = OUTPUT_DIR / f"{video_basename}_config.json"
+    if session_config_path.exists():
+        try:
+            with open(session_config_path, "r", encoding="utf-8") as f:
+                session_config = json.load(f)
+
+            session_config["flag_x_offset"] = flag_bbox.x
+            session_config["flag_y_offset"] = flag_bbox.y
+            session_config["flag_width"] = flag_bbox.width
+            session_config["flag_height"] = flag_bbox.height
+
+            with open(session_config_path, "w", encoding="utf-8") as f:
+                json.dump(session_config, f, indent=2)
+
+            logger.info("Updated session config: %s", session_config_path)
+        except Exception as e:
+            logger.warning("Could not update session config: %s", e)
+
     return output_path
 
 
@@ -421,32 +462,84 @@ def print_instructions() -> None:
     input("\nPress Enter to start selection...")
 
 
+def parse_args():
+    """Parse command line arguments."""
+    import argparse
+
+    parser = argparse.ArgumentParser(description="Select FLAG region for a video")
+    parser.add_argument(
+        "--video",
+        type=str,
+        default=str(DEFAULT_VIDEO),
+        help="Path to video file (default: OSU vs Tenn 12.21.24.mkv)",
+    )
+    parser.add_argument(
+        "--start-time",
+        type=float,
+        default=DEFAULT_START_TIME,
+        help="Start time in seconds for frame extraction (default: 38:40)",
+    )
+    return parser.parse_args()
+
+
+def load_scorebug_config_for_video(video_path: str) -> Optional[Tuple[BBox, str]]:
+    """Load scorebug config for a specific video."""
+    video_basename = get_video_basename(video_path)
+    session_config_path = OUTPUT_DIR / f"{video_basename}_config.json"
+
+    if session_config_path.exists():
+        with open(session_config_path, "r", encoding="utf-8") as f:
+            config = json.load(f)
+
+        scorebug_bbox = BBox(
+            x=config["scorebug_x"],
+            y=config["scorebug_y"],
+            width=config["scorebug_width"],
+            height=config["scorebug_height"],
+        )
+        template_path = config.get("template_path", "")
+        logger.info("Loaded session config from: %s", session_config_path)
+        return scorebug_bbox, template_path
+
+    # Fall back to generic config
+    logger.warning("No session config found for video, trying generic config...")
+    return load_saved_scorebug_config()
+
+
 def main() -> int:
     """Main entry point for FLAG region selection test."""
+    args = parse_args()
     print_banner()
 
-    # Load existing scorebug config
-    result = load_saved_scorebug_config()
+    # Determine video path
+    video_path = Path(args.video)
+    if not video_path.exists():
+        # Try relative to project root
+        video_path = PROJECT_ROOT / args.video
+        if not video_path.exists():
+            print(f"\nERROR: Video not found: {args.video}")
+            return 1
+
+    print(f"Using video: {video_path.name}")
+    video_basename = get_video_basename(str(video_path))
+    print(f"Config basename: {video_basename}")
+
+    # Load scorebug config for this video
+    result = load_scorebug_config_for_video(str(video_path))
     if result is None:
         print("\nERROR: No existing scorebug config found.")
-        print("Please run the main pipeline first to set up the scorebug region.")
+        print("Please run main.py first to set up the scorebug region for this video.")
         return 1
 
     scorebug_bbox, template_path = result
     print(f"\nLoaded scorebug region: {scorebug_bbox.to_tuple()}")
 
-    # Determine video path
-    video_path = DEFAULT_VIDEO
-    if not video_path.exists():
-        print(f"\nERROR: Default video not found: {video_path}")
-        return 1
-
-    print(f"Using video: {video_path.name}")
-    print(f"Starting at: {DEFAULT_START_TIME // 60}:{DEFAULT_START_TIME % 60:02d}")
+    start_time = args.start_time
+    print(f"Starting at: {int(start_time) // 60}:{int(start_time) % 60:02d}")
 
     # Extract sample frames
     print("\nExtracting sample frames...")
-    frames = extract_sample_frames(str(video_path), DEFAULT_START_TIME, num_frames=1, interval=0.0)
+    frames = extract_sample_frames(str(video_path), start_time, num_frames=1, interval=0.0)
 
     if not frames:
         print("ERROR: Failed to extract frames from video")
@@ -486,7 +579,7 @@ def main() -> int:
         video_name = video_path.name
         template_name = Path(template_path).name if template_path else "unknown"
 
-        save_path = save_flag_region(flag_bbox, video_name, template_name)
+        save_path = save_flag_region(flag_bbox, video_name, template_name, str(video_path))
         print(f"\n✓ FLAG region saved to: {save_path}")
     else:
         print("\nSelection not saved.")

scripts/test_frame_alignment.py

@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""Test frame alignment between different FFmpeg extraction methods."""
+
+import sys
+import logging
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+import json
+import numpy as np
+from src.video.ffmpeg_reader import FFmpegFrameReader
+from src.readers.flags import FlagReader
+
+logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
+logger = logging.getLogger(__name__)
+
+
+def test_frame_at_timestamp(video_path: str, target_time: float, flag_reader: FlagReader, scorebug_x: int, scorebug_y: int):
+    """Extract and test a specific frame."""
+    # Calculate absolute flag region
+    x = scorebug_x + flag_reader.flag_x_offset
+    y = scorebug_y + flag_reader.flag_y_offset
+    w = flag_reader.flag_width
+    h = flag_reader.flag_height
+
+    with FFmpegFrameReader(video_path, target_time, target_time + 0.5, 0.5) as reader:
+        for timestamp, frame in reader:
+            result = flag_reader.read_from_fixed_location(frame, (x, y, w, h))
+            yellow_pct = result.yellow_ratio * 100
+            logger.info(f"  t={timestamp:.1f}s: yellow={yellow_pct:.1f}%, hue={result.mean_hue:.1f}, detected={result.detected}")
+            return timestamp, yellow_pct, result.mean_hue
+
+    return None, 0, 0
+
+
+def main():
+    video_path = "/Users/andytaylor/Documents/Personal/cfb40/full_videos/OSU vs Tenn 12.21.24.mkv"
+
+    # Load session config
+    with open("output/OSU_vs_Tenn_12_21_24_config.json", "r") as f:
+        config = json.load(f)
+
+    # Create flag reader
+    flag_reader = FlagReader(
+        flag_x_offset=config["flag_x_offset"],
+        flag_y_offset=config["flag_y_offset"],
+        flag_width=config["flag_width"],
+        flag_height=config["flag_height"],
+    )
+
+    scorebug_x = config["scorebug_x"]
+    scorebug_y = config["scorebug_y"]
+
+    logger.info("=" * 80)
+    logger.info("FRAME ALIGNMENT TEST")
+    logger.info(f"Flag region: offset=({flag_reader.flag_x_offset}, {flag_reader.flag_y_offset})")
+    logger.info("=" * 80)
+
+    # Test 1: Extract frame at 340.5s directly (like diagnostic)
+    logger.info("\nTest 1: Extract frame at 340.5s directly (start_time=340.5)")
+    test_frame_at_timestamp(video_path, 340.5, flag_reader, scorebug_x, scorebug_y)
+
+    # Test 2: Extract frame 681 from start (like pipeline chunk 0)
+    logger.info("\nTest 2: Extract from start and iterate to frame 681 (340.5s)")
+    frame_count = 0
+    target_frame = 681  # 340.5 / 0.5 = 681
+    
+    # Only extract 5 frames around the target to save time
+    start_time = (target_frame - 2) * 0.5  # 339.5s
+    end_time = (target_frame + 3) * 0.5  # 342.0s
+    
+    x = scorebug_x + flag_reader.flag_x_offset
+    y = scorebug_y + flag_reader.flag_y_offset
+    w = flag_reader.flag_width
+    h = flag_reader.flag_height
+
+    with FFmpegFrameReader(video_path, start_time, end_time, 0.5) as reader:
+        for timestamp, frame in reader:
+            result = flag_reader.read_from_fixed_location(frame, (x, y, w, h))
+            yellow_pct = result.yellow_ratio * 100
+            logger.info(f"  t={timestamp:.1f}s: yellow={yellow_pct:.1f}%, hue={result.mean_hue:.1f}, detected={result.detected}")
+
+    # Test 3: Extract from time 0 and iterate to 340.5s (exactly like pipeline)
+    # This would take too long, so let's just test the first few frames to see if there's a pattern
+    logger.info("\nTest 3: Extract first 10 frames from time 0 (like pipeline chunk 0)")
+    with FFmpegFrameReader(video_path, 0.0, 5.0, 0.5) as reader:
+        for timestamp, frame in reader:
+            # Just print frame info, no flag detection (not expected at start of video)
+            logger.info(f"  t={timestamp:.1f}s: frame shape={frame.shape}")
+
+    logger.info("\n" + "=" * 80)
+    logger.info("CONCLUSION: If Test 1 and Test 2 show same results, frame alignment is correct")
+    logger.info("=" * 80)
+
+
+if __name__ == "__main__":
+    main()

scripts/test_timeout_at_transitions.py

@@ -7,15 +7,23 @@ This script:
 3. Compares the change in timeouts to determine if this was a timeout event
 4. Compares results against ground truth
 
-Ground truth timeouts:
+Ground truth timeouts (Tennessee video):
 - 4:25 (HOME)      -> transition at ~4:26
 - 1:07:30 (AWAY)   -> transition at ~1:07:24
 - 1:09:40 (AWAY)   -> transition at ~1:09:38
 - 1:14:07 (HOME)   -> transition at ~1:14:05
 - 1:16:06 (HOME)   -> transition at ~1:16:03
 - 1:44:54 (AWAY)   -> transition at ~1:44:48
+
+Usage:
+    # Default Tennessee video
+    python scripts/test_timeout_at_transitions.py
+
+    # Specific video
+    python scripts/test_timeout_at_transitions.py --video "full_videos/OSU vs Texas 01.10.25.mkv"
 """
 
+import argparse
 import json
 import logging
 import sys
@@ -33,6 +41,19 @@ from detection.timeouts import DetectTimeouts
 logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
 logger = logging.getLogger(__name__)
 
+PROJECT_ROOT = Path(__file__).parent.parent
+OUTPUT_DIR = PROJECT_ROOT / "output"
+
+
+def get_video_basename(video_path: str) -> str:
+    """Get a clean basename from video path for config naming."""
+    basename = Path(video_path).stem
+    for char in [" ", ".", "-"]:
+        basename = basename.replace(char, "_")
+    while "__" in basename:
+        basename = basename.replace("__", "_")
+    return basename.strip("_")
+
 
 # Ground truth timeouts (timestamp in seconds, team)
 GROUND_TRUTH_TIMEOUTS = [
@@ -87,12 +108,27 @@ def test_timeout_at_transitions():
     transitions = cache["transitions_40_to_25"]
     logger.info("Loaded %d transitions from cache", len(transitions))
 
-    # Load timeout tracker config
-    config_path = Path("data/config/timeout_tracker_region.json")
+    # Parse arguments
+    parser = argparse.ArgumentParser(description="Test timeout tracking at transitions")
+    parser.add_argument("--video", type=str, default="full_videos/OSU vs Tenn 12.21.24.mkv", help="Path to video file")
+    args = parser.parse_args()
+
+    video_path = args.video
+    video_basename = get_video_basename(video_path)
+
+    # Try video-specific timeout config first
+    config_path = OUTPUT_DIR / f"{video_basename}_timeout_config.json"
+    if not config_path.exists():
+        # Fall back to generic config
+        config_path = Path("data/config/timeout_tracker_region.json")
+
     if not config_path.exists():
         logger.error("Timeout config not found: %s", config_path)
+        logger.error("Try running main.py first to generate timeout config for this video")
         return
 
+    logger.info("Using timeout config: %s", config_path)
+
     # Initialize timeout tracker
     tracker = DetectTimeouts(config_path=str(config_path))
     if not tracker.is_configured():
@@ -100,7 +136,6 @@ def test_timeout_at_transitions():
         return
 
     # Open video
-    video_path = "full_videos/OSU vs Tenn 12.21.24.mkv"
     cap = cv2.VideoCapture(video_path)
     if not cap.isOpened():
         logger.error("Could not open video: %s", video_path)

scripts/verify_config_loading.py

@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+"""Verify what configuration is being loaded for each video."""
+
+import json
+from pathlib import Path
+
+OUTPUT_DIR = Path("output")
+DATA_CONFIG_DIR = Path("data/config")
+
+
+def get_video_basename(video_name: str) -> str:
+    """Get a clean basename from video name."""
+    basename = Path(video_name).stem
+    for char in [" ", ".", "-"]:
+        basename = basename.replace(char, "_")
+    while "__" in basename:
+        basename = basename.replace("__", "_")
+    return basename.strip("_")
+
+
+def print_video_config(video_name: str):
+    """Print configuration for a specific video."""
+    basename = get_video_basename(video_name)
+    print(f"\n{'='*60}")
+    print(f"VIDEO: {video_name}")
+    print(f"Config basename: {basename}")
+    print("=" * 60)
+
+    # Check session config
+    session_config_path = OUTPUT_DIR / f"{basename}_config.json"
+    if session_config_path.exists():
+        with open(session_config_path, "r") as f:
+            config = json.load(f)
+        print(f"\n  Session Config ({session_config_path.name}):")
+        print(f"    Scorebug: ({config.get('scorebug_x')}, {config.get('scorebug_y')})")
+        print(f"    Flag offset: ({config.get('flag_x_offset')}, {config.get('flag_y_offset')})")
+        print(f"    Flag size: {config.get('flag_width')}x{config.get('flag_height')}")
+    else:
+        print(f"\n  Session Config: NOT FOUND (expected: {session_config_path})")
+
+    # Check timeout config
+    timeout_config_path = OUTPUT_DIR / f"{basename}_timeout_config.json"
+    if timeout_config_path.exists():
+        print(f"  Timeout Config: {timeout_config_path.name}")
+    else:
+        print(f"  Timeout Config: NOT FOUND")
+
+    # Check playclock config
+    playclock_config_path = OUTPUT_DIR / f"{basename}_playclock_config.json"
+    if playclock_config_path.exists():
+        print(f"  Playclock Config: {playclock_config_path.name}")
+    else:
+        print(f"  Playclock Config: NOT FOUND")
+
+
+def main():
+    print("=" * 80)
+    print("CONFIGURATION FILES STATUS")
+    print("=" * 80)
+
+    # Known videos
+    videos = [
+        "OSU vs Tenn 12.21.24.mkv",
+        "OSU vs Texas 01.10.25.mkv",
+        "OSU vs Oregon 01.01.25.mkv",
+        "OSU vs ND 01.20.25.mp4",
+    ]
+
+    for video in videos:
+        print_video_config(video)
+
+    # Check for GENERIC configs that might cause confusion
+    print("\n" + "=" * 80)
+    print("WARNING: GENERIC CONFIGS (may cause video cross-contamination)")
+    print("=" * 80)
+
+    generic_configs = [
+        DATA_CONFIG_DIR / "flag_region.json",
+        DATA_CONFIG_DIR / "play_clock_region.json",
+        DATA_CONFIG_DIR / "timeout_tracker_region.json",
+    ]
+
+    for config_path in generic_configs:
+        if config_path.exists():
+            with open(config_path, "r") as f:
+                config = json.load(f)
+            source = config.get("source_video", "UNKNOWN")
+            print(f"  {config_path.name}: source={source}")
+            print(f"    -> Scripts should use video-specific configs instead!")
+        else:
+            print(f"  {config_path.name}: NOT FOUND (OK)")
+
+
+if __name__ == "__main__":
+    main()

src/pipeline/models.py

@@ -103,6 +103,7 @@ class ParallelProcessingConfig(BaseModel):
     fixed_scorebug_coords: Tuple[int, int, int, int] = Field(..., description="(x, y, w, h) for scorebug region")
     template_library_path: Optional[str] = Field(None, description="Path to template library directory")
     timeout_config_path: Optional[str] = Field(None, description="Path to timeout tracker config")
+    scorebug_template_path: Optional[str] = Field(None, description="Path to scorebug template image for verification")
     # FLAG region config (offsets relative to scorebug)
     flag_x_offset: Optional[int] = Field(None, description="FLAG region X offset from scorebug")
     flag_y_offset: Optional[int] = Field(None, description="FLAG region Y offset from scorebug")

src/pipeline/parallel.py

@@ -51,8 +51,9 @@ def _init_chunk_detectors(config: ParallelProcessingConfig) -> Tuple[Any, Any, A
     from readers import FlagReader, ReadPlayClock
     from setup import DigitTemplateLibrary, PlayClockRegionConfig, PlayClockRegionExtractor
 
-    # Create scorebug detector with fixed region
-    scorebug_detector = DetectScoreBug(template_path=None, use_split_detection=True)
+    # Create scorebug detector with fixed region and template for verification
+    # Template is needed to correctly detect when scorebug is NOT present (replays, commercials)
+    scorebug_detector = DetectScoreBug(template_path=config.scorebug_template_path, use_split_detection=True)
     scorebug_detector.set_fixed_region(config.fixed_scorebug_coords)
 
     # Create play clock region extractor
@@ -115,6 +116,7 @@ def _process_frame(
     flag_reader: Any,
     stats: Dict[str, int],
     fixed_playclock_coords: Optional[Tuple[int, int, int, int]] = None,
+    fixed_scorebug_coords: Optional[Tuple[int, int, int, int]] = None,
 ) -> Dict[str, Any]:
     """
     Process a single video frame and extract detection results.
@@ -129,21 +131,32 @@ def _process_frame(
         flag_reader: Initialized FlagReader (or None).
         stats: Mutable dict to update with detection statistics.
         fixed_playclock_coords: Optional fixed play clock coordinates for padded matching.
+        fixed_scorebug_coords: Optional fixed scorebug coordinates.
 
     Returns:
         Dict with frame detection results.
     """
     # Detect scorebug (fast path with fixed region)
+    # In fixed region mode, this does template matching to verify scorebug presence
     scorebug = scorebug_detector.detect(img)
 
+    # For normal play detection, assume scorebug is present at fixed location
+    # (don't gate normal detection on template matching - only use for FLAG validation)
+    scorebug_assumed = fixed_scorebug_coords is not None
+    scorebug_bbox = fixed_scorebug_coords if scorebug_assumed else (scorebug.bbox if scorebug.detected else None)
+
     # Initialize frame result using shared factory
+    # scorebug_detected = True for fixed region mode (for backward compatibility)
+    # scorebug_verified = actual template match result (for FLAG validation)
     frame_result = create_frame_result(
         timestamp=timestamp,
-        scorebug_detected=scorebug.detected,
-        scorebug_bbox=scorebug.bbox if scorebug.detected else None,
+        scorebug_detected=scorebug_assumed or scorebug.detected,
+        scorebug_bbox=scorebug_bbox,
     )
+    # Track actual template verification for FLAG validation
+    frame_result["scorebug_verified"] = scorebug.detected
 
-    if scorebug.detected:
+    if scorebug_assumed or scorebug.detected:
         stats["frames_with_scorebug"] += 1
 
         # Read timeout indicators if available
@@ -154,8 +167,9 @@ def _process_frame(
             frame_result["timeout_confidence"] = timeout_reading.confidence
 
         # Read FLAG indicator if reader is configured
-        if flag_reader:
-            flag_reading = flag_reader.read(img, scorebug.bbox)
+        # Always read FLAG (will be validated later using scorebug_verified)
+        if flag_reader and scorebug_bbox:
+            flag_reading = flag_reader.read(img, scorebug_bbox)
             frame_result["flag_detected"] = flag_reading.detected
             frame_result["flag_yellow_ratio"] = flag_reading.yellow_ratio
             frame_result["flag_mean_hue"] = flag_reading.mean_hue
@@ -168,9 +182,9 @@ def _process_frame(
             frame_result["clock_value"] = clock_result.value
             if clock_result.detected:
                 stats["frames_with_clock"] += 1
-        elif template_reader:
+        elif template_reader and scorebug_bbox:
             # Fallback: extract region then match (for non-fixed-coords mode)
-            play_clock_region = clock_reader.extract_region(img, scorebug.bbox)
+            play_clock_region = clock_reader.extract_region(img, scorebug_bbox)
             if play_clock_region is not None:
                 clock_result = template_reader.read(play_clock_region)
                 frame_result["clock_detected"] = clock_result.detected
@@ -190,10 +204,11 @@ def _process_chunk(
     progress_dict: Optional[MutableMapping[int, Any]] = None,
 ) -> ChunkResult:
     """
-    Process a single video chunk using OpenCV.
+    Process a single video chunk using FFmpeg pipe for accurate VFR handling.
 
     This function runs in a separate process and must be self-contained.
-    It opens its own video file handle and creates its own detector instances.
+    It uses FFmpeg for frame extraction which correctly handles Variable Frame Rate
+    videos where OpenCV seeking would return frames out of chronological order.
 
     Args:
         chunk_id: Identifier for this chunk (for logging).
@@ -206,28 +221,15 @@ def _process_chunk(
         ChunkResult with processing results.
     """
     # pylint: disable=import-outside-toplevel
-    # cv2 import must be inside function for multiprocessing - each subprocess
-    # needs its own fresh import to avoid issues with OpenCV's internal state
-    import cv2
+    # Import must be inside function for multiprocessing - each subprocess
+    # needs its own fresh imports to avoid pickling errors
+    from video.ffmpeg_reader import FFmpegFrameReader
 
     t_start = time.perf_counter()
 
     # Initialize all detection components
     scorebug_detector, clock_reader, template_reader, timeout_tracker, flag_reader = _init_chunk_detectors(config)
 
-    # Open video and seek to start
-    t_io_start = time.perf_counter()
-    cap = cv2.VideoCapture(config.video_path)
-    if not cap.isOpened():
-        raise RuntimeError(f"Could not open video: {config.video_path}")
-
-    fps = cap.get(cv2.CAP_PROP_FPS)
-    frame_skip = int(config.frame_interval * fps)
-    start_frame = int(chunk_start * fps)
-    end_frame = int(chunk_end * fps)
-    cap.set(cv2.CAP_PROP_POS_FRAMES, start_frame)
-    io_time = time.perf_counter() - t_io_start
-
     # Initialize processing state
     frame_data: List[Dict[str, Any]] = []
     stats = {"total_frames": 0, "frames_with_scorebug": 0, "frames_with_clock": 0}
@@ -237,45 +239,32 @@ def _process_chunk(
     if progress_dict is not None:
         progress_dict[chunk_id] = {"frames": 0, "total": total_expected_frames, "status": "running"}
 
-    # Process frames in chunk
-    current_frame = start_frame
-    while current_frame < end_frame:
-        # Read frame with I/O timing
-        t_io_start = time.perf_counter()
-        ret, img = cap.read()
-        io_time += time.perf_counter() - t_io_start
-
-        if not ret:
-            break
-
-        # Process this frame
-        timestamp = current_frame / fps
-        stats["total_frames"] += 1
-        frame_result = _process_frame(
-            img,
-            timestamp,
-            scorebug_detector,
-            clock_reader,
-            template_reader,
-            timeout_tracker,
-            flag_reader,
-            stats,
-            fixed_playclock_coords=config.fixed_playclock_coords,
-        )
-        frame_data.append(frame_result)
-
-        # Update progress
-        if progress_dict is not None:
-            progress_dict[chunk_id] = {"frames": stats["total_frames"], "total": total_expected_frames, "status": "running"}
+    # Use FFmpeg pipe for accurate timestamp handling (handles VFR videos correctly)
+    # This is ~36x faster than OpenCV seeking and produces correct frame order
+    with FFmpegFrameReader(config.video_path, chunk_start, chunk_end, config.frame_interval) as reader:
+        for timestamp, img in reader:
+            # Process this frame
+            stats["total_frames"] += 1
+            frame_result = _process_frame(
+                img,
+                timestamp,
+                scorebug_detector,
+                clock_reader,
+                template_reader,
+                timeout_tracker,
+                flag_reader,
+                stats,
+                fixed_playclock_coords=config.fixed_playclock_coords,
+                fixed_scorebug_coords=config.fixed_scorebug_coords,
+            )
+            frame_data.append(frame_result)
 
-        # Skip to next sample frame
-        t_io_start = time.perf_counter()
-        for _ in range(frame_skip - 1):
-            cap.grab()
-        io_time += time.perf_counter() - t_io_start
-        current_frame += frame_skip
+            # Update progress
+            if progress_dict is not None:
+                progress_dict[chunk_id] = {"frames": stats["total_frames"], "total": total_expected_frames, "status": "running"}
 
-    cap.release()
+        # Get I/O timing from reader
+        _, io_time = reader.get_stats()
 
     # Mark chunk as complete
     if progress_dict is not None:

src/pipeline/play_extractor.py

@@ -738,6 +738,7 @@ class PlayExtractor:
             fixed_scorebug_coords=self.config.fixed_scorebug_coords,
             template_library_path=str(template_path) if template_path else None,
             timeout_config_path=timeout_config_path,
+            scorebug_template_path=self.config.template_path,  # For scorebug verification during FLAG detection
             flag_x_offset=flag_x_offset,
             flag_y_offset=flag_y_offset,
             flag_width=flag_width,
@@ -790,12 +791,14 @@ class PlayExtractor:
                     confidence=frame.get("timeout_confidence", 0.0),
                 )
             # Create FLAG info for penalty flag tracking
+            # scorebug_verified is used to filter false positives during replays/commercials
             flag_info = None
             if frame.get("flag_detected") is not None:
                 flag_info = FlagInfo(
                     detected=frame.get("flag_detected", False),
                     yellow_ratio=frame.get("flag_yellow_ratio", 0.0),
                     mean_hue=frame.get("flag_mean_hue", 0.0),
+                    scorebug_verified=frame.get("scorebug_verified", True),
                 )
             self.state_machine.update(frame["timestamp"], scorebug, clock_reading, timeout_info, flag_info)
         timing["state_machine"] = time.perf_counter() - t_sm_start

src/pipeline/template_builder_pass.py

@@ -127,6 +127,8 @@ def _scan_video(
     accounts for rendering changes that occur during broadcasts (different score
     states, different lighting, etc.).
 
+    Uses FFmpeg for frame extraction to correctly handle VFR videos.
+
     Args:
         config: Detection configuration
         clock_reader: Play clock region extractor
@@ -140,7 +142,9 @@ def _scan_video(
     Returns:
         Tuple of (valid_samples, frames_scanned, frames_with_scorebug)
     """
-    # Open video
+    from video.ffmpeg_reader import FFmpegFrameReader, get_video_dimensions
+
+    # Get video duration using OpenCV (quick metadata read)
     cap = cv2.VideoCapture(config.video_path)
     if not cap.isOpened():
         logger.error("Pass 0: Could not open video")
@@ -149,6 +153,7 @@ def _scan_video(
     fps = cap.get(cv2.CAP_PROP_FPS)
     total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
     video_duration = total_frames / fps
+    cap.release()
 
     # Determine effective range to scan (respect start_time/end_time if set)
     effective_start = config.start_time if config.start_time else 0.0
@@ -166,7 +171,8 @@ def _scan_video(
 
     # Frames to scan per sample point
     frames_per_point = max_scan_frames // len(sample_points)
-    frame_skip = int(config.frame_interval * fps)
+    # Duration to scan per point based on frame interval
+    duration_per_point = frames_per_point * config.frame_interval
 
     logger.info("  Multi-point template building enabled (4 sample points)")
     logger.info("  Sample points: %s", [f"{t:.0f}s" for t in sample_points])
@@ -176,25 +182,16 @@ def _scan_video(
     frames_scanned = 0
     frames_with_scorebug = 0
 
-    try:
-        for point_idx, start_time in enumerate(sample_points):
-            # Calculate start frame for this sample point
-            start_frame = int(start_time * fps)
-            cap.set(cv2.CAP_PROP_POS_FRAMES, start_frame)
+    for point_idx, start_time in enumerate(sample_points):
+        point_frames_scanned = 0
+        point_valid_samples = 0
+        end_time_point = min(start_time + duration_per_point, effective_end)
 
-            point_frames_scanned = 0
-            point_valid_samples = 0
-
-            logger.info("  Scanning from %.0fs (point %d/%d)...", start_time, point_idx + 1, len(sample_points))
-
-            while point_frames_scanned < frames_per_point:
-                ret, frame = cap.read()
-                if not ret:
-                    break
-
-                current_frame = int(cap.get(cv2.CAP_PROP_POS_FRAMES)) - 1
-                current_time = current_frame / fps
+        logger.info("  Scanning from %.0fs (point %d/%d)...", start_time, point_idx + 1, len(sample_points))
 
+        # Use FFmpeg pipe for accurate VFR handling
+        with FFmpegFrameReader(config.video_path, start_time, end_time_point, config.frame_interval) as frame_reader:
+            for current_time, frame in frame_reader:
                 frames_scanned += 1
                 point_frames_scanned += 1
 
@@ -224,14 +221,11 @@ def _scan_video(
                         logger.info("  Completion criteria met!")
                         return valid_samples, frames_scanned, frames_with_scorebug
 
-                # Skip frames
-                for _ in range(frame_skip - 1):
-                    cap.grab()
-
-            logger.info("  Point %d complete: %d samples from %d frames", point_idx + 1, point_valid_samples, point_frames_scanned)
+                # Stop if we've scanned enough frames from this point
+                if point_frames_scanned >= frames_per_point:
+                    break
 
-    finally:
-        cap.release()
+        logger.info("  Point %d complete: %d samples from %d frames", point_idx + 1, point_valid_samples, point_frames_scanned)
 
     return valid_samples, frames_scanned, frames_with_scorebug
 

src/tracking/flag_tracker.py

@@ -36,15 +36,29 @@ class FlagEventData:
     frame_count: int = 0
     yellow_sum: float = 0.0
     hue_sum: float = 0.0
+    scorebug_frames: int = 0  # Frames where scorebug was detected
 
-    def update(self, yellow_ratio: float, mean_hue: float) -> None:
-        """Update running statistics with a new frame."""
+    def update(self, yellow_ratio: float, mean_hue: float, scorebug_verified: bool = True) -> None:
+        """Update running statistics with a new frame.
+
+        Args:
+            yellow_ratio: Yellow pixel ratio for this frame
+            mean_hue: Mean hue of yellow pixels
+            scorebug_verified: Whether scorebug was verified present via template matching
+        """
         self.frame_count += 1
         self.yellow_sum += yellow_ratio
         self.hue_sum += mean_hue
         self.peak_yellow_ratio = max(self.peak_yellow_ratio, yellow_ratio)
         self.avg_yellow_ratio = self.yellow_sum / self.frame_count
         self.avg_hue = self.hue_sum / self.frame_count
+        if scorebug_verified:
+            self.scorebug_frames += 1
+
+    @property
+    def scorebug_ratio(self) -> float:
+        """Ratio of frames where scorebug was detected."""
+        return self.scorebug_frames / self.frame_count if self.frame_count > 0 else 0.0
 
 
 @dataclass
@@ -93,9 +107,10 @@ class FlagTracker:
     GAP_TOLERANCE = 12.0  # Maximum gap (seconds) between FLAG sightings to consider same event
     MIN_PEAK_YELLOW = 0.70  # Real FLAGs peak at 80%+, false positives at 30-40%
     MIN_AVG_YELLOW = 0.60  # Real FLAGs average 70%+, false positives much lower
-    # Hue filtering: Real FLAG yellow has hue ~28-29, orange ~16-17, other graphics ~24-25
-    MIN_MEAN_HUE = 28.0  # Ground truth FLAGs have hue >= 28
+    # Hue filtering: Real FLAG yellow has hue ~26-30, orange ~16-17, other graphics ~24-25
+    MIN_MEAN_HUE = 25.0  # Ground truth FLAGs have hue >= 25 (lowered from 28)
     MAX_MEAN_HUE = 31.0  # Reject lime-green graphics (hue > 31)
+    MIN_SCOREBUG_RATIO = 0.50  # Require scorebug in at least 50% of FLAG frames (filters replays/commercials)
 
     def __init__(
         self,
@@ -105,6 +120,7 @@ class FlagTracker:
         min_avg_yellow: float = MIN_AVG_YELLOW,
         min_mean_hue: float = MIN_MEAN_HUE,
         max_mean_hue: float = MAX_MEAN_HUE,
+        min_scorebug_ratio: float = MIN_SCOREBUG_RATIO,
     ):
         """
         Initialize the FLAG tracker.
@@ -116,6 +132,7 @@ class FlagTracker:
             min_avg_yellow: Minimum average yellow ratio required
             min_mean_hue: Minimum mean hue required (rejects orange/other yellow graphics)
             max_mean_hue: Maximum mean hue allowed (rejects lime-green graphics)
+            min_scorebug_ratio: Minimum ratio of frames with scorebug present (filters replays/commercials)
         """
         self.min_flag_duration = min_flag_duration
         self.gap_tolerance = gap_tolerance
@@ -123,6 +140,7 @@ class FlagTracker:
         self.min_avg_yellow = min_avg_yellow
         self.min_mean_hue = min_mean_hue
         self.max_mean_hue = max_mean_hue
+        self.min_scorebug_ratio = min_scorebug_ratio
         self._state = FlagTrackerState()
         self._play_count = 0  # Running count for play numbering
         self._last_flag_seen_at: Optional[float] = None  # For gap tolerance
@@ -177,7 +195,7 @@ class FlagTracker:
         # No scorebug (e.g., replay) - use gap tolerance
         return self._handle_no_scorebug(timestamp)
 
-    def _handle_flag_detected(self, timestamp: float, flag_info: FlagInfo) -> Optional[PlayEvent]: # pylint: disable=useless-return
+    def _handle_flag_detected(self, timestamp: float, flag_info: FlagInfo) -> Optional[PlayEvent]:  # pylint: disable=useless-return
         """Handle FLAG being detected."""
         self._last_flag_seen_at = timestamp
 
@@ -193,9 +211,9 @@ class FlagTracker:
                 flag_info.mean_hue,
             )
 
-        # Update current FLAG statistics
+        # Update current FLAG statistics (including scorebug verification from FlagInfo)
         if self._state.current_flag is not None:
-            self._state.current_flag.update(flag_info.yellow_ratio, flag_info.mean_hue)
+            self._state.current_flag.update(flag_info.yellow_ratio, flag_info.mean_hue, flag_info.scorebug_verified)
 
         return None
 
@@ -254,13 +272,16 @@ class FlagTracker:
         # Store completed flag data
         self._state.completed_flags.append(self._state.current_flag)
 
+        scorebug_ratio = self._state.current_flag.scorebug_ratio
+
         logger.info(
-            "FLAG EVENT ended at %.1fs (duration=%.1fs, peak=%.0f%%, avg=%.0f%%, hue=%.1f)",
+            "FLAG EVENT ended at %.1fs (duration=%.1fs, peak=%.0f%%, avg=%.0f%%, hue=%.1f, scorebug=%.0f%%)",
             self._state.current_flag.end_time,
             duration,
             peak_yellow * 100,
             avg_yellow * 100,
             avg_hue,
+            scorebug_ratio * 100,
         )
 
         # Check if FLAG event meets all criteria to become a FLAG PLAY
@@ -282,6 +303,9 @@ class FlagTracker:
         if avg_hue > self.max_mean_hue:
             reject_reasons.append(f"hue {avg_hue:.1f} > {self.max_mean_hue:.1f} (lime-green, not yellow)")
 
+        if scorebug_ratio < self.min_scorebug_ratio:
+            reject_reasons.append(f"scorebug {scorebug_ratio:.0%} < {self.min_scorebug_ratio:.0%} (likely replay/commercial)")
+
         if reject_reasons:
             logger.debug(
                 "FLAG event rejected: %s",

src/tracking/models.py

@@ -128,6 +128,7 @@ class FlagInfo(BaseModel):
     detected: bool = Field(False, description="Whether FLAG is detected (yellow present AND valid hue)")
     yellow_ratio: float = Field(0.0, description="Ratio of yellow pixels in FLAG region")
     mean_hue: float = Field(0.0, description="Mean hue of yellow pixels (helps distinguish orange)")
+    scorebug_verified: bool = Field(True, description="Whether scorebug was verified present via template matching")
     is_valid_yellow: bool = Field(False, description="True if mean_hue >= threshold (not orange)")
 
 

src/video/__init__.py

@@ -2,6 +2,7 @@
 
 from .frame_extractor import extract_sample_frames, get_video_duration
 from .frame_reader import ThreadedFrameReader
+from .ffmpeg_reader import FFmpegFrameReader, extract_frames_ffmpeg_pipe, iter_frames_ffmpeg, get_video_dimensions
 from .ffmpeg_ops import (
     extract_clip_stream_copy,
     extract_clip_reencode,
@@ -14,6 +15,10 @@ __all__ = [
     "extract_sample_frames",
     "get_video_duration",
     "ThreadedFrameReader",
+    "FFmpegFrameReader",
+    "extract_frames_ffmpeg_pipe",
+    "iter_frames_ffmpeg",
+    "get_video_dimensions",
     "extract_clip_stream_copy",
     "extract_clip_reencode",
     "concatenate_clips",

src/video/ffmpeg_reader.py

@@ -0,0 +1,334 @@
+"""
+FFmpeg-based frame reader for accurate VFR (Variable Frame Rate) video handling.
+
+This module provides frame extraction using FFmpeg's accurate timestamp seeking,
+which correctly handles VFR videos where OpenCV's seeking fails.
+
+Key advantages over OpenCV seeking:
+- Accurate timestamp handling for VFR videos
+- ~36x faster than OpenCV's CAP_PROP_POS_FRAMES seeking
+- Frames are returned in correct chronological order
+"""
+
+import logging
+import subprocess
+from typing import Any, Callable, Generator, Optional, Tuple
+
+import cv2
+import numpy as np
+
+logger = logging.getLogger(__name__)
+
+
+def get_video_dimensions(video_path: str) -> Tuple[int, int]:
+    """
+    Get video dimensions (width, height) using OpenCV.
+
+    Args:
+        video_path: Path to video file.
+
+    Returns:
+        Tuple of (width, height).
+
+    Raises:
+        ValueError: If video cannot be opened.
+    """
+    cap = cv2.VideoCapture(video_path)
+    if not cap.isOpened():
+        raise ValueError(f"Could not open video: {video_path}")
+
+    width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+    cap.release()
+
+    return width, height
+
+
+def extract_frames_ffmpeg_pipe(
+    video_path: str,
+    start_time: float,
+    end_time: float,
+    frame_interval: float,
+    callback: Callable[[float, np.ndarray[Any, Any]], bool],
+) -> Tuple[int, float]:
+    """
+    Extract frames using FFmpeg pipe for accurate VFR handling.
+
+    FFmpeg seeks accurately to the start position and outputs frames at the
+    specified interval. Frames are piped directly to Python as raw BGR data,
+    avoiding temp files and providing accurate timestamps.
+
+    Args:
+        video_path: Path to video file.
+        start_time: Start time in seconds.
+        end_time: End time in seconds.
+        frame_interval: Interval between frames in seconds (e.g., 0.5 for 2 fps).
+        callback: Function called for each frame.
+                  Signature: callback(timestamp: float, frame: np.ndarray) -> bool
+                  Return False to stop processing early.
+
+    Returns:
+        Tuple of (frames_processed, io_time).
+    """
+    import time
+
+    # Get video dimensions
+    width, height = get_video_dimensions(video_path)
+    frame_size = width * height * 3  # BGR format
+
+    # Calculate output fps from interval
+    output_fps = 1.0 / frame_interval
+    duration = end_time - start_time
+
+    t_io_start = time.perf_counter()
+
+    # Build ffmpeg command
+    # -ss before -i enables fast seeking to keyframe, then accurate frame output
+    cmd = [
+        "ffmpeg",
+        "-ss",
+        str(start_time),
+        "-i",
+        str(video_path),
+        "-t",
+        str(duration),
+        "-vf",
+        f"fps={output_fps}",  # Output at specified fps
+        "-f",
+        "rawvideo",
+        "-pix_fmt",
+        "bgr24",  # OpenCV uses BGR format
+        "-loglevel",
+        "error",
+        "-",  # Output to stdout
+    ]
+
+    # Start ffmpeg process
+    process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+    frames_processed = 0
+    current_time = start_time
+
+    try:
+        while True:
+            # Read raw frame data from stdout
+            raw_frame = process.stdout.read(frame_size)
+
+            # Check for end of stream
+            if len(raw_frame) != frame_size:
+                break
+
+            # Convert to numpy array (BGR format, same as OpenCV)
+            frame = np.frombuffer(raw_frame, dtype=np.uint8).reshape((height, width, 3))
+
+            # Call the callback with timestamp and frame
+            # Make a copy to ensure the frame data is not overwritten
+            continue_processing = callback(current_time, frame.copy())
+            frames_processed += 1
+
+            if not continue_processing:
+                break
+
+            current_time += frame_interval
+
+    finally:
+        # Clean up process
+        process.stdout.close()
+        process.stderr.close()
+        process.terminate()
+        process.wait()
+
+    io_time = time.perf_counter() - t_io_start
+
+    return frames_processed, io_time
+
+
+def iter_frames_ffmpeg(
+    video_path: str,
+    start_time: float,
+    end_time: float,
+    frame_interval: float,
+) -> Generator[Tuple[float, np.ndarray[Any, Any]], None, None]:
+    """
+    Generator that yields frames using FFmpeg pipe.
+
+    This is an alternative interface for iterating over frames without a callback.
+
+    Args:
+        video_path: Path to video file.
+        start_time: Start time in seconds.
+        end_time: End time in seconds.
+        frame_interval: Interval between frames in seconds.
+
+    Yields:
+        Tuple of (timestamp, frame) for each frame.
+    """
+    import time
+
+    # Get video dimensions
+    width, height = get_video_dimensions(video_path)
+    frame_size = width * height * 3
+
+    # Calculate output fps from interval
+    output_fps = 1.0 / frame_interval
+    duration = end_time - start_time
+
+    # Build ffmpeg command
+    cmd = [
+        "ffmpeg",
+        "-ss",
+        str(start_time),
+        "-i",
+        str(video_path),
+        "-t",
+        str(duration),
+        "-vf",
+        f"fps={output_fps}",
+        "-f",
+        "rawvideo",
+        "-pix_fmt",
+        "bgr24",
+        "-loglevel",
+        "error",
+        "-",
+    ]
+
+    process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    current_time = start_time
+
+    try:
+        while True:
+            raw_frame = process.stdout.read(frame_size)
+            if len(raw_frame) != frame_size:
+                break
+
+            frame = np.frombuffer(raw_frame, dtype=np.uint8).reshape((height, width, 3))
+            yield current_time, frame.copy()
+            current_time += frame_interval
+
+    finally:
+        process.stdout.close()
+        process.stderr.close()
+        process.terminate()
+        process.wait()
+
+
+class FFmpegFrameReader:
+    """
+    Context manager for reading frames from video using FFmpeg pipe.
+
+    This class provides a cleaner interface for reading frames in a processing loop,
+    handling resource cleanup automatically.
+
+    Example:
+        with FFmpegFrameReader(video_path, start, end, interval) as reader:
+            for timestamp, frame in reader:
+                process_frame(timestamp, frame)
+    """
+
+    def __init__(self, video_path: str, start_time: float, end_time: float, frame_interval: float):
+        """
+        Initialize the FFmpeg frame reader.
+
+        Args:
+            video_path: Path to video file.
+            start_time: Start time in seconds.
+            end_time: End time in seconds.
+            frame_interval: Interval between frames in seconds.
+        """
+        self.video_path = video_path
+        self.start_time = start_time
+        self.end_time = end_time
+        self.frame_interval = frame_interval
+
+        self.process: Optional[subprocess.Popen[bytes]] = None
+        self.width = 0
+        self.height = 0
+        self.frame_size = 0
+        self.current_time = start_time
+        self.frames_read = 0
+        self.io_time = 0.0
+
+    def __enter__(self) -> "FFmpegFrameReader":
+        """Start the FFmpeg process."""
+        import time
+
+        # Get video dimensions
+        self.width, self.height = get_video_dimensions(self.video_path)
+        self.frame_size = self.width * self.height * 3
+
+        # Calculate parameters
+        output_fps = 1.0 / self.frame_interval
+        duration = self.end_time - self.start_time
+
+        # Build and start ffmpeg command
+        cmd = [
+            "ffmpeg",
+            "-ss",
+            str(self.start_time),
+            "-i",
+            str(self.video_path),
+            "-t",
+            str(duration),
+            "-vf",
+            f"fps={output_fps}",
+            "-f",
+            "rawvideo",
+            "-pix_fmt",
+            "bgr24",
+            "-loglevel",
+            "error",
+            "-",
+        ]
+
+        t_start = time.perf_counter()
+        self.process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        self.io_time = time.perf_counter() - t_start
+
+        self.current_time = self.start_time
+        self.frames_read = 0
+
+        return self
+
+    def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
+        """Clean up the FFmpeg process."""
+        if self.process:
+            self.process.stdout.close()
+            self.process.stderr.close()
+            self.process.terminate()
+            self.process.wait()
+
+    def __iter__(self) -> "FFmpegFrameReader":
+        """Return self as iterator."""
+        return self
+
+    def __next__(self) -> Tuple[float, np.ndarray[Any, Any]]:
+        """Read and return the next frame."""
+        import time
+
+        if self.process is None:
+            raise StopIteration
+
+        t_start = time.perf_counter()
+        raw_frame = self.process.stdout.read(self.frame_size)
+        self.io_time += time.perf_counter() - t_start
+
+        if len(raw_frame) != self.frame_size:
+            raise StopIteration
+
+        frame = np.frombuffer(raw_frame, dtype=np.uint8).reshape((self.height, self.width, 3))
+        timestamp = self.current_time
+
+        self.current_time += self.frame_interval
+        self.frames_read += 1
+
+        return timestamp, frame.copy()
+
+    def get_stats(self) -> Tuple[int, float]:
+        """
+        Get reading statistics.
+
+        Returns:
+            Tuple of (frames_read, io_time).
+        """
+        return self.frames_read, self.io_time