docs/PHASE4_PLAN.md

Phase 4 Master Plan: Post-Processing & Integration

1. Executive Summary

Phase 4 focuses on converting the raw model outputs into a stable, high-frequency gaze stream. We implement the Dual-State Asymmetric Pipeline logic, integrate temporal filtering to eliminate jitter, and perform hardware profiling to ensure the system meets its efficiency targets.

2. Technical Implementation

A. Dual-State Blending Logic

The system maintains a high frame rate (30+ FPS) by alternating between appearance-heavy and geometry-only processing:

• State A (Active Appearance): Executed every $t \pmod 3 = 0$. Processes patches and landmarks.
• State B (Missing Appearance): Executed when $t \pmod 3 \neq 0$. Bypasses the CNN branch.
• Blending Formula: $G_t = \alpha \cdot G_t^{geo} + (1-\alpha) \cdot G_{cached}^{app}$ (Current $\alpha=0.7$).

B. Temporal Stability (One Euro Filter)

• Mechanism: Adaptive low-pass filtering.
• Parameters: min_cutoff=0.1, beta=0.01 (Tuned for high sensitivity at rest and low lag during movement).

C. Resource Profiling

• Target: < 0.12 GFLOPs, < 45 MB RAM.
• Current Performance: ~1400 FPS on CPU (P95 Latency < 1.5ms).

3. Execution Roadmap

Step 1: Inference Pipeline Development

• Implement src/inference_pipeline.py.
• Integrate OneEuroFilter for pitch and yaw.
• Implement asymmetric state switching logic.

Step 2: Benchmarking & Optimization

• Latency Profiling: Measure CPU execution time for State A vs State B.
• Throughput Testing: Verify FPS under simulated real-time conditions.
• Memory Audit: Measure peak RAM usage during inference.

Step 3: Final Demo Integration

• Connect DualStatePipeline to demo_integration.py.
• Add visualization overlay for "State A" vs "State B" indicators.
• Implement a toggle for "Filter ON/OFF" to demonstrate jitter reduction.

Step 4: Final Validation

• Run end-to-end demo on live webcam or sample video.
• Verify angular error on a held-out verification set using the full pipeline.