# Phase 4 Master Plan: Post-Processing & Integration ## 1. Executive Summary Phase 4 focuses on converting the raw model outputs into a stable, high-frequency gaze stream. We implement the **Dual-State Asymmetric Pipeline** logic, integrate temporal filtering to eliminate jitter, and perform hardware profiling to ensure the system meets its efficiency targets. ## 2. Technical Implementation ### A. Dual-State Blending Logic The system maintains a high frame rate (30+ FPS) by alternating between appearance-heavy and geometry-only processing: * **State A (Active Appearance):** Executed every $t \pmod 3 = 0$. Processes patches and landmarks. * **State B (Missing Appearance):** Executed when $t \pmod 3 \neq 0$. Bypasses the CNN branch. * **Blending Formula:** $G_t = \alpha \cdot G_t^{geo} + (1-\alpha) \cdot G_{cached}^{app}$ (Current $\alpha=0.7$). ### B. Temporal Stability (One Euro Filter) * **Mechanism:** Adaptive low-pass filtering. * **Parameters:** `min_cutoff=0.1`, `beta=0.01` (Tuned for high sensitivity at rest and low lag during movement). ### C. Resource Profiling * **Target:** < 0.12 GFLOPs, < 45 MB RAM. * **Current Performance:** ~1400 FPS on CPU (P95 Latency < 1.5ms). ## 3. Execution Roadmap ### Step 1: Inference Pipeline Development - [x] Implement `src/inference_pipeline.py`. - [x] Integrate `OneEuroFilter` for pitch and yaw. - [x] Implement asymmetric state switching logic. ### Step 2: Benchmarking & Optimization - [x] **Latency Profiling:** Measure CPU execution time for State A vs State B. - [x] **Throughput Testing:** Verify FPS under simulated real-time conditions. - [ ] **Memory Audit:** Measure peak RAM usage during inference. ### Step 3: Final Demo Integration - [ ] Connect `DualStatePipeline` to `demo_integration.py`. - [ ] Add visualization overlay for "State A" vs "State B" indicators. - [ ] Implement a toggle for "Filter ON/OFF" to demonstrate jitter reduction. ### Step 4: Final Validation - [ ] Run end-to-end demo on live webcam or sample video. - [ ] Verify angular error on a held-out verification set using the full pipeline.