| # Phase 4 Master Plan: Post-Processing & Integration |
|
|
| ## 1. Executive Summary |
| Phase 4 focuses on converting the raw model outputs into a stable, high-frequency gaze stream. We implement the **Dual-State Asymmetric Pipeline** logic, integrate temporal filtering to eliminate jitter, and perform hardware profiling to ensure the system meets its efficiency targets. |
|
|
| ## 2. Technical Implementation |
|
|
| ### A. Dual-State Blending Logic |
| The system maintains a high frame rate (30+ FPS) by alternating between appearance-heavy and geometry-only processing: |
| * **State A (Active Appearance):** Executed every $t \pmod 3 = 0$. Processes patches and landmarks. |
| * **State B (Missing Appearance):** Executed when $t \pmod 3 \neq 0$. Bypasses the CNN branch. |
| * **Blending Formula:** $G_t = \alpha \cdot G_t^{geo} + (1-\alpha) \cdot G_{cached}^{app}$ (Current $\alpha=0.7$). |
| |
| ### B. Temporal Stability (One Euro Filter) |
| * **Mechanism:** Adaptive low-pass filtering. |
| * **Parameters:** `min_cutoff=0.1`, `beta=0.01` (Tuned for high sensitivity at rest and low lag during movement). |
|
|
| ### C. Resource Profiling |
| * **Target:** < 0.12 GFLOPs, < 45 MB RAM. |
| * **Current Performance:** ~1400 FPS on CPU (P95 Latency < 1.5ms). |
|
|
| ## 3. Execution Roadmap |
|
|
| ### Step 1: Inference Pipeline Development |
| - [x] Implement `src/inference_pipeline.py`. |
| - [x] Integrate `OneEuroFilter` for pitch and yaw. |
| - [x] Implement asymmetric state switching logic. |
|
|
| ### Step 2: Benchmarking & Optimization |
| - [x] **Latency Profiling:** Measure CPU execution time for State A vs State B. |
| - [x] **Throughput Testing:** Verify FPS under simulated real-time conditions. |
| - [ ] **Memory Audit:** Measure peak RAM usage during inference. |
|
|
| ### Step 3: Final Demo Integration |
| - [ ] Connect `DualStatePipeline` to `demo_integration.py`. |
| - [ ] Add visualization overlay for "State A" vs "State B" indicators. |
| - [ ] Implement a toggle for "Filter ON/OFF" to demonstrate jitter reduction. |
|
|
| ### Step 4: Final Validation |
| - [ ] Run end-to-end demo on live webcam or sample video. |
| - [ ] Verify angular error on a held-out verification set using the full pipeline. |
|
|
