File size: 2,110 Bytes
a10ba7f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | # Phase 4 Master Plan: Post-Processing & Integration
## 1. Executive Summary
Phase 4 focuses on converting the raw model outputs into a stable, high-frequency gaze stream. We implement the **Dual-State Asymmetric Pipeline** logic, integrate temporal filtering to eliminate jitter, and perform hardware profiling to ensure the system meets its efficiency targets.
## 2. Technical Implementation
### A. Dual-State Blending Logic
The system maintains a high frame rate (30+ FPS) by alternating between appearance-heavy and geometry-only processing:
* **State A (Active Appearance):** Executed every $t \pmod 3 = 0$. Processes patches and landmarks.
* **State B (Missing Appearance):** Executed when $t \pmod 3 \neq 0$. Bypasses the CNN branch.
* **Blending Formula:** $G_t = \alpha \cdot G_t^{geo} + (1-\alpha) \cdot G_{cached}^{app}$ (Current $\alpha=0.7$).
### B. Temporal Stability (One Euro Filter)
* **Mechanism:** Adaptive low-pass filtering.
* **Parameters:** `min_cutoff=0.1`, `beta=0.01` (Tuned for high sensitivity at rest and low lag during movement).
### C. Resource Profiling
* **Target:** < 0.12 GFLOPs, < 45 MB RAM.
* **Current Performance:** ~1400 FPS on CPU (P95 Latency < 1.5ms).
## 3. Execution Roadmap
### Step 1: Inference Pipeline Development
- [x] Implement `src/inference_pipeline.py`.
- [x] Integrate `OneEuroFilter` for pitch and yaw.
- [x] Implement asymmetric state switching logic.
### Step 2: Benchmarking & Optimization
- [x] **Latency Profiling:** Measure CPU execution time for State A vs State B.
- [x] **Throughput Testing:** Verify FPS under simulated real-time conditions.
- [ ] **Memory Audit:** Measure peak RAM usage during inference.
### Step 3: Final Demo Integration
- [ ] Connect `DualStatePipeline` to `demo_integration.py`.
- [ ] Add visualization overlay for "State A" vs "State B" indicators.
- [ ] Implement a toggle for "Filter ON/OFF" to demonstrate jitter reduction.
### Step 4: Final Validation
- [ ] Run end-to-end demo on live webcam or sample video.
- [ ] Verify angular error on a held-out verification set using the full pipeline.
|