# LIPE V2: SYSTEM WORKFLOW SPECIFICATION (INTEGRATED-HARDWARE COMPLIANT)

## 1. System Overview & Target Benchmarks
* **System Name:** LIPE V2 (Landmark-guided Image Patch Embedder V2).
* **Domain:** Driver Monitoring Systems (DMS) / Real-time Edge Gaze Estimation.
* **Core Philosophy:** Input-driven model compression via asymmetric spatio-temporal modeling. Optimized specifically for commodity host systems lacking discrete deployment accelerators.
* **Target Optimization Pareto Frontier (Integrated Hardware):**
  * Computational Cost: $< 0.12$ GFLOPs.
  * RAM Consumption: $< 45$ MB (Well within the 16GB system boundary).
  * Inference Speed: $\ge 30$ FPS running purely on host CPU / Integrated GPU (iGPU).
  * Accuracy: Mean 3D Angular Error $\mathcal{E}_{angular} < 4.2^{\circ}$.

---

## 2. Mathematical Framework & Optimization Core
* **Asymmetric Input Resolution:** Image domain is restricted to $K=4$ localized patches of size $8 \times 8 \times 1$ pixels centered around the pupil coordinate $P_{pupil}$.
* **Loss Function:** **Adaptive Wing Loss (AW Loss)** instead of MSE/L1 to resolve optical aliasing under low-resolution conditions:
  $$\mathcal{L}_{AW}(y,\hat{y})=\begin{cases}\omega~ln(1+|\frac{y-\hat{y}}{\epsilon}|^{n}) & \text{if } |y-\hat{y}|<\theta \\ A|y-\hat{y}|-B & \text{if } |y-\hat{y}|\ge\theta\end{cases}$$
  * *High-gradient penalty mechanism:* Exponentially spikes the gradient when errors fall below 1 pixel, forcing the Fine Regressor to converge accurately at the pupil center without requiring dense multi-layer CNN parameters.

---

## 3. End-to-End Operational Pipeline (30 FPS Stream)

```text
[Camera Raw Input (30 FPS)]
         │
         ▼
[MediaPipe Face Mesh] ──> Extract Geometry Vector (V_t) [Executed via CPU/iGPU OpenCL]
         │
         ▼
[Affine Scale Normalization] ──> Standardized Coordinate Vector (V_t^norm)
         │
         ├───► [Condition: t mod 3 == 0] (State A - 10 FPS)
         │         │
         │         ├──► Landmark-Guided Patch Division (4x 8x8 Patches)
         │         ├──► Mini Conv-Embedder (Shallow 2-layer CNN Feature Extraction)
         │         ├──► Token-pair Feature Injection (Geometry + Appearance Concatenation)
         │         └──► Fine Regressor (MLP) ──► G_t^fine
         │
         └───► [Condition: t mod 3 != 0] (State B - 20 FPS)
                   │
                   ├──► Freeze Image Memory I/O (Maintains Low RAM & CPU Overhead)
                   ├──► Coarse Regressor (MLP Using V_t^norm Only) ──► G_t^coarse
                   └──► Gaze Interpolation Stage (Dynamic Blending with G_t-1^fine) ──► G_t
         │
         ▼
[Lightweight Temporal Fusion Filter] (EMA / One Dollar Filter)
         │
         ▼
[Final Gaze Vector Output] ──► Pitch & Yaw (Radians)