docs/Fomula.md

Step 1: Pre-processing & Scale Stabilization (Executes at 30 FPS)

Capture local eye landmark coordinates using an ultra-lightweight geometric extractor (e.g., MediaPipe Face Mesh):

$${\zeta }_t=\{P_{inner},P_{outer},P_{pupil},P_{eyelid}\}\in R^{4\times 2}\backslash zeta\_\{t\}=\backslash \{P\_\{inner\},P\_\{outer\},P\_\{pupil\},P\_\{eyelid\}\backslash \}\backslash in\; R^\{4\backslash times2\}$$

Apply an adaptive affine scaling matrix $\mathcal{T}_{scale}$ to normalize distance/scale variations:

$$V_t^{norm}={\mathcal{T}}_{scale}(V_t)V\_\{t\}^\{norm\}=\backslash mathcal\{T\}\_\{scale\}(V\_\{t\})$$

---

Step 2: Asynchronous Multi-rate Branch Routing

The system splits operations based on the current frame index $t$:

CASE I: State A (Active Appearance Frame) — Triggered at $t \pmod 3 = 0$ (10 FPS)

* Landmark-Guided Patch Division: Isolate the eye socket and extract $K=4$ sub-patches $I_{k}\in R^{8\times8\times1}$ using the localized pupil center $P_{pupil}$.

* Mini Conv-Embedder: Pass each patch through a shallow 3-layer CNN to obtain appearance tokens:

$$f_k^{app}=\text{Flatten}(\text{MaxPool}(\text{Conv2D}(I_k)))\in R^{D}f\_\{k\}^\{app\}=\backslash text\{Flatten\}(\backslash text\{MaxPool\}(\backslash text\{Conv2D\}(I\_\{k\})))\backslash in\; R^\{D\}$$

* Token-pair Feature Injection: Project appearance and geometric markers onto a shared latent space, then concatenate into joint structural-appearance tokens:

$$T_k=[{\mathcal{W}}_{app}{f}_k^{app}\mathrm{\mid }\mathrm{\mid }{\mathcal{W}}_{geo}{P}_k]T\_\{k\}=[\backslash mathcal\{W\}\_\{app\}f\_\{k\}^\{app\}||\backslash mathcal\{W\}\_\{geo\}P\_\{k\}]$$

* Fine Regressor: Feed $T=\text{Concat}(T_{1},...,T_{K})$ into a flat MLP block to infer fine-grained gaze vectors:

$$G_t^{fine}=({\theta }_{fine},{\varphi }_{fine})G\_\{t\}^\{fine\}=(\backslash theta\_\{fine\},\backslash phi\_\{fine\})$$

CASE II: State B (Missing Appearance Frame) — Triggered at $t \pmod 3 \neq 0$ (20 FPS)

* CPU Duty Cycle Optimization: Power down the entire CNN inference branch and freeze memory image I/O operations.

* Coarse Regressor: Pass the normalized geometric vector $V_{t}^{norm}$ directly through an independent lightweight MLP:

$$G_t^{coarse}=\mathcal{M}\mathcal{L}{\mathcal{P}}_{coarse}(V_t^{norm})G\_\{t\}^\{coarse\}=\backslash mathcal\{M\}\backslash mathcal\{L\}\backslash mathcal\{P\}\_\{coarse\}(V\_\{t\}^\{norm\})$$

* Gaze Interpolation: Compute the final coordinates by dynamically blending the current coarse prediction with the cached fine prediction from the last active frame:

$$G_t=\alpha \cdot G_t^{coarse}+(1-\alpha )\cdot G_{t-1}^{fine}G\_\{t\}=\backslash alpha\backslash cdot\; G\_\{t\}^\{coarse\}+(1-\backslash alpha)\backslash cdot\; G\_\{t-1\}^\{fine\}$$

(where $\alpha \in [0, 1]$ is dynamically tuned based on eye landmark movement velocity ).

---

Step 3: Lightweight Temporal Fusion (Post-Processing)

Pass the output vector $G_t$ through an Exponential Moving Average (EMA) or a One Dollar Filter to suppress tracking jitter and prevent spatial lag across state-switching intervals:

$$G_{smooth}=\beta \cdot G_t+(1-\beta )\cdot G_{smooth,t-1}G\_\{smooth\}\; =\; \backslash beta\; \backslash cdot\; G\_t\; +\; (1\; -\; \backslash beta)\; \backslash cdot\; G\_\{smooth,\; t-1\}$$

---

4. Hardware and Deployment Optimization Roadmap

A. Training & Knowledge Distillation (KD) Strategy

• Hardware Configuration: NVIDIA GeForce RTX 3060 Laptop GPU.
• Framework: PyTorch 2.x with CUDA acceleration.
• Knowledge Distillation Pipeline:
• Teacher Network: SOTA Appearance-based model (e.g., L2CS-Net with a ResNet-50 backbone) operating on high-resolution $224 \times 224$ inputs.
• Student Network: LIPE V2 shallow MLP architecture.
• Mechanism: Train LIPE V2 to replicate the soft output distributions and logit mappings of L2CS-Net, retaining deep geometric representations inside a highly compact model.