docs/Fomula.md

Step 1: Pre-processing & Scale Stabilization (Executes at 30 FPS) 

Capture local eye landmark coordinates using an ultra-lightweight geometric extractor (e.g., MediaPipe Face Mesh):

$$\zeta_{t}=\{P_{inner},P_{outer},P_{pupil},P_{eyelid}\}\in R^{4\times2}$$



Apply an adaptive affine scaling matrix $\mathcal{T}_{scale}$ to normalize distance/scale variations:


$$V_{t}^{norm}=\mathcal{T}_{scale}(V_{t})$$



---

Step 2: Asynchronous Multi-rate Branch Routing 

The system splits operations based on the current frame index $t$:

CASE I: State A (Active Appearance Frame) — Triggered at $t \pmod 3 = 0$ (10 FPS) 

* 
**Landmark-Guided Patch Division:** Isolate the eye socket and extract $K=4$ sub-patches $I_{k}\in R^{8\times8\times1}$ using the localized pupil center $P_{pupil}$.


* 
**Mini Conv-Embedder:** Pass each patch through a shallow 3-layer CNN to obtain appearance tokens:



$$f_{k}^{app}=\text{Flatten}(\text{MaxPool}(\text{Conv2D}(I_{k})))\in R^{D}$$





* 
**Token-pair Feature Injection:** Project appearance and geometric markers onto a shared latent space, then concatenate into joint structural-appearance tokens:



$$T_{k}=[\mathcal{W}_{app}f_{k}^{app}||\mathcal{W}_{geo}P_{k}]$$





* 
**Fine Regressor:** Feed $T=\text{Concat}(T_{1},...,T_{K})$ into a flat MLP block to infer fine-grained gaze vectors:



$$G_{t}^{fine}=(\theta_{fine},\phi_{fine})$$






CASE II: State B (Missing Appearance Frame) — Triggered at $t \pmod 3 \neq 0$ (20 FPS) 

* 
**CPU Duty Cycle Optimization:** Power down the entire CNN inference branch and freeze memory image I/O operations.


* 
**Coarse Regressor:** Pass the normalized geometric vector $V_{t}^{norm}$ directly through an independent lightweight MLP:



$$G_{t}^{coarse}=\mathcal{M}\mathcal{L}\mathcal{P}_{coarse}(V_{t}^{norm})$$





* 
**Gaze Interpolation:** Compute the final coordinates by dynamically blending the current coarse prediction with the cached fine prediction from the last active frame:



$$G_{t}=\alpha\cdot G_{t}^{coarse}+(1-\alpha)\cdot G_{t-1}^{fine}$$






(where $\alpha \in [0, 1]$ is dynamically tuned based on eye landmark movement velocity ).



---

### Step 3: Lightweight Temporal Fusion (Post-Processing)

Pass the output vector $G_t$ through an Exponential Moving Average (EMA) or a One Dollar Filter to suppress tracking jitter and prevent spatial lag across state-switching intervals:


$$G_{smooth} = \beta \cdot G_t + (1 - \beta) \cdot G_{smooth, t-1}$$

---

## 4. Hardware and Deployment Optimization Roadmap

### A. Training & Knowledge Distillation (KD) Strategy

* **Hardware Configuration:** NVIDIA GeForce RTX 3060 Laptop GPU.
* **Framework:** PyTorch 2.x with CUDA acceleration.
* **Knowledge Distillation Pipeline:**
* **Teacher Network:** SOTA Appearance-based model (e.g., L2CS-Net with a ResNet-50 backbone) operating on high-resolution $224 \times 224$ inputs.
* **Student Network:** LIPE V2 shallow MLP architecture.
* **Mechanism:** Train LIPE V2 to replicate the soft output distributions and logit mappings of L2CS-Net, retaining deep geometric representations inside a highly compact model.