docs/model_student.md

LIPE V2 Student Model Specification & Tracking

1. Architecture Overview

The Student model is an asymmetric dual-branch network designed for high-efficiency gaze estimation on CPU/iGPU.

Branch A: Mini Conv-Embedder (Appearance)

• Purpose: Extract appearance features from eye patches.
• Input: (Batch, 4, 8, 8) - 4 grayscale patches of 8x8 pixels.
• Layers:
  • Shared CNN Backbone:
    • Conv2d(1 -> 16, k=3, p=0) + ReLU (Out: 6x6)
    • Conv2d(16 -> 32, k=3, p=0) + ReLU (Out: 4x4)
    • Conv2d(32 -> 64, k=3, p=0) + ReLU (Out: 2x2)
    • Global Average Pooling (GAP) (Out: 64)
  • Flatten & Reshape: $4 \times 64 = 256$ features.

Branch B: Geometric MLP (Coarse)

• Purpose: Extract geometric features from facial landmarks.
• Input: (Batch, 956) - 478 landmarks (x, y) flattened and Zero-Centered.
• Layers:
  • Linear(956 -> 256) + LayerNorm + ReLU + Dropout(0.05)
  • Linear(256 -> 256) + ReLU

Fusion & Regression Heads

• Fusion: Residual Addition (Appearance [256] + Geometry [256] = 256).
• Pitch Head: Linear(256 -> 64) -> ReLU -> Linear(64 -> 1)
• Yaw Head: Linear(256 -> 64) -> ReLU -> Linear(64 -> 1)

---

2. Technical Targets

Metric	Target Value	Current Status
Computational Cost	< 0.12 GFLOPs	~0.001 GFLOPs (Verified)
RAM Usage	< 45 MB	~1.5 MB (Weights only)
Inference Speed	>= 30 FPS (CPU)	TBD (Estimated >> 100 FPS)
Angular Error	< 4.2°	TBD

---

3. Implementation Checklist

• Define LIPEV2Student class in src/models/student.py.
• Implement Mini Conv-Embedder with shared weights.
• Implement Geometric MLP branch.
• Implement forward method with State A/B switching logic (Residual Addition).
• Implement AdaptiveWingLoss in src/models/loss.py.
• Weight initialization (Kaiming/Xavier).
• Verify forward pass with dummy tensors.
• Estimate GFLOPs using thop or manual calculation.

---

4. Input/Output Specs

• Input Patches: torch.Tensor of shape (N, 4, 8, 8), dtype float32, range [0, 1].
• Input Landmarks: torch.Tensor of shape (N, 956), dtype float32, Zero-Centered.
• Output: torch.Tensor of shape (N, 2) representing [Pitch, Yaw] in Radians.