docs/model_refinement.md

LIPE V2 Student Model: Refinement & Contingency Plan

1. CNN Bottleneck: Valid Padding Strategy

• Primary Plan: Use padding=0 for all Conv layers.
  • Input: (1, 8, 8)
  • Conv1 (k3, p0): -> (16, 6, 6)
  • Conv2 (k3, p0): -> (32, 4, 4)
  • Conv3 (k3, p0): -> (64, 2, 2)
  • GAP: -> (64,)
• Contingency: If FLOPs are still high, implement Depthwise Separable Convolutions for layers 2 and 3.

2. Fusion Logic: Asymmetric Routing

• Primary Plan (Residual Addition):
  • Geometric MLP output dim = 256 (matches total Appearance tokens).
  • if State A: Combined = Appearance + Geometry
  • if State B: Combined = Geometry
  • Note: This requires nn.Linear in Geometry branch to output 256.
• Contingency (EMA Caching):
  • Maintain a 384-dim Concatenated vector.
  • if State B: Use Appearance_tokens from t-1 (cached/EMA) to avoid shape mismatch and zero-multiplication overhead.

3. Geometric Stability: LayerNorm Integration

• Primary Plan: Replace Dropout(0.1) with nn.LayerNorm(256) after the first hidden layer of the Geometric branch.
• Contingency: If CPU latency increases, switch to nn.utils.weight_norm on Linear layers to stabilize gradients without explicit normalization steps.

4. Summary of Architecture Changes

Component	From (Old Spec)	To (Refined Spec)
CNN Padding	p=1 (Same)	p=0 (Valid)
Fusion Mode	Concatenation(384)	Residual Addition(256)
Regularization	Dropout(0.1)	LayerNorm + Dropout(0.05)