DLoc Phase 2 Results Summary
Experiment Setup
- Dataset: WILD Jacobs Hall (Fig 10b), 4 APs, 161x361 heatmaps at 0.1m/pixel
- Training data: Session 1 (Jul28) + Session 2 train split (Jul28_2) — 17,574 samples
- Test data: Session 2 test split — 4,260 samples
- Hardware: NVIDIA H100 80GB, PyTorch 2.6.0+cu124
- Date: 2026-06-08/09
Step 0: Student Model Training (50 epochs each)
| Model | Params | Median | P90 | P99 |
|---|---|---|---|---|
| MobileNet standalone | 2.27M | 0.707m | 1.616m | 3.992m |
| MobileNet distilled | 2.27M | 0.707m | 1.612m | 4.003m |
| TinyCNN standalone | 46.7K | 3.922m | 13.919m | 23.244m |
| TinyCNN distilled | 46.7K | 3.706m | 14.784m | 25.101m |
Distillation loss: L = 0.7 * MSE(student, GT) + 0.3 * MSE(student, teacher_output)
Step 1: Cross-Session Generalization
Models trained on Jul28 sessions only, tested on unseen Aug16 sessions.
| Model | Sess 2 (in-dist) | Sess 3 (furniture) | Sess 4 (diff furn) | Sess 5 (reflector) |
|---|---|---|---|---|
| MobileNet standalone | 0.707m | 1.118m | 1.204m | 1.500m |
| MobileNet distilled | 0.707m | 1.118m | 1.204m | 1.442m |
| TinyCNN standalone | 3.922m | 5.964m | 6.327m | 6.605m |
| TinyCNN distilled | 3.706m | 5.852m | 5.903m | 6.888m |
| U-Net cosine standalone | 0.640m | 1.063m | 1.005m | 1.334m |
| U-Net cosine distilled | 0.640m | 1.020m | 1.005m | 1.281m |
U-Net distilled generalizes best across all sessions.
Steps 2 & 3: Robustness Evaluation
MobileNet Standalone vs Distilled
| Condition | Standalone | Distilled | Improvement |
|---|---|---|---|
| Normal | 0.707m | 0.707m | — |
| Random 1-AP dropout | 1.100m | 1.166m | -6% |
| Random 2-AP dropout | 2.402m | 2.516m | -5% |
| Gaussian noise σ=0.1 | 16.368m | 4.769m | +71% |
| Gaussian noise σ=0.2 | 16.307m | 5.369m | +67% |
| Blur (k=5, σ=1) | 0.849m | 0.854m | — |
Key finding: Distillation improves noise robustness by 3.4x for MobileNet.
U-Net Cosine (Standalone)
| Condition | Median |
|---|---|
| Normal | 0.640m |
| Random 1-AP dropout | 1.281m |
| Gaussian noise σ=0.1 | 3.000m |
| Gaussian noise σ=0.2 | 5.741m |
| Blur (k=5, σ=1) | 0.671m |
U-Net is the most robust architecture overall.
Step 5: Architectural Improvements
U-Net + LR Scheduler Comparison (120 epochs)
| Architecture | Scheduler | Mode | Median | P90 | P99 |
|---|---|---|---|---|---|
| U-Net (2.41M) | Cosine | standalone | 0.640m | 1.523m | 2.750m |
| U-Net (2.41M) | Step | standalone | 0.640m | 1.552m | 2.918m |
| U-Net (2.41M) | Cosine | distill | 0.640m | 1.513m | 3.587m |
Both schedulers converge to same accuracy. Step LR converges faster (epoch 63 vs 110). U-Net skip connections improve median by 9.5% vs plain MobileNet (0.707m -> 0.640m).
Key Findings
- MobileNetV2 matches DLoc baseline with 7x fewer parameters (2.3M vs 16.5M)
- U-Net skip connections improve accuracy by 9.5% (0.640m vs 0.707m)
- Distillation has minimal impact on accuracy but dramatically improves noise robustness (3.4x for MobileNet)
- Cross-session: U-Net distilled degrades only +0.38-0.64m across environment changes
- TinyCNN (47K params, 350x smaller) achieves 3.7-3.9m — too coarse for practical use
- Cosine vs Step LR: same final accuracy; step converges faster