docs/V11_QUICK_START.md

v11 Architecture - Quick Start Guide

What is v11?

The v11 series represents a major architectural enhancement addressing the classification accuracy plateau at ~51% in the Spatial-BEATs model. It introduces:

1. SpatialDeltaPatchAdapterV2: Enhanced front-end spatial encoder (17.4M params)
2. SpatialAdapterLayer: In-trunk spatial conditioning (1.2M params total)
3. Multiple routing options: Route A/B (track-based) or Route C (ACCDOA class-based)

---

Four v11 Variants

1. v11_phase1_cls - Phase 1 Classification Refinement

Use this first to diagnose if the new adapter improves classification accuracy.

What it does:

• Enables SpatialDeltaPatchAdapterV2 only
• Freezes direction/distance heads
• Trains classification + num_active heads
• Hot-starts from v10 phase-1 best checkpoint

Command:

./run_ov1_v11_phase1_cls.sh

Environment variables:

SPATIAL_EPOCHS=10          # Default: 10 epochs
SPATIAL_LR=7.5e-6          # Default: 7.5e-6
BATCH_SIZE=8               # Default: 8
GPUS=8                     # Default: 8

Expected output:

Epoch 1: cls_acc=0.720 (should be at least v10 level)
Epoch 5: cls_acc=0.755 (expect improvement trend)
Epoch 10: cls_acc=0.78+ (best of phase-1)

What to look for:

• Does cls_acc improve beyond v10 phase-1 peak (0.78)?
• How quickly does it converge?
• Does val_loss plateau or continue improving?

---

2. v11a_ov123_top4 - Route B + Spatial Demixer (Full Architecture)

Use after v11_phase1_cls confirms improvement.

What it does:

• Enables SpatialDeltaPatchAdapterV2 + trunk adapters + spatial_head_demixer
• Trains all heads (activity, class, direction, distance)
• Uses demixer for both class AND spatial heads
• Hot-starts from v9 ov123_top4 best checkpoint

Command:

./run_ov1_v11a_ov123_top4.sh

Key metrics:

• azi_mae_deg: Azimuth mean absolute error (primary DOA metric)
• class_acc: Matched-source class accuracy
• activity_f1: Source presence F1-score

Expected improvement:

Metric                  v9 Baseline    v11a Target    Expected Delta
────────────────────────────────────────────────────────────────────
azi_mae_deg (train)     10°            8-9°           -1 to -2°
azi_mae_deg (val)       30°            24-26°         -4 to -6°
class_acc (val)         73%            75%+           +2% 

What to look for:

• Validation azimuth error should be significantly lower
• Train/val gap should narrow (from ~20° toward ~15°)
• No collapse in accuracy metrics

---

3. v11b_ov123_top4 - Route B + LocalSpatial Demixer KV

Use for comparison with v11a.

What it does:

• Same as v11a, BUT
• Demixer attends to LocalSpatial's 7-channel pre-pool (FOA + IV)
• Instead of BEATs mono mel-filterbank features
• Hypothesis: Spatial features better for DOA decomposition

Command:

./run_ov1_v11b_ov123_top4.sh

Comparison with v11a:

Aspect              v11a (BEATs KV)        v11b (LocalSpatial KV)
─────────────────────────────────────────────────────────────────
Demixer KV source   BEATs trunk            LocalSpatial pre-pool
Channels            1 (mono fbank)         7 (4 FOA + 3 IV)
Prior knowledge     Semantic               Spatial physics
Expected advantage  Better for class       Better for direction
Computational cost  Lower                  Higher

When to pick v11b over v11a:

• If DOA error (azi_mae_deg) is more important than class accuracy
• If you have GPU budget for extra feature processing
• For acoustic scenes where spatial features matter more

---

4. v11c_ov123_accdoa - Paradigm Shift to ACCDOA (Route C)

Use as a "simplicity first" baseline.

What it does:

• Enables SpatialDeltaPatchAdapterV2 + trunk adapters
• Replaces query decoder + Hungarian matching with per-class ACCDOA heads
• Each class gets its own spatial slot (no matching needed)
• Activity encoded in vector magnitude, direction in unit vector

Command:

./run_ov1_v11c_ov123_accdoa.sh

Key differences from v11a:

Aspect              v11a (Route B, Track)      v11c (Route C, ACCDOA)
─────────────────────────────────────────────────────────────────────
Paradigm            K learnable tracks         Per-class slots
Matching            Hungarian (clip-level)     None (inherent per-class)
Activity loss       Binary cross-entropy       MSE on magnitude
Direction repr.     L2 normalized vector      Unit vector (normalized)
Scalability         O(K×T_s) per-frame        O(num_classes×T_s)
ov2/ov3 fit         Good (overlap ambiguity)  Better (same-class=0)

When to pick v11c:

• For DCASE evaluation (uses official SELD metrics)
• If Hungarian matching is a bottleneck
• For datasets with no overlapping same-class sources (ov2/ov3 constraints)
• For interpretability (each class = one direction)

---

Decision Tree: Which v11 to Run?

START
  │
  ├─→ "Do I want to diagnose if new adapters help classification?"
  │   └─→ YES: Run v11_phase1_cls
  │           ↓ (wait for results)
  │           Does cls_acc improve? 
  │           ├─→ YES ✓
  │           │   └─→ Proceed to multi-head experiments
  │           └─→ NO ✗
  │               └─→ Back to drawing board (architecture issue)
  │
  ├─→ "Is direction-of-arrival (DOA) error my primary concern?"
  │   ├─→ YES: Need DOA focus
  │   │   ├─→ "Do I have GPU budget for LocalSpatial features?"
  │   │   │   ├─→ YES: Run v11b_ov123_top4
  │   │   │   └─→ NO: Run v11a_ov123_top4
  │   └─→ NO: Skip v11a/v11b
  │
  └─→ "Am I targeting DCASE evaluation / ov2/ov3 constraints?"
      ├─→ YES: Run v11c_ov123_accdoa
      └─→ NO: Run v11a_ov123_top4 (default full-featured)

---

Monitoring Experiments

Key Metrics to Track

Classification:

• class_acc: Top-1 accuracy on matched sources
• class_precision: Per-class precision
• class_recall: Per-class recall

Direction (DOA):

• azi_mae_deg: Primary metric - azimuth mean absolute error
• ele_mae_deg: Elevation mean absolute error
• azi_std_deg: Azimuth error standard deviation

Distance:

• dist_mae_m: Distance mean absolute error

Activity:

• activity_f1: Source presence F1-score
• num_active_mae: Mean absolute error in source count

Gap Analysis:

• train_azi_mae_deg: Training set azimuth error
• val_azi_mae_deg: Validation set azimuth error
• gap = val - train: Gap should decrease with v11

TensorBoard Visualization

tensorboard --logdir=checkpoints/spatial_beats_v11_phase1_cls_exp/ov123_top4 --port=6006

Plots to monitor:

• metrics/val_azi_mae_deg: Should decrease smoothly
• metrics/train_azi_mae_deg: Should decrease with training
• loss/total: Should follow training dynamics (may oscillate)
• loss/frame_direction: DOA-specific loss component

---

Checkpoint Management

Hot-Start Strategy

Each v11 variant is designed to hot-start from a previous checkpoint:

v11_phase1_cls:

Loads from: v10_phase1_cls best.pt
Missing params: V2 adapter + trunk adapters
Initialize with: Zero-init adapters (identity at epoch-0)
Benefit: Inherits v10's frozen classification features

v11a_ov123_top4:

Loads from: v9_ov123_top4 best.pt
Missing params: V2 + trunk adapters + spatial_demixer (added to heads)
Initialize with: Zero-init everything (identity at epoch-0)
Benefit: Inherits v9's proven multi-head balance

v11b_ov123_top4:

Same as v11a, but adds LocalSpatial pre-pool processing

v11c_ov123_accdoa:

Loads from: ov1_local_spatial baseline (v9 incompatible)
Missing params: ACCDOAHeads (entire head replacement)
Initialize with: Zero-init (no class/spatial heads to inherit)
Benefit: Simpler routing = faster convergence

How to Load a Checkpoint Manually

import torch
from train_spatial_beats import make_ov1_local_spatial_v11a_ov123_top4_config
from spatial_beats import SpatialBEATs

# Create model with v11a config
cfg = make_ov1_local_spatial_v11a_ov123_top4_config()
model = SpatialBEATs(cfg)

# Load v9 checkpoint (strict=False ignores new params)
ckpt = torch.load('checkpoints/.../v9_best.pt')
model.load_state_dict(ckpt['model'], strict=False)

# New params are zero-initialized (identity behavior)
# Ready to train!
model.train()

---

Troubleshooting

Issue: "CUDA out of memory"

Solution: Reduce batch size or sequence length

BATCH_SIZE=4 ./run_ov1_v11a_ov123_top4.sh

Issue: "ClassHeadSpectralDemixer not initialized"

Solution: Ensure config enables it:

cfg.use_class_head_demixer = True  # For v11a
cfg.use_spatial_head_demixer = True  # For v11a (added in v11)

Issue: "Large train/val gap not shrinking"

Diagnosis steps:

1. Check if Dropout is OFF during evaluation
2. Verify SpecAugment is applied only during training
3. Run diagnostic: evaluate same checkpoint in train/eval modes

python -c "
model.eval()
val_error_no_dropout = evaluate(model, val_loader)
model.train()
val_error_with_dropout = evaluate(model, val_loader)
print(f'Dropout effect: {val_error_with_dropout - val_error_no_dropout:.1f}°')
"

Issue: "Trunk adapters not being applied"

Check: Verify config flag is True

if not cfg.use_trunk_spatial_adapters:
    print("WARNING: Trunk adapters disabled!")
    cfg.use_trunk_spatial_adapters = True

---

Next Steps After v11 Experiments

1. Analyze results (docs/V11_IMPLEMENTATION_SUMMARY.md contains diagnostic templates)
2. Pick best variant based on your primary metric
3. Fine-tune hyperparameters (learning rate, dropout rate if you modify later)
4. Run official evaluation on test set using DCASE metrics
5. Consider multi-stage training:
   • Stage 1: Classification only (v11_phase1_cls)
   • Stage 2: Full pipeline (v11a/b/c)
   • Stage 3: Fine-tuning (reduce LR, increase epochs)

---

Citation & References

This architecture is built on:

• BEATs (Microsoft): Base semantic encoder (https://arxiv.org/abs/2212.09058)
• DCASE SELD: Official evaluation metrics (https://github.com/sharathadavanne/seld-dcase2023)
• EINV2 paradigm: Track-based source modeling
• Spatial audio physics: FOA (First-Order Ambisonics) + Intensity Vectors

For detailed technical justification, see:

• docs/V11_IMPLEMENTATION_SUMMARY.md
• docs/doa_train_valid_gap_analysis.md
• SPATIAL_AUDIO_FRAMEWORKS_ANALYSIS_COMPREHENSIVE.md