docs/BA_VALIDATION_DIAGNOSTICS.md

# BA Validation Diagnostics

This document explains the diagnostic information available when running BA validation to help understand why frames are being rejected.

## Overview

When all frames are rejected, it's important to understand the root cause. The enhanced validation script now provides detailed diagnostics to help identify issues.

## Diagnostic Information

### 1. Frame Categorization Statistics

Shows how many frames fall into each category:

- **Accepted** (< 2° rotation error): Frames where DA3 poses are very close to ARKit ground truth
- **Rejected-Learnable** (2-30° rotation error): Frames with moderate error that could be improved with training
- **Rejected-Outlier** (> 30° rotation error): Frames with very high error, likely outliers

### 2. Error Distribution

Provides statistical breakdown of rotation errors:

- **Q1, Median, Q3**: Quartiles showing error distribution
- **90th, 95th, 99th percentiles**: High-end error values
- Helps identify if errors are uniformly high or if there are specific problem frames

### 3. Alignment Diagnostics

Checks if pose alignment is working correctly:

- **Scale factor**: Should be ~1.0 if DA3 and ARKit trajectories have similar scale
- **Rotation matrix determinant**: Should be ~1.0 for a valid rotation matrix
- **Translation centers**: Mean translation values for both pose sets

### 4. Per-Frame Error Breakdown

Shows rotation and translation error for each frame:

- Helps identify specific problematic frames
- Shows which frames are close to thresholds
- Useful for understanding error patterns

### 5. Pose Statistics

Translation magnitude statistics:

- **DA3 poses**: Range and magnitude of DA3 camera positions
- **ARKit poses**: Range and magnitude of ARKit camera positions
- Helps identify scale mismatches

## Common Issues and Diagnostics

### All Frames Rejected as Outliers

**Possible causes:**

1. **Coordinate system mismatch**: Check alignment rotation det (should be ~1.0)
2. **Scale mismatch**: Check scale factor (should be ~1.0)
3. **DA3 model issues**: Very high errors suggest DA3 poses are fundamentally wrong
4. **ARKit data quality**: Check if ARKit tracking was successful

**Diagnostics to check:**

- Alignment scale factor (if far from 1.0, there's a scale issue)
- Rotation error distribution (if all errors are > 170°, likely coordinate system issue)
- Translation error magnitude (if very large, scale or coordinate issue)

### High but Variable Errors

**Possible causes:**

1. **DA3 model limitations**: Model may struggle with certain scene types
2. **Motion blur**: Fast camera movement can cause tracking issues
3. **Low texture**: Scenes with little texture are harder for visual odometry

**Diagnostics to check:**

- Error distribution quartiles (if spread is large, some frames are better)
- Per-frame errors (identify which frames are problematic)

### Alignment Issues

**Symptoms:**

- Scale factor far from 1.0
- Rotation matrix det not ~1.0
- Very high translation errors

**Solutions:**

- Check coordinate system conversion
- Verify ARKit to OpenCV conversion is correct
- Ensure poses are in the same format (w2c vs c2w)

## Using Diagnostics in API

The API now returns diagnostics in the validation results:

```python
{
  "validation_stats": {
    "total_frames": 10,
    "accepted": 0,
    "rejected_learnable": 0,
    "rejected_outlier": 10,
    "diagnostics": {
      "error_distribution": {...},
      "alignment_info": {...},
      "per_frame_errors": [...]
    }
  }
}
```

## Example Output

```
=== BA Validation Statistics ===
Total Frames Processed: 10

Frame Categorization:
  ✓ Accepted (< 2°):          0 frames (  0.0%)
  ⚠ Rejected-Learnable (2-30°):   0 frames (  0.0%)
  ✗ Rejected-Outlier (> 30°):   10 frames (100.0%)

Total Rejected: 10 frames (100.0%)

BA Validation Status: rejected_outlier
Max Rotation Error: 177.76°

=== Detailed Diagnostics ===
Rotation Error Distribution:
  Q1 (25th): 170.55°
  Median:    177.76°
  Q3 (75th): 177.76°
  90th:      177.76°
  95th:      177.76°

Alignment Diagnostics:
  Scale factor: 1.000000 (should be ~1.0)
  Rotation det: 1.000000 (should be ~1.0)

Sample Frame Errors (first 5):
  Frame 0: 177.76° rot, 1.740m trans - rejected_outlier
  Frame 1: 176.50° rot, 1.800m trans - rejected_outlier
  ...
```

## Next Steps

If all frames are rejected:

1. Check alignment diagnostics (scale factor, rotation det)
2. Review error distribution to see if errors are uniformly high
3. Check per-frame errors to identify patterns
4. Verify coordinate system conversions
5. Check ARKit tracking quality
6. Consider if DA3 model is appropriate for this scene type