File size: 4,673 Bytes
7a87926 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 |
# BA Validation Diagnostics
This document explains the diagnostic information available when running BA validation to help understand why frames are being rejected.
## Overview
When all frames are rejected, it's important to understand the root cause. The enhanced validation script now provides detailed diagnostics to help identify issues.
## Diagnostic Information
### 1. Frame Categorization Statistics
Shows how many frames fall into each category:
- **Accepted** (< 2° rotation error): Frames where DA3 poses are very close to ARKit ground truth
- **Rejected-Learnable** (2-30° rotation error): Frames with moderate error that could be improved with training
- **Rejected-Outlier** (> 30° rotation error): Frames with very high error, likely outliers
### 2. Error Distribution
Provides statistical breakdown of rotation errors:
- **Q1, Median, Q3**: Quartiles showing error distribution
- **90th, 95th, 99th percentiles**: High-end error values
- Helps identify if errors are uniformly high or if there are specific problem frames
### 3. Alignment Diagnostics
Checks if pose alignment is working correctly:
- **Scale factor**: Should be ~1.0 if DA3 and ARKit trajectories have similar scale
- **Rotation matrix determinant**: Should be ~1.0 for a valid rotation matrix
- **Translation centers**: Mean translation values for both pose sets
### 4. Per-Frame Error Breakdown
Shows rotation and translation error for each frame:
- Helps identify specific problematic frames
- Shows which frames are close to thresholds
- Useful for understanding error patterns
### 5. Pose Statistics
Translation magnitude statistics:
- **DA3 poses**: Range and magnitude of DA3 camera positions
- **ARKit poses**: Range and magnitude of ARKit camera positions
- Helps identify scale mismatches
## Common Issues and Diagnostics
### All Frames Rejected as Outliers
**Possible causes:**
1. **Coordinate system mismatch**: Check alignment rotation det (should be ~1.0)
2. **Scale mismatch**: Check scale factor (should be ~1.0)
3. **DA3 model issues**: Very high errors suggest DA3 poses are fundamentally wrong
4. **ARKit data quality**: Check if ARKit tracking was successful
**Diagnostics to check:**
- Alignment scale factor (if far from 1.0, there's a scale issue)
- Rotation error distribution (if all errors are > 170°, likely coordinate system issue)
- Translation error magnitude (if very large, scale or coordinate issue)
### High but Variable Errors
**Possible causes:**
1. **DA3 model limitations**: Model may struggle with certain scene types
2. **Motion blur**: Fast camera movement can cause tracking issues
3. **Low texture**: Scenes with little texture are harder for visual odometry
**Diagnostics to check:**
- Error distribution quartiles (if spread is large, some frames are better)
- Per-frame errors (identify which frames are problematic)
### Alignment Issues
**Symptoms:**
- Scale factor far from 1.0
- Rotation matrix det not ~1.0
- Very high translation errors
**Solutions:**
- Check coordinate system conversion
- Verify ARKit to OpenCV conversion is correct
- Ensure poses are in the same format (w2c vs c2w)
## Using Diagnostics in API
The API now returns diagnostics in the validation results:
```python
{
"validation_stats": {
"total_frames": 10,
"accepted": 0,
"rejected_learnable": 0,
"rejected_outlier": 10,
"diagnostics": {
"error_distribution": {...},
"alignment_info": {...},
"per_frame_errors": [...]
}
}
}
```
## Example Output
```
=== BA Validation Statistics ===
Total Frames Processed: 10
Frame Categorization:
✓ Accepted (< 2°): 0 frames ( 0.0%)
⚠ Rejected-Learnable (2-30°): 0 frames ( 0.0%)
✗ Rejected-Outlier (> 30°): 10 frames (100.0%)
Total Rejected: 10 frames (100.0%)
BA Validation Status: rejected_outlier
Max Rotation Error: 177.76°
=== Detailed Diagnostics ===
Rotation Error Distribution:
Q1 (25th): 170.55°
Median: 177.76°
Q3 (75th): 177.76°
90th: 177.76°
95th: 177.76°
Alignment Diagnostics:
Scale factor: 1.000000 (should be ~1.0)
Rotation det: 1.000000 (should be ~1.0)
Sample Frame Errors (first 5):
Frame 0: 177.76° rot, 1.740m trans - rejected_outlier
Frame 1: 176.50° rot, 1.800m trans - rejected_outlier
...
```
## Next Steps
If all frames are rejected:
1. Check alignment diagnostics (scale factor, rotation det)
2. Review error distribution to see if errors are uniformly high
3. Check per-frame errors to identify patterns
4. Verify coordinate system conversions
5. Check ARKit tracking quality
6. Consider if DA3 model is appropriate for this scene type
|