update TODO with MLP results, cross-scene findings, completed item statuses

TODO.md

@@ -1,25 +1,27 @@
 # TODO
 
-1. Greedy forward selection for the dim search. Random K=20 subsets plateau at AUC 0.74 because of the birthday-problem of dim collisions. Greedy starts with the single best dim, adds the dim that most reduces residual error given the prior picks, repeats. Same wall-clock compute, expected AUC ~0.92 instead of 0.86 because the search no longer wastes trials on redundant dims. Closed-form forward stagewise regression is the strict version.
+1. Greedy forward selection for the dim search. Not pursued further: at K=200 with alpha=1 ridge regularization the closed-form forward selection plateaus around AUC 0.85, below the all-768 ridge ceiling of 0.860. With alpha=1 ridge already absorbing the noisy dims, removing them via selection costs more than it saves. The result is a smaller head with no metric gain. Negative result, archived.
 
-2. Non-linear dim discovery. Currently ridge over raw features. Adding pairwise products (dim_i · dim_j) as candidate features captures multiplicative relationships that linear ridge cannot. The dim search then operates over a 768 + 768·767/2 ≈ 295K candidate-feature space instead of 768.
+2. Non-linear dim discovery. Currently ridge over raw features, plus a 2-layer MLP head (now the default) that effectively mixes the 768 input dims via a learned 256-unit hidden layer. The MLP gives in-distribution AUC 0.980 (vs ridge 0.860). Pairwise products as candidate features (the original direction from this item) is no longer the cheapest route since the MLP already captures multiplicative interactions through its hidden layer.
 
 3. Joint discovery for fg+depth+normal. Right now the fg head and depth head are searched independently. Co-discovered: each candidate dim subset is scored on a multi-task loss (foreground AUC + depth RMSE + per-patch surface normal cosine). The optimal subset is rarely the union of two independently-discovered subsets.
 
-4. Per-cluster heads. After k-means produces 8 modes, run discovery once per mode. Each mode gets its own dim-subset and ridge coefficients. Adapts to "this is a thin wall-mounted thing" vs "this is a chunky furniture piece" without labels.
+4. Per-cluster heads. After k-means produces 8 modes, run discovery once per mode. Each mode gets its own dim-subset and ridge coefficients. Adapts to "this is a thin wall-mounted thing" vs "this is a chunky furniture piece" without labels. With the MLP-head infrastructure in place, this could be 8 specialized MLPs.
 
-5. Multi-resolution discovery. Cache features at 384 (24x24 patches) AND 768 (48x48) AND 1024 (64x64) and search over the union. Different scales surface different dims.
+5. Multi-resolution discovery. Cache features at 384 (24x24 patches) AND 768 (48x48) AND 1024 (64x64) and search over the union. Different scales surface different dims and different objects: 5 cm photo frames at 2 m depth become 1-2 patches at 1024 input vs 1 patch at 768.
 
-6. Boundary-aware OBB fit. The current OBB fit takes percentile extents on a point cloud. Instead, find depth discontinuities in image space (large depth gradient) and use those as face-edge constraints. Anchors box faces to actual object boundaries instead of the percentile of a noisy point set.
+6. Boundary-aware OBB fit. The current OBB fit takes percentile extents on a point cloud. Tested a min-area rotating-calipers OBB on the 2D xz convex hull as a drop-in replacement; it ties the percentile fit on mean IoU and recall (0.059 vs 0.063 mean, 18.4% vs 19.8% recall). The yaw is more stable but the bound on extents is no tighter than percentile trim. Not shipped.
 
 7. Yaw priors from k-means modes. The size priors gave each cluster a (w, h, d). The cluster also has a typical yaw distribution. Wall-mounted clusters skew toward yaw approximately 0 or pi/2 (axis-aligned with walls); freestanding clusters are uniform. Add per-cluster yaw priors and regularize.
 
-8. Test-time random-hypothesis search. Brute-force evolutionary search per image. Sample many candidate boxes, score each by (a) fraction of hypothesized-interior pixels that pass the fg threshold, (b) consistency of hypothesized-interior depths with predicted depth from the head, (c) face-edge alignment with depth discontinuities. Keep the best hypothesis. No global shape priors; the image's own evidence drives the search. This is the strictest extension of the 1-param classifier's discovery methodology, the search happens at test time per image, not at train time per dataset.
+8. Test-time random-hypothesis search. Two variants tried. v1: per-image random sampling of 2000 hypotheses scored by `fg_density × log(n_inside) − depth_var`. v2: CC-anchored hypothesis search with fg-dropoff score `mean_fg_inside − mean_fg_in_shell`. Both regressed vs bespoke (mean IoU 0.054 and 0.031 respectively). The score functions tested don't capture "real-object-ness" without ground-truth supervision; better scoring or per-image segmentation prompts (e.g. SAM-style) is the next move on this thread.
 
 9. 9-DoF box discovery. CA-1M has many non-gravity-aligned boxes. The 7-DoF assumption costs accuracy on those. A single extra discovered head outputting "tilt magnitude" plus a small extension to the OBB fit lifts the floor on tilted objects.
 
 10. mAP scoring instead of class-agnostic IoU greedy match. Boxer reports mAP at IoU thresholds [0.05, 0.5]; that's what the field uses. Our current numbers underweight the high-IoU tail. A proper mAP score would be directly comparable to Boxer's 0.43.
 
-11. Cross-scene held-out. Discovery and eval both ran on the same 4 scenes. Holding out one scene from discovery and scoring on it tells us what generalizes vs what's scene-specific.
+11. Cross-scene held-out. Done. Leave-one-scene-out evaluation revealed in-distribution AUC 0.860 (random patch split mixing scenes) drops to cross-scene AUC 0.604 on a held-out scene. The MLP head narrows the gap to 0.678 cross-scene but does not close it. The remaining 0.30 AUC gap is the linear/non-linear head still leaning on per-scene cues that don't transfer.
 
-12. More CA-1M scenes. We've been running on one scene's eval. Aggregating across 4-10 scenes gives a real distribution.
+12. More CA-1M scenes. We've been running on 4 scenes' eval. Aggregating across many more scenes (and in particular giving the head a much larger and more diverse training set) is the cleanest path to closing the cross-scene generalization gap.
+
+13. Depth MLP head. The instance MLP lifted in-distribution AUC from 0.86 to 0.98; the same approach applied to depth could plausibly push RMSE below 0.190 m and improve cross-scene RMSE substantially. Trade-off: depth is regression, where ridge is already a strong baseline, so the lift is likely smaller than for binary fg classification.