auto-sync 2026-07-04T01:43:46Z workspace (part 3)

workspace/reports/2026-07-03.md

@@ -34,6 +34,7 @@
 - Extended `scripts/eval_learned_dominance_selector.py` with `--threshold-scope task`, a Mondrian execute/fallback threshold over visible `task_id` buckets. The model weights and task thresholds are fit on calibration rows only; held-out eval outcomes are used only for reporting.
 - Extended `scripts/eval_learned_dominance_selector.py` with `--fit-objective {pointwise,pairwise,hybrid_pairwise}`. The pairwise objectives train the ridge selector on within-chart measured causal comparisons from calibration rows only, then calibrate execute/fallback thresholds on candidate-row predictions. Added regression coverage in `tests/test_dominance_selector.py` for pairwise design rows outnumbering candidate rows.
 - Extended `scripts/eval_learned_dominance_selector.py` with deployment-visible chart-compatibility feature sets and `--selector-chart-feature-mode`. The new path loads target/source chart features without hidden outcomes, compares whole/base/RGB-stat/object-layout segments, logs selector/source split hashes, and has regression coverage in `tests/test_dominance_selector.py`.
+- Fixed `scripts/eval_nonlinear_dominance_selector.py` to load deployment-visible chart-compatibility maps via `--selector-chart-feature-mode` and to record source/index hashes when chart compatibility or source evidence is used. Added regression coverage that nonlinear chart-compat provenance records the default train source index.
 - Fixed `scripts/summarize_ctt_runs.py` so summary artifacts include `ctt*_learned_dominance*/metrics.json` runs such as `ctt_base_context_obs_learned_dominance_*`, not only the older `ctt_learned_dominance*` naming pattern.
 - Extended `scripts/eval_dominance_selector.py` with `--score-source checkpoint`, cached chart embeddings/base scores, and single-thread torch scoring so a standalone train-only utility-energy checkpoint can be used for dominance evaluation.
 - Updated `scripts/train_utility_energy.py` to save `chart_dim` in checkpoints.
@@ -720,6 +721,12 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
     - `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_success_weighted_task_envclip_k16_train_to_test`: selected success `0.3333`, coverage `0.7222`, selector gap `0.2569`
     - `runs/ctt_base_context_obs_learned_dominance_chartcompat_obj_success_weighted_task_envclip_k16_train_to_test`: selected success `0.3125`, coverage `0.5486`, selector gap `0.2847`
     - `runs/ctt_base_context_obs_learned_dominance_chartcompat_success_weighted_task_envclip_k16_train_to_test`: selected success `0.3264`, coverage `0.5486`, selector gap `0.2639`
+  - nonlinear chart-compatibility K=16 diagnostics after fixing feature loading:
+    - `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_utility_envclip_k16_train_to_test`: selected success `0.2847`, coverage `0.3542`, selector gap `0.2986`
+    - `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_success_weighted_envclip_k16_train_to_test`: selected success `0.2847`, coverage `0.3542`, selector gap `0.2986`
+    - `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_positive_envclip_k16_train_to_test`: selected success `0.2847`, coverage `0.3542`, selector gap `0.2986`
+    - `runs/ctt_base_context_obs_nonlinear_dominance_chart_sourcecompat_obs_utility_envclip_k16_train_to_test`: selected success `0.2639`, coverage `0.5278`, selector gap `0.3264`
+    - regressor-only follow-up `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_success_weighted_reg_envclip_k16_train_to_test`: selected success `0.3264`, coverage `0.3403`, selector gap `0.2639`
 
 ### Best Metric Table
 
@@ -741,7 +748,10 @@ with a utility-margin ridge model and task-specific Mondrian thresholds. The
 previous basic success-weighted task-threshold row reached 32.64%, and the
 RGB-stat success-weighted row reaches 33.33%. Object-layout, source-evidence,
 nonlinear, pairwise within-chart ranking, and hybrid pointwise-plus-pairwise
-variants do not resolve the selector bottleneck.
+variants do not resolve the selector bottleneck. After fixing nonlinear
+chart-compat feature loading, HGB/RF nonlinear selectors still peak at 32.64%,
+so the remaining issue is not merely nonlinear model capacity over the current
+hand-built chart statistics.
 
 ### Paper Changes
 
@@ -761,6 +771,10 @@ variants do not resolve the selector bottleneck.
   `latex/main.tex`: train-only RGB-stat target--source compatibility improves
   selected success to `0.3542`, but the remaining selector gap keeps it a
   diagnostic rather than a solved deployment method.
+- Added the fixed nonlinear chart-compat selector negative diagnostic to
+  `latex/main.tex`: nonlinear selectors now load the same deployment-visible
+  chart maps and source hashes as the ridge selector, but do not improve over
+  the linear chart-compat row.
 
 ### Leakage and Data Status
 
@@ -852,7 +866,8 @@ variants do not resolve the selector bottleneck.
 1. Treat K=16 `env_clip` as the current declared bounded-action diagnostic convention: it is action-bound-clean and now support-strong on held-out test (`0.5694` proposal oracle, `0.5486` OutcomePTR@16), exposing the selector/dominance blocker sharply.
 2. Replace deterministic RGB/object descriptors with learned visual-language or task/object tokens that distinguish target, distractor, gripper/contact region, and task stage.
 3. Train a dominance/utility selector that uses richer deployment-visible chart features under K=16 `env_clip`; the best train-clean row is now `0.3542` selected success against a `0.5694` proposal oracle, leaving a `0.2431` selector gap.
-4. Improve the decoder/action representation beyond post-hoc scalar or per-dimension scaling; both collapse support, while tanh trades support for validity.
-5. Stop treating deterministic object-layout as a likely win: it passed proxy but failed measured support relative to RGB-stat.
-6. Rerun measured support/selection after the learned representation/selector fix under exactly the K=16 `env_clip` action convention before changing the support generator again.
-7. Add actual collision/contact labels after action-bound validity is resolved.
+4. Do not spend more time on model-capacity-only selector variants over the current deterministic chart statistics: fixed nonlinear chart-compat peaks at `0.3264`, below the ridge chart-compat row.
+5. Improve the decoder/action representation beyond post-hoc scalar or per-dimension scaling; both collapse support, while tanh trades support for validity.
+6. Stop treating deterministic object-layout as a likely win: it passed proxy but failed measured support relative to RGB-stat.
+7. Rerun measured support/selection after the learned representation/selector fix under exactly the K=16 `env_clip` action convention before changing the support generator again.
+8. Add actual collision/contact labels after action-bound validity is resolved.

workspace/reports/draft_audit_ctt.md

@@ -138,6 +138,15 @@ diagnostic, but it is still hand-built RGB statistics rather than learned
 object-centric CTT geometry. Object-layout-only reaches 0.3125, combined
 RGB-stat/object-layout ties 0.3264, and RGB-stat success-weighted reaches
 0.3333, so deterministic object-layout features are not yet the missing method.
+The nonlinear selector path previously accepted chart-compat feature sets
+without loading selector chart maps, making that diagnostic too weak. It now
+loads the same deployment-visible selector chart maps as the ridge selector and
+records source/index hashes. The rerun is still negative: HGB classifier
+variants over RGB-stat compatibility reach 0.2847 selected success,
+source+RGB-stat compatibility reaches 0.2639, and the best regressor-only
+RGB-stat compatibility row reaches 0.3264. Therefore the current selector
+failure is not resolved by nonlinear capacity over the deterministic chart
+statistics.
 The current continuation adds a leakage-clean deterministic object-layout chart
 token path, but it is not yet promoted as a method result. The exporter
 `scripts/export_chart_object_embeddings.py` writes 64D RGB foreground
@@ -336,6 +345,11 @@ Current artifact evidence:
   selected success 0.3542, coverage 0.6597, proposal oracle 0.5694, and
   success selector gap 0.2431. It uses train-only target--source RGB-stat
   chart compatibility and remains a selector diagnostic.
+- `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*_envclip_k16_train_to_test`
+  and `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*_reg_envclip_k16_train_to_test`
+  contain the fixed nonlinear chart-compat diagnostics. They now log
+  `selector_chart_feature_mode=base_context_obs` and train-source hashes; the
+  best selected success is 0.3264, below the ridge chart-compat row.
 - `runs/ctt_base_context_obs_{val,test}_tanh_rollout_comparison` contain the
   full bounded-decoder tanh measured validation/test diagnostics. They are
   action-bound-clean but support-negative relative to clipped/no-clipping
@@ -595,9 +609,9 @@ Current missing:
   improves selected success to 0.3542, with a 0.2431 selector gap to the
   proposal oracle. This is better than the earlier 0.3264 basic
   success-weighted task-threshold row, but not enough for a method-success
-  claim. Nonlinear K=16 selectors, source-evidence/task-threshold variants,
-  deterministic object-layout compatibility, and pairwise/hybrid ridge
-  objectives do not materially improve the diagnosis.
+  claim. Fixed nonlinear chart-compat selectors, source-evidence/task-threshold
+  variants, deterministic object-layout compatibility, and pairwise/hybrid
+  ridge objectives do not materially improve the diagnosis.
 
 ## Immediate Fix Order
 
@@ -622,9 +636,10 @@ Next:
    object-centric chart tokens; the current K=16 support row reaches 0.5694
    proposal oracle, but the best train-clean selector reaches only 0.3542.
 3. Replace the validation-calibrated ridge/context selector with a stronger
-   deployment-clean train-only dominance model. The new pairwise/hybrid ridge
-   diagnostic is negative: within-chart causal-ranking loss alone does not beat
-   the current chart-compatibility K=16 selector.
+   deployment-clean train-only dominance model. The pairwise/hybrid ridge and
+   fixed nonlinear chart-compat diagnostics are negative: within-chart
+   causal-ranking loss or nonlinear capacity alone does not beat the current
+   chart-compatibility K=16 selector.
 4. Fix action representation/decoding rather than applying a global scalar:
    no-clipping jobs `15117739`--`15117744` show widespread bound violations,
    while scaled0.215 fixes base bounds but collapses generated support and