anhtld commited on
Commit
4079eaf
·
verified ·
1 Parent(s): 842a8d9

auto-sync 2026-07-04T02:06:24Z workspace (part 3)

Browse files
workspace/reports/2026-07-03.md CHANGED
@@ -54,6 +54,9 @@
54
  - Extended `cil/chart_features.py` with `base_context_obj` and `base_context_obs_obj` so CTT can test object-layout-only and RGB-stat-plus-object-layout chart tokens.
55
  - Updated `scripts/audit_chart_feature_sources.py` to audit `object_embedding_path` coverage and report feature dimensions for the new object-layout modes.
56
  - Updated selector/comparison scripts so new metrics artifacts expose top-level `data_hash` and `split_hash`.
 
 
 
57
  - Updated CTT loaders to attach `_chart_root` metadata so relative observation-embedding paths resolve correctly in train, proxy, rollout, utility-energy, and dominance code paths.
58
  - Added `scripts/slurm/train_ctt_feature_proxy.sbatch` for feature-mode CTT proxy sweeps; it trains residual CTT and refreshes the proxy comparison/summary artifacts for a specified chart feature mode.
59
  - Fixed validation/test dominance selectors to load chart maps with the checkpoint's recorded chart feature mode. This prevents 112D `base` features from being fed into 162D `base_context_obs` checkpoints when row-scored rollouts still need checkpoint base scores.
@@ -585,7 +588,8 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
585
  - `data/cil_charts/test`: outcomes visible only to evaluator, retrieval forbidden.
586
  - Cross-split chart/state overlap audit: pass.
587
  - Expert branches remain excluded from deployment candidates by index contract.
588
- - `data/cil_charts_rgb_refs/{train,val,test}`: same split/audience contract, RGB-ref leakage audit pass with 0 violations, and 100% observation-ref plus 100% RGB-stat embedding coverage in `runs/chart_feature_audit_rgb_refs`.
 
589
  - `data/cil_charts_rgb_refs/{train,val,test}` after object-layout export: same split/audience contract, leakage audit pass with 0 violations in `runs/leakage_audit_rgb_refs_object`, and 100% object-layout embedding coverage in `runs/chart_feature_audit_rgb_refs_object`.
590
 
591
  ## Data Accounting Status
@@ -598,6 +602,7 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
598
  - `runs/chart_feature_audit` documents the original `data/cil_charts` inputs: instruction and scene ids exist, but observation embeddings/raw refs do not.
599
  - `runs/chart_feature_audit_rgb_refs` documents the new non-destructive RGB-ref export: all splits have observation refs and deterministic 32D RGB-stat embeddings. This supports visual-stat diagnostics only; it does not yet support a learned visual-language/object-centric claim.
600
  - `runs/chart_feature_audit_rgb_refs_object` documents the deterministic 64D RGB object-layout export. The full proxy rows and measured validation rollout are complete; the measured result is negative relative to RGB-stat support.
 
601
 
602
  ## Paper Changes
603
 
@@ -628,6 +633,7 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
628
  - Updated the abstract with the latest visual-stat test support numbers, clipped-convention train-calibrated selected-success number (`31.25%`), and bounded-tanh train-calibrated selected-success number (`38.19%`), while keeping the conclusion diagnostic.
629
  - Added `scripts/eval_nonlinear_dominance_selector.py` and summary support for nonlinear train-calibrated selector diagnostics. The paper/report keep these as negative selector evidence because the best nonlinear row reaches only `30.56%`.
630
  - Updated limitations: current evidence includes validation/test measured rollout, but selected-action deployment remains unsolved; real-robot recovery, unsafe-contact measurement, and dominance-calibrated deployment remain missing.
 
631
  - Added local-atlas/proxy-comparison scripts to the reproducibility artifact list.
632
  - Added generated-rollout script and Slurm wrapper to the reproducibility artifact list.
633
  - Added no-clipping validation/test aggregate tables via `\input{../runs/ctt_base_context_obs_val_noclip_rollout_comparison/table}` and `\input{../runs/ctt_base_context_obs_test_noclip_rollout_comparison/table}`. The draft now states that raw-action replay confirms support remains measurable but exposes action scaling/decoding as a blocker.
 
54
  - Extended `cil/chart_features.py` with `base_context_obj` and `base_context_obs_obj` so CTT can test object-layout-only and RGB-stat-plus-object-layout chart tokens.
55
  - Updated `scripts/audit_chart_feature_sources.py` to audit `object_embedding_path` coverage and report feature dimensions for the new object-layout modes.
56
  - Updated selector/comparison scripts so new metrics artifacts expose top-level `data_hash` and `split_hash`.
57
+ - Updated `scripts/check_tangent_reconstruction.py` so reconstruction artifacts expose structured top-level `data_hash` and `split_hash` manifests in `metrics.json`, not only sidecar text files.
58
+ - Updated `scripts/audit_cil_charts.py` so the run-hash audit accepts either top-level `metrics.json` hashes or nonempty `data_hash.txt`/`split_hash.txt` sidecars, which matches the required run artifact contract.
59
+ - Updated `scripts/build_ctt_rollout_comparison.py` so measured rollout comparison summaries expose top-level `data_hash` and `split_hash`, then refreshed the legacy `runs/ctt_val16_rollout_comparison` artifact.
60
  - Updated CTT loaders to attach `_chart_root` metadata so relative observation-embedding paths resolve correctly in train, proxy, rollout, utility-energy, and dominance code paths.
61
  - Added `scripts/slurm/train_ctt_feature_proxy.sbatch` for feature-mode CTT proxy sweeps; it trains residual CTT and refreshes the proxy comparison/summary artifacts for a specified chart feature mode.
62
  - Fixed validation/test dominance selectors to load chart maps with the checkpoint's recorded chart feature mode. This prevents 112D `base` features from being fed into 162D `base_context_obs` checkpoints when row-scored rollouts still need checkpoint base scores.
 
588
  - `data/cil_charts/test`: outcomes visible only to evaluator, retrieval forbidden.
589
  - Cross-split chart/state overlap audit: pass.
590
  - Expert branches remain excluded from deployment candidates by index contract.
591
+ - `runs/leakage_audit` and `runs/leakage_audit_rgb_refs` now pass with 0 violations and 0 warnings after accepting valid hash sidecars and refreshing the legacy val16 comparison artifact.
592
+ - `data/cil_charts_rgb_refs/{train,val,test}`: same split/audience contract, RGB-ref leakage audit pass with 0 violations/0 warnings, and 100% observation-ref plus 100% RGB-stat embedding coverage in `runs/chart_feature_audit_rgb_refs`.
593
  - `data/cil_charts_rgb_refs/{train,val,test}` after object-layout export: same split/audience contract, leakage audit pass with 0 violations in `runs/leakage_audit_rgb_refs_object`, and 100% object-layout embedding coverage in `runs/chart_feature_audit_rgb_refs_object`.
594
 
595
  ## Data Accounting Status
 
602
  - `runs/chart_feature_audit` documents the original `data/cil_charts` inputs: instruction and scene ids exist, but observation embeddings/raw refs do not.
603
  - `runs/chart_feature_audit_rgb_refs` documents the new non-destructive RGB-ref export: all splits have observation refs and deterministic 32D RGB-stat embeddings. This supports visual-stat diagnostics only; it does not yet support a learned visual-language/object-centric claim.
604
  - `runs/chart_feature_audit_rgb_refs_object` documents the deterministic 64D RGB object-layout export. The full proxy rows and measured validation rollout are complete; the measured result is negative relative to RGB-stat support.
605
+ - `runs/tangent_reconstruction` and `runs/tangent_reconstruction_rgb_refs` verify 45,968 exported spline tangent rows each with 0 failures, max absolute error `0.0`, and structured data/split hash manifests in `metrics.json`.
606
 
607
  ## Paper Changes
608
 
 
633
  - Updated the abstract with the latest visual-stat test support numbers, clipped-convention train-calibrated selected-success number (`31.25%`), and bounded-tanh train-calibrated selected-success number (`38.19%`), while keeping the conclusion diagnostic.
634
  - Added `scripts/eval_nonlinear_dominance_selector.py` and summary support for nonlinear train-calibrated selector diagnostics. The paper/report keep these as negative selector evidence because the best nonlinear row reaches only `30.56%`.
635
  - Updated limitations: current evidence includes validation/test measured rollout, but selected-action deployment remains unsolved; real-robot recovery, unsafe-contact measurement, and dominance-calibrated deployment remain missing.
636
+ - Added explicit tangent reconstruction and hash-audited leakage evidence to the reproducibility story: both original and RGB-ref chart DBs now have 45,968-row deterministic tangent checks with zero failures, and leakage audits report 0 violations/0 warnings.
637
  - Added local-atlas/proxy-comparison scripts to the reproducibility artifact list.
638
  - Added generated-rollout script and Slurm wrapper to the reproducibility artifact list.
639
  - Added no-clipping validation/test aggregate tables via `\input{../runs/ctt_base_context_obs_val_noclip_rollout_comparison/table}` and `\input{../runs/ctt_base_context_obs_test_noclip_rollout_comparison/table}`. The draft now states that raw-action replay confirms support remains measurable but exposes action scaling/decoding as a blocker.
workspace/reports/draft_audit_ctt.md CHANGED
@@ -68,7 +68,8 @@ hidden evaluator fields. Its one-seed proxy artifact,
68
  PPTC@0.40 = 0.6232, NegativeNear@0.20 = 0.0182, and mean positive distance =
69
  0.4429 on 69 validation charts. The new non-destructive RGB-ref export fixes the
70
  source-input problem for a simple visual-stat diagnostic:
71
- `data/cil_charts_rgb_refs` passes leakage audit, `runs/chart_feature_audit_rgb_refs`
 
72
  shows 100% observation refs and 100% 32D RGB-stat embeddings in every split, and
73
  the three-seed `base_context_obs` aggregate reaches PPTC@0.20 = 0.2464,
74
  PPTC@0.40 = 0.6425, NegativeNear@0.20 = 0.0343, and mean positive distance =
@@ -519,6 +520,7 @@ Resolved in this pass:
519
  - `paper/sections/theory.tex`
520
  - `paper/notes/theory_ctt.md`
521
  - `runs/tangent_reconstruction/*`
 
522
  - `runs/utility_energy_smoke/*`
523
  - `runs/ctt_gated_residual_smoke/*`
524
  - `runs/ctt_gated_residual_smoke_proxy/*`
@@ -573,6 +575,12 @@ The following rows are diagnostic/baseline evidence, not the main method:
573
  Current pass:
574
 
575
  - Train/validation/test chart DB audit passes with zero violations.
 
 
 
 
 
 
576
  - Validation/test indexes are evaluator-only and retrieval-forbidden. They now
577
  contain measured outcomes for metric scripts; deployment-time proposal and
578
  selection code must not load those fields.
 
68
  PPTC@0.40 = 0.6232, NegativeNear@0.20 = 0.0182, and mean positive distance =
69
  0.4429 on 69 validation charts. The new non-destructive RGB-ref export fixes the
70
  source-input problem for a simple visual-stat diagnostic:
71
+ `data/cil_charts_rgb_refs` passes leakage audit with zero violations and zero
72
+ hash-artifact warnings, `runs/chart_feature_audit_rgb_refs`
73
  shows 100% observation refs and 100% 32D RGB-stat embeddings in every split, and
74
  the three-seed `base_context_obs` aggregate reaches PPTC@0.20 = 0.2464,
75
  PPTC@0.40 = 0.6425, NegativeNear@0.20 = 0.0343, and mean positive distance =
 
520
  - `paper/sections/theory.tex`
521
  - `paper/notes/theory_ctt.md`
522
  - `runs/tangent_reconstruction/*`
523
+ - `runs/tangent_reconstruction_rgb_refs/*`
524
  - `runs/utility_energy_smoke/*`
525
  - `runs/ctt_gated_residual_smoke/*`
526
  - `runs/ctt_gated_residual_smoke_proxy/*`
 
575
  Current pass:
576
 
577
  - Train/validation/test chart DB audit passes with zero violations.
578
+ - The original and RGB-ref leakage reports now pass with zero warnings after
579
+ accepting valid hash sidecars and refreshing the legacy val16 comparison
580
+ artifact.
581
+ - Tangent reconstruction checks pass for both original and RGB-ref chart DBs:
582
+ 45,968 rows each, zero failures, max absolute error 0.0, with top-level
583
+ data/split hash manifests.
584
  - Validation/test indexes are evaluator-only and retrieval-forbidden. They now
585
  contain measured outcomes for metric scripts; deployment-time proposal and
586
  selection code must not load those fields.