auto-sync 2026-07-04T02:06:24Z workspace (part 3)
Browse files
workspace/reports/2026-07-03.md
CHANGED
|
@@ -54,6 +54,9 @@
|
|
| 54 |
- Extended `cil/chart_features.py` with `base_context_obj` and `base_context_obs_obj` so CTT can test object-layout-only and RGB-stat-plus-object-layout chart tokens.
|
| 55 |
- Updated `scripts/audit_chart_feature_sources.py` to audit `object_embedding_path` coverage and report feature dimensions for the new object-layout modes.
|
| 56 |
- Updated selector/comparison scripts so new metrics artifacts expose top-level `data_hash` and `split_hash`.
|
|
|
|
|
|
|
|
|
|
| 57 |
- Updated CTT loaders to attach `_chart_root` metadata so relative observation-embedding paths resolve correctly in train, proxy, rollout, utility-energy, and dominance code paths.
|
| 58 |
- Added `scripts/slurm/train_ctt_feature_proxy.sbatch` for feature-mode CTT proxy sweeps; it trains residual CTT and refreshes the proxy comparison/summary artifacts for a specified chart feature mode.
|
| 59 |
- Fixed validation/test dominance selectors to load chart maps with the checkpoint's recorded chart feature mode. This prevents 112D `base` features from being fed into 162D `base_context_obs` checkpoints when row-scored rollouts still need checkpoint base scores.
|
|
@@ -585,7 +588,8 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
|
|
| 585 |
- `data/cil_charts/test`: outcomes visible only to evaluator, retrieval forbidden.
|
| 586 |
- Cross-split chart/state overlap audit: pass.
|
| 587 |
- Expert branches remain excluded from deployment candidates by index contract.
|
| 588 |
-
- `
|
|
|
|
| 589 |
- `data/cil_charts_rgb_refs/{train,val,test}` after object-layout export: same split/audience contract, leakage audit pass with 0 violations in `runs/leakage_audit_rgb_refs_object`, and 100% object-layout embedding coverage in `runs/chart_feature_audit_rgb_refs_object`.
|
| 590 |
|
| 591 |
## Data Accounting Status
|
|
@@ -598,6 +602,7 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
|
|
| 598 |
- `runs/chart_feature_audit` documents the original `data/cil_charts` inputs: instruction and scene ids exist, but observation embeddings/raw refs do not.
|
| 599 |
- `runs/chart_feature_audit_rgb_refs` documents the new non-destructive RGB-ref export: all splits have observation refs and deterministic 32D RGB-stat embeddings. This supports visual-stat diagnostics only; it does not yet support a learned visual-language/object-centric claim.
|
| 600 |
- `runs/chart_feature_audit_rgb_refs_object` documents the deterministic 64D RGB object-layout export. The full proxy rows and measured validation rollout are complete; the measured result is negative relative to RGB-stat support.
|
|
|
|
| 601 |
|
| 602 |
## Paper Changes
|
| 603 |
|
|
@@ -628,6 +633,7 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
|
|
| 628 |
- Updated the abstract with the latest visual-stat test support numbers, clipped-convention train-calibrated selected-success number (`31.25%`), and bounded-tanh train-calibrated selected-success number (`38.19%`), while keeping the conclusion diagnostic.
|
| 629 |
- Added `scripts/eval_nonlinear_dominance_selector.py` and summary support for nonlinear train-calibrated selector diagnostics. The paper/report keep these as negative selector evidence because the best nonlinear row reaches only `30.56%`.
|
| 630 |
- Updated limitations: current evidence includes validation/test measured rollout, but selected-action deployment remains unsolved; real-robot recovery, unsafe-contact measurement, and dominance-calibrated deployment remain missing.
|
|
|
|
| 631 |
- Added local-atlas/proxy-comparison scripts to the reproducibility artifact list.
|
| 632 |
- Added generated-rollout script and Slurm wrapper to the reproducibility artifact list.
|
| 633 |
- Added no-clipping validation/test aggregate tables via `\input{../runs/ctt_base_context_obs_val_noclip_rollout_comparison/table}` and `\input{../runs/ctt_base_context_obs_test_noclip_rollout_comparison/table}`. The draft now states that raw-action replay confirms support remains measurable but exposes action scaling/decoding as a blocker.
|
|
|
|
| 54 |
- Extended `cil/chart_features.py` with `base_context_obj` and `base_context_obs_obj` so CTT can test object-layout-only and RGB-stat-plus-object-layout chart tokens.
|
| 55 |
- Updated `scripts/audit_chart_feature_sources.py` to audit `object_embedding_path` coverage and report feature dimensions for the new object-layout modes.
|
| 56 |
- Updated selector/comparison scripts so new metrics artifacts expose top-level `data_hash` and `split_hash`.
|
| 57 |
+
- Updated `scripts/check_tangent_reconstruction.py` so reconstruction artifacts expose structured top-level `data_hash` and `split_hash` manifests in `metrics.json`, not only sidecar text files.
|
| 58 |
+
- Updated `scripts/audit_cil_charts.py` so the run-hash audit accepts either top-level `metrics.json` hashes or nonempty `data_hash.txt`/`split_hash.txt` sidecars, which matches the required run artifact contract.
|
| 59 |
+
- Updated `scripts/build_ctt_rollout_comparison.py` so measured rollout comparison summaries expose top-level `data_hash` and `split_hash`, then refreshed the legacy `runs/ctt_val16_rollout_comparison` artifact.
|
| 60 |
- Updated CTT loaders to attach `_chart_root` metadata so relative observation-embedding paths resolve correctly in train, proxy, rollout, utility-energy, and dominance code paths.
|
| 61 |
- Added `scripts/slurm/train_ctt_feature_proxy.sbatch` for feature-mode CTT proxy sweeps; it trains residual CTT and refreshes the proxy comparison/summary artifacts for a specified chart feature mode.
|
| 62 |
- Fixed validation/test dominance selectors to load chart maps with the checkpoint's recorded chart feature mode. This prevents 112D `base` features from being fed into 162D `base_context_obs` checkpoints when row-scored rollouts still need checkpoint base scores.
|
|
|
|
| 588 |
- `data/cil_charts/test`: outcomes visible only to evaluator, retrieval forbidden.
|
| 589 |
- Cross-split chart/state overlap audit: pass.
|
| 590 |
- Expert branches remain excluded from deployment candidates by index contract.
|
| 591 |
+
- `runs/leakage_audit` and `runs/leakage_audit_rgb_refs` now pass with 0 violations and 0 warnings after accepting valid hash sidecars and refreshing the legacy val16 comparison artifact.
|
| 592 |
+
- `data/cil_charts_rgb_refs/{train,val,test}`: same split/audience contract, RGB-ref leakage audit pass with 0 violations/0 warnings, and 100% observation-ref plus 100% RGB-stat embedding coverage in `runs/chart_feature_audit_rgb_refs`.
|
| 593 |
- `data/cil_charts_rgb_refs/{train,val,test}` after object-layout export: same split/audience contract, leakage audit pass with 0 violations in `runs/leakage_audit_rgb_refs_object`, and 100% object-layout embedding coverage in `runs/chart_feature_audit_rgb_refs_object`.
|
| 594 |
|
| 595 |
## Data Accounting Status
|
|
|
|
| 602 |
- `runs/chart_feature_audit` documents the original `data/cil_charts` inputs: instruction and scene ids exist, but observation embeddings/raw refs do not.
|
| 603 |
- `runs/chart_feature_audit_rgb_refs` documents the new non-destructive RGB-ref export: all splits have observation refs and deterministic 32D RGB-stat embeddings. This supports visual-stat diagnostics only; it does not yet support a learned visual-language/object-centric claim.
|
| 604 |
- `runs/chart_feature_audit_rgb_refs_object` documents the deterministic 64D RGB object-layout export. The full proxy rows and measured validation rollout are complete; the measured result is negative relative to RGB-stat support.
|
| 605 |
+
- `runs/tangent_reconstruction` and `runs/tangent_reconstruction_rgb_refs` verify 45,968 exported spline tangent rows each with 0 failures, max absolute error `0.0`, and structured data/split hash manifests in `metrics.json`.
|
| 606 |
|
| 607 |
## Paper Changes
|
| 608 |
|
|
|
|
| 633 |
- Updated the abstract with the latest visual-stat test support numbers, clipped-convention train-calibrated selected-success number (`31.25%`), and bounded-tanh train-calibrated selected-success number (`38.19%`), while keeping the conclusion diagnostic.
|
| 634 |
- Added `scripts/eval_nonlinear_dominance_selector.py` and summary support for nonlinear train-calibrated selector diagnostics. The paper/report keep these as negative selector evidence because the best nonlinear row reaches only `30.56%`.
|
| 635 |
- Updated limitations: current evidence includes validation/test measured rollout, but selected-action deployment remains unsolved; real-robot recovery, unsafe-contact measurement, and dominance-calibrated deployment remain missing.
|
| 636 |
+
- Added explicit tangent reconstruction and hash-audited leakage evidence to the reproducibility story: both original and RGB-ref chart DBs now have 45,968-row deterministic tangent checks with zero failures, and leakage audits report 0 violations/0 warnings.
|
| 637 |
- Added local-atlas/proxy-comparison scripts to the reproducibility artifact list.
|
| 638 |
- Added generated-rollout script and Slurm wrapper to the reproducibility artifact list.
|
| 639 |
- Added no-clipping validation/test aggregate tables via `\input{../runs/ctt_base_context_obs_val_noclip_rollout_comparison/table}` and `\input{../runs/ctt_base_context_obs_test_noclip_rollout_comparison/table}`. The draft now states that raw-action replay confirms support remains measurable but exposes action scaling/decoding as a blocker.
|
workspace/reports/draft_audit_ctt.md
CHANGED
|
@@ -68,7 +68,8 @@ hidden evaluator fields. Its one-seed proxy artifact,
|
|
| 68 |
PPTC@0.40 = 0.6232, NegativeNear@0.20 = 0.0182, and mean positive distance =
|
| 69 |
0.4429 on 69 validation charts. The new non-destructive RGB-ref export fixes the
|
| 70 |
source-input problem for a simple visual-stat diagnostic:
|
| 71 |
-
`data/cil_charts_rgb_refs` passes leakage audit
|
|
|
|
| 72 |
shows 100% observation refs and 100% 32D RGB-stat embeddings in every split, and
|
| 73 |
the three-seed `base_context_obs` aggregate reaches PPTC@0.20 = 0.2464,
|
| 74 |
PPTC@0.40 = 0.6425, NegativeNear@0.20 = 0.0343, and mean positive distance =
|
|
@@ -519,6 +520,7 @@ Resolved in this pass:
|
|
| 519 |
- `paper/sections/theory.tex`
|
| 520 |
- `paper/notes/theory_ctt.md`
|
| 521 |
- `runs/tangent_reconstruction/*`
|
|
|
|
| 522 |
- `runs/utility_energy_smoke/*`
|
| 523 |
- `runs/ctt_gated_residual_smoke/*`
|
| 524 |
- `runs/ctt_gated_residual_smoke_proxy/*`
|
|
@@ -573,6 +575,12 @@ The following rows are diagnostic/baseline evidence, not the main method:
|
|
| 573 |
Current pass:
|
| 574 |
|
| 575 |
- Train/validation/test chart DB audit passes with zero violations.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 576 |
- Validation/test indexes are evaluator-only and retrieval-forbidden. They now
|
| 577 |
contain measured outcomes for metric scripts; deployment-time proposal and
|
| 578 |
selection code must not load those fields.
|
|
|
|
| 68 |
PPTC@0.40 = 0.6232, NegativeNear@0.20 = 0.0182, and mean positive distance =
|
| 69 |
0.4429 on 69 validation charts. The new non-destructive RGB-ref export fixes the
|
| 70 |
source-input problem for a simple visual-stat diagnostic:
|
| 71 |
+
`data/cil_charts_rgb_refs` passes leakage audit with zero violations and zero
|
| 72 |
+
hash-artifact warnings, `runs/chart_feature_audit_rgb_refs`
|
| 73 |
shows 100% observation refs and 100% 32D RGB-stat embeddings in every split, and
|
| 74 |
the three-seed `base_context_obs` aggregate reaches PPTC@0.20 = 0.2464,
|
| 75 |
PPTC@0.40 = 0.6425, NegativeNear@0.20 = 0.0343, and mean positive distance =
|
|
|
|
| 520 |
- `paper/sections/theory.tex`
|
| 521 |
- `paper/notes/theory_ctt.md`
|
| 522 |
- `runs/tangent_reconstruction/*`
|
| 523 |
+
- `runs/tangent_reconstruction_rgb_refs/*`
|
| 524 |
- `runs/utility_energy_smoke/*`
|
| 525 |
- `runs/ctt_gated_residual_smoke/*`
|
| 526 |
- `runs/ctt_gated_residual_smoke_proxy/*`
|
|
|
|
| 575 |
Current pass:
|
| 576 |
|
| 577 |
- Train/validation/test chart DB audit passes with zero violations.
|
| 578 |
+
- The original and RGB-ref leakage reports now pass with zero warnings after
|
| 579 |
+
accepting valid hash sidecars and refreshing the legacy val16 comparison
|
| 580 |
+
artifact.
|
| 581 |
+
- Tangent reconstruction checks pass for both original and RGB-ref chart DBs:
|
| 582 |
+
45,968 rows each, zero failures, max absolute error 0.0, with top-level
|
| 583 |
+
data/split hash manifests.
|
| 584 |
- Validation/test indexes are evaluator-only and retrieval-forbidden. They now
|
| 585 |
contain measured outcomes for metric scripts; deployment-time proposal and
|
| 586 |
selection code must not load those fields.
|