anhtld commited on
Commit
664892c
·
verified ·
1 Parent(s): 5d230c4

ctt artifacts 2026-07-02 workspace/docs/cil_format.md

Browse files
Files changed (1) hide show
  1. workspace/docs/cil_format.md +25 -16
workspace/docs/cil_format.md CHANGED
@@ -43,10 +43,15 @@ ranking, CAR, and oracle computations.
43
 
44
  Canonical metric API:
45
 
46
- - `cil/metrics.py` defines Branch CAR, PTR@K, SelectorRegret@K, SupportGap,
47
- support/selector CAR decomposition, negative-near rate,
 
48
  positives-closer-than-negatives, pairwise causal dominance ECE, and
49
  micro/task-macro/seed-macro bootstrap summaries.
 
 
 
 
50
  - `runs/reproduce_v0/metrics.json` is the current Phase-0 provenance bundle:
51
  git hash, data hash, split hash, input file hashes, gate status, support
52
  proxy rows, and leakage-audit summary.
@@ -68,15 +73,18 @@ Canonical branch families:
68
  - `negative_antigoal`
69
  - `learned_generator`
70
 
71
- Generator baselines:
72
 
73
  - V0 transported residual retrieval copies train-state tangents into the current
74
  state and lets the learned utility field select among them.
75
  - V1 utility-weighted residual retrieval changes proposal support before
76
  selection by weighting train tangents with retrieval affinity and measured
77
  source advantage, `exp(-distance / tau + rho * delta_utility)`.
78
- - The full CIL-Atlas generator should learn positive causal tangent support
79
- directly, with V0/V1 kept as diagnostic baselines rather than the main novelty.
 
 
 
80
 
81
  Learned generator targets:
82
 
@@ -86,11 +94,12 @@ Learned generator targets:
86
  `data/cil_charts/{split}/charts_*.npz` plus `index.json`. The train export
87
  is the only retrieval/generator-training index; non-train exports must be
88
  evaluator-only.
89
- - `scripts/audit_leakage.py` checks chart indexes for split misuse,
90
  non-train outcome exposure, missing shards, and same-state group/state-hash
91
- overlap between train and eval splits. The current train chart DB contains
92
- 2,873 charts and 45,968 rows including base branches, and its audit passes
93
- with zero violations.
 
94
  - The base priority is deployment-clean (`policy`, `policy_residual`, `anchor`,
95
  `base`) with `expert` as a final fallback for current train-only CIL shards.
96
  - Labels are measured from same-state utility contrast:
@@ -102,9 +111,9 @@ Learned generator targets:
102
  object-centric spline tangent space.
103
  - `scripts/eval_positive_tangent_memory.py` is a leakage-checked diagnostic
104
  baseline: train-only positive tangents become diverse task-level prototypes,
105
- and heldout groups report PTR proxy, negative-near rate, and whether the
106
- nearest proposal is closer to a hidden positive than to hidden negatives. It
107
- should be reported as support evidence, not as the final learned generator.
108
  - `scripts/eval_positive_tangent_local_atlas.py` evaluates local chart-neighborhood
109
  reuse: for each heldout chart it retrieves train-only positive tangents from
110
  nearby observation-language-task charts. This is not the final generator, but
@@ -112,22 +121,22 @@ Learned generator targets:
112
  also score candidates by distance from local train negatives, which is a
113
  diagnostic for whether negative boundaries alone can replace local positive
114
  support. Current K16 support-proxy results say no: pool16 local positives
115
- reach 23.66% PTR at RMS<=0.20 and 52.69% at RMS<=0.40, while pool64
116
- negative-margin reranking keeps or lowers strict PTR and does not reduce the
117
  5.33% strict negative-near rate.
118
  - `scripts/eval_positive_tangent_chart_synthesis.py` is the next local-chart
119
  diagnostic. It keeps train-only positive chart atoms and adds barycentric
120
  means over nearby chart neighborhoods, testing whether positive support is
121
  better expressed as local chart coordinates than as raw prototype replay.
122
  Current best keeps 15 direct local atoms plus one 16-neighbor chart mean,
123
- preserving K16 PTR at 23.66% / 52.69% and strict negative-near at 5.33% while
124
  improving positives-closer-than-negatives to 65.33%. This remains an offline
125
  support proxy; successful settings should motivate the learned object-centric
126
  atlas generator rather than become the final deployment method by themselves.
127
  - `scripts/train_positive_tangent_cvae.py` trains a first raw-action CVAE over
128
  train-only positive tangents. The companion
129
  `scripts/summarize_positive_tangent_cvae_sweep.py` ranks temperature/beta
130
- sweeps by heldout PTR and negative-near rates. This baseline is useful for
131
  falsifying raw action-chunk likelihood as the final geometry.
132
  - `scripts/train_positive_tangent_spline_cvae.py` trains a keyframe-spline CVAE
133
  over start/midpoint/endpoint residual codes and decodes samples back into
 
43
 
44
  Canonical metric API:
45
 
46
+ - `cil/metrics.py` defines BranchCAR, measured OutcomePTR@K,
47
+ SelectorRegret@K, measured SupportGap, support/selector CAR decomposition,
48
+ ProxyPositiveTangentCoverage@K (PPTC@K), negative-near rate,
49
  positives-closer-than-negatives, pairwise causal dominance ECE, and
50
  micro/task-macro/seed-macro bootstrap summaries.
51
+ - Distance-only support diagnostics must be reported as PPTC, never as PTR or
52
+ OutcomePTR. `scripts/eval_metrics.py --mode measured` refuses rows without
53
+ `candidates_evaluated=true`; `--mode proxy` exports PPTC/NegativeNear/proxy
54
+ support distance.
55
  - `runs/reproduce_v0/metrics.json` is the current Phase-0 provenance bundle:
56
  git hash, data hash, split hash, input file hashes, gate status, support
57
  proxy rows, and leakage-audit summary.
 
73
  - `negative_antigoal`
74
  - `learned_generator`
75
 
76
+ Generator baselines and CTT:
77
 
78
  - V0 transported residual retrieval copies train-state tangents into the current
79
  state and lets the learned utility field select among them.
80
  - V1 utility-weighted residual retrieval changes proposal support before
81
  selection by weighting train tangents with retrieval affinity and measured
82
  source advantage, `exp(-distance / tau + rho * delta_utility)`.
83
+ - Causal Tangent Transport (CTT) is the current method spine:
84
+ `T_phi(z_source, z_target, xi_source_positive) -> xi_target_positive`.
85
+ CTT starts from measured train positive tangents and transports them into the
86
+ target chart. V0/V1/CVAE/flow rows remain diagnostic baselines rather than the
87
+ main novelty.
88
 
89
  Learned generator targets:
90
 
 
94
  `data/cil_charts/{split}/charts_*.npz` plus `index.json`. The train export
95
  is the only retrieval/generator-training index; non-train exports must be
96
  evaluator-only.
97
+ - `scripts/audit_cil_charts.py` checks chart indexes for split misuse,
98
  non-train outcome exposure, missing shards, and same-state group/state-hash
99
+ overlap between train and eval splits. The current split export contains
100
+ 2,873 charts and 45,968 rows including base branches. The train retrieval DB
101
+ exposes 2,044 charts and 32,704 rows; validation/test are evaluator-only.
102
+ The current audit passes with zero violations.
103
  - The base priority is deployment-clean (`policy`, `policy_residual`, `anchor`,
104
  `base`) with `expert` as a final fallback for current train-only CIL shards.
105
  - Labels are measured from same-state utility contrast:
 
111
  object-centric spline tangent space.
112
  - `scripts/eval_positive_tangent_memory.py` is a leakage-checked diagnostic
113
  baseline: train-only positive tangents become diverse task-level prototypes,
114
+ and heldout groups report PPTC, negative-near rate, and whether the nearest
115
+ proposal is closer to a hidden positive than to hidden negatives. It should be
116
+ reported as support evidence, not as the final learned generator.
117
  - `scripts/eval_positive_tangent_local_atlas.py` evaluates local chart-neighborhood
118
  reuse: for each heldout chart it retrieves train-only positive tangents from
119
  nearby observation-language-task charts. This is not the final generator, but
 
121
  also score candidates by distance from local train negatives, which is a
122
  diagnostic for whether negative boundaries alone can replace local positive
123
  support. Current K16 support-proxy results say no: pool16 local positives
124
+ reach 23.66% PPTC at RMS<=0.20 and 52.69% at RMS<=0.40, while pool64
125
+ negative-margin reranking keeps or lowers strict PPTC and does not reduce the
126
  5.33% strict negative-near rate.
127
  - `scripts/eval_positive_tangent_chart_synthesis.py` is the next local-chart
128
  diagnostic. It keeps train-only positive chart atoms and adds barycentric
129
  means over nearby chart neighborhoods, testing whether positive support is
130
  better expressed as local chart coordinates than as raw prototype replay.
131
  Current best keeps 15 direct local atoms plus one 16-neighbor chart mean,
132
+ preserving K16 PPTC at 23.66% / 52.69% and strict negative-near at 5.33% while
133
  improving positives-closer-than-negatives to 65.33%. This remains an offline
134
  support proxy; successful settings should motivate the learned object-centric
135
  atlas generator rather than become the final deployment method by themselves.
136
  - `scripts/train_positive_tangent_cvae.py` trains a first raw-action CVAE over
137
  train-only positive tangents. The companion
138
  `scripts/summarize_positive_tangent_cvae_sweep.py` ranks temperature/beta
139
+ sweeps by heldout PPTC and negative-near rates. This baseline is useful for
140
  falsifying raw action-chunk likelihood as the final geometry.
141
  - `scripts/train_positive_tangent_spline_cvae.py` trains a keyframe-spline CVAE
142
  over start/midpoint/endpoint residual codes and decodes samples back into