ctt artifacts 2026-07-02 workspace/docs/cil_format.md
Browse files- workspace/docs/cil_format.md +25 -16
workspace/docs/cil_format.md
CHANGED
|
@@ -43,10 +43,15 @@ ranking, CAR, and oracle computations.
|
|
| 43 |
|
| 44 |
Canonical metric API:
|
| 45 |
|
| 46 |
-
- `cil/metrics.py` defines
|
| 47 |
-
support/selector CAR decomposition,
|
|
|
|
| 48 |
positives-closer-than-negatives, pairwise causal dominance ECE, and
|
| 49 |
micro/task-macro/seed-macro bootstrap summaries.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
- `runs/reproduce_v0/metrics.json` is the current Phase-0 provenance bundle:
|
| 51 |
git hash, data hash, split hash, input file hashes, gate status, support
|
| 52 |
proxy rows, and leakage-audit summary.
|
|
@@ -68,15 +73,18 @@ Canonical branch families:
|
|
| 68 |
- `negative_antigoal`
|
| 69 |
- `learned_generator`
|
| 70 |
|
| 71 |
-
Generator baselines:
|
| 72 |
|
| 73 |
- V0 transported residual retrieval copies train-state tangents into the current
|
| 74 |
state and lets the learned utility field select among them.
|
| 75 |
- V1 utility-weighted residual retrieval changes proposal support before
|
| 76 |
selection by weighting train tangents with retrieval affinity and measured
|
| 77 |
source advantage, `exp(-distance / tau + rho * delta_utility)`.
|
| 78 |
-
-
|
| 79 |
-
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
Learned generator targets:
|
| 82 |
|
|
@@ -86,11 +94,12 @@ Learned generator targets:
|
|
| 86 |
`data/cil_charts/{split}/charts_*.npz` plus `index.json`. The train export
|
| 87 |
is the only retrieval/generator-training index; non-train exports must be
|
| 88 |
evaluator-only.
|
| 89 |
-
- `scripts/
|
| 90 |
non-train outcome exposure, missing shards, and same-state group/state-hash
|
| 91 |
-
overlap between train and eval splits. The current
|
| 92 |
-
2,873 charts and 45,968 rows including base branches
|
| 93 |
-
|
|
|
|
| 94 |
- The base priority is deployment-clean (`policy`, `policy_residual`, `anchor`,
|
| 95 |
`base`) with `expert` as a final fallback for current train-only CIL shards.
|
| 96 |
- Labels are measured from same-state utility contrast:
|
|
@@ -102,9 +111,9 @@ Learned generator targets:
|
|
| 102 |
object-centric spline tangent space.
|
| 103 |
- `scripts/eval_positive_tangent_memory.py` is a leakage-checked diagnostic
|
| 104 |
baseline: train-only positive tangents become diverse task-level prototypes,
|
| 105 |
-
and heldout groups report
|
| 106 |
-
|
| 107 |
-
|
| 108 |
- `scripts/eval_positive_tangent_local_atlas.py` evaluates local chart-neighborhood
|
| 109 |
reuse: for each heldout chart it retrieves train-only positive tangents from
|
| 110 |
nearby observation-language-task charts. This is not the final generator, but
|
|
@@ -112,22 +121,22 @@ Learned generator targets:
|
|
| 112 |
also score candidates by distance from local train negatives, which is a
|
| 113 |
diagnostic for whether negative boundaries alone can replace local positive
|
| 114 |
support. Current K16 support-proxy results say no: pool16 local positives
|
| 115 |
-
reach 23.66%
|
| 116 |
-
negative-margin reranking keeps or lowers strict
|
| 117 |
5.33% strict negative-near rate.
|
| 118 |
- `scripts/eval_positive_tangent_chart_synthesis.py` is the next local-chart
|
| 119 |
diagnostic. It keeps train-only positive chart atoms and adds barycentric
|
| 120 |
means over nearby chart neighborhoods, testing whether positive support is
|
| 121 |
better expressed as local chart coordinates than as raw prototype replay.
|
| 122 |
Current best keeps 15 direct local atoms plus one 16-neighbor chart mean,
|
| 123 |
-
preserving K16
|
| 124 |
improving positives-closer-than-negatives to 65.33%. This remains an offline
|
| 125 |
support proxy; successful settings should motivate the learned object-centric
|
| 126 |
atlas generator rather than become the final deployment method by themselves.
|
| 127 |
- `scripts/train_positive_tangent_cvae.py` trains a first raw-action CVAE over
|
| 128 |
train-only positive tangents. The companion
|
| 129 |
`scripts/summarize_positive_tangent_cvae_sweep.py` ranks temperature/beta
|
| 130 |
-
sweeps by heldout
|
| 131 |
falsifying raw action-chunk likelihood as the final geometry.
|
| 132 |
- `scripts/train_positive_tangent_spline_cvae.py` trains a keyframe-spline CVAE
|
| 133 |
over start/midpoint/endpoint residual codes and decodes samples back into
|
|
|
|
| 43 |
|
| 44 |
Canonical metric API:
|
| 45 |
|
| 46 |
+
- `cil/metrics.py` defines BranchCAR, measured OutcomePTR@K,
|
| 47 |
+
SelectorRegret@K, measured SupportGap, support/selector CAR decomposition,
|
| 48 |
+
ProxyPositiveTangentCoverage@K (PPTC@K), negative-near rate,
|
| 49 |
positives-closer-than-negatives, pairwise causal dominance ECE, and
|
| 50 |
micro/task-macro/seed-macro bootstrap summaries.
|
| 51 |
+
- Distance-only support diagnostics must be reported as PPTC, never as PTR or
|
| 52 |
+
OutcomePTR. `scripts/eval_metrics.py --mode measured` refuses rows without
|
| 53 |
+
`candidates_evaluated=true`; `--mode proxy` exports PPTC/NegativeNear/proxy
|
| 54 |
+
support distance.
|
| 55 |
- `runs/reproduce_v0/metrics.json` is the current Phase-0 provenance bundle:
|
| 56 |
git hash, data hash, split hash, input file hashes, gate status, support
|
| 57 |
proxy rows, and leakage-audit summary.
|
|
|
|
| 73 |
- `negative_antigoal`
|
| 74 |
- `learned_generator`
|
| 75 |
|
| 76 |
+
Generator baselines and CTT:
|
| 77 |
|
| 78 |
- V0 transported residual retrieval copies train-state tangents into the current
|
| 79 |
state and lets the learned utility field select among them.
|
| 80 |
- V1 utility-weighted residual retrieval changes proposal support before
|
| 81 |
selection by weighting train tangents with retrieval affinity and measured
|
| 82 |
source advantage, `exp(-distance / tau + rho * delta_utility)`.
|
| 83 |
+
- Causal Tangent Transport (CTT) is the current method spine:
|
| 84 |
+
`T_phi(z_source, z_target, xi_source_positive) -> xi_target_positive`.
|
| 85 |
+
CTT starts from measured train positive tangents and transports them into the
|
| 86 |
+
target chart. V0/V1/CVAE/flow rows remain diagnostic baselines rather than the
|
| 87 |
+
main novelty.
|
| 88 |
|
| 89 |
Learned generator targets:
|
| 90 |
|
|
|
|
| 94 |
`data/cil_charts/{split}/charts_*.npz` plus `index.json`. The train export
|
| 95 |
is the only retrieval/generator-training index; non-train exports must be
|
| 96 |
evaluator-only.
|
| 97 |
+
- `scripts/audit_cil_charts.py` checks chart indexes for split misuse,
|
| 98 |
non-train outcome exposure, missing shards, and same-state group/state-hash
|
| 99 |
+
overlap between train and eval splits. The current split export contains
|
| 100 |
+
2,873 charts and 45,968 rows including base branches. The train retrieval DB
|
| 101 |
+
exposes 2,044 charts and 32,704 rows; validation/test are evaluator-only.
|
| 102 |
+
The current audit passes with zero violations.
|
| 103 |
- The base priority is deployment-clean (`policy`, `policy_residual`, `anchor`,
|
| 104 |
`base`) with `expert` as a final fallback for current train-only CIL shards.
|
| 105 |
- Labels are measured from same-state utility contrast:
|
|
|
|
| 111 |
object-centric spline tangent space.
|
| 112 |
- `scripts/eval_positive_tangent_memory.py` is a leakage-checked diagnostic
|
| 113 |
baseline: train-only positive tangents become diverse task-level prototypes,
|
| 114 |
+
and heldout groups report PPTC, negative-near rate, and whether the nearest
|
| 115 |
+
proposal is closer to a hidden positive than to hidden negatives. It should be
|
| 116 |
+
reported as support evidence, not as the final learned generator.
|
| 117 |
- `scripts/eval_positive_tangent_local_atlas.py` evaluates local chart-neighborhood
|
| 118 |
reuse: for each heldout chart it retrieves train-only positive tangents from
|
| 119 |
nearby observation-language-task charts. This is not the final generator, but
|
|
|
|
| 121 |
also score candidates by distance from local train negatives, which is a
|
| 122 |
diagnostic for whether negative boundaries alone can replace local positive
|
| 123 |
support. Current K16 support-proxy results say no: pool16 local positives
|
| 124 |
+
reach 23.66% PPTC at RMS<=0.20 and 52.69% at RMS<=0.40, while pool64
|
| 125 |
+
negative-margin reranking keeps or lowers strict PPTC and does not reduce the
|
| 126 |
5.33% strict negative-near rate.
|
| 127 |
- `scripts/eval_positive_tangent_chart_synthesis.py` is the next local-chart
|
| 128 |
diagnostic. It keeps train-only positive chart atoms and adds barycentric
|
| 129 |
means over nearby chart neighborhoods, testing whether positive support is
|
| 130 |
better expressed as local chart coordinates than as raw prototype replay.
|
| 131 |
Current best keeps 15 direct local atoms plus one 16-neighbor chart mean,
|
| 132 |
+
preserving K16 PPTC at 23.66% / 52.69% and strict negative-near at 5.33% while
|
| 133 |
improving positives-closer-than-negatives to 65.33%. This remains an offline
|
| 134 |
support proxy; successful settings should motivate the learned object-centric
|
| 135 |
atlas generator rather than become the final deployment method by themselves.
|
| 136 |
- `scripts/train_positive_tangent_cvae.py` trains a first raw-action CVAE over
|
| 137 |
train-only positive tangents. The companion
|
| 138 |
`scripts/summarize_positive_tangent_cvae_sweep.py` ranks temperature/beta
|
| 139 |
+
sweeps by heldout PPTC and negative-near rates. This baseline is useful for
|
| 140 |
falsifying raw action-chunk likelihood as the final geometry.
|
| 141 |
- `scripts/train_positive_tangent_spline_cvae.py` trains a keyframe-spline CVAE
|
| 142 |
over start/midpoint/endpoint residual codes and decodes samples back into
|