workspace/docs/cil_format.md

# Counterfactual Action Atlas / CILBench Format

CILBench stores same-state do-action alternatives. A `CILBenchGroup` is a local
action chart: the simulator state, observation, and instruction are held fixed,
then multiple action branches are executed from that identical state.

Core group fields:

- `group_id`
- `task_id`
- `split_id`
- `simulator_name`
- `simulator_state_hash`
- `random_seed`
- `instruction`
- `observation_ref` or `observation_inline`
- `scene_metadata`
- `anchor_policy`
- `branches`
- `metadata`

Each branch stores:

- `branch_id`
- `action`
- `action_representation`
- `branch_family`
- `outcome`
- `rollout_trace_ref` or `rollout_trace_inline`
- `train_only`
- `metadata`

The outcome vector is:

```text
[success, progress, contact_quality, safety_violation,
 task_stage_quality, smoothness, recovery]
```

The scalar utility used for ranking is a view over this vector. Reports should
publish the components separately, then use the scalar only for training,
ranking, CAR, and oracle computations.

Benchmark tracks:

- Ranking: choose among measured branches without seeing outcomes.
- Generation: propose new action tangents from the same observation and instruction.
- Deployment: execute deployment-clean proposals without same-state validation rewards.
- Recovery: evaluate near-miss states and safety/abstention behavior.

Canonical branch families:

- `anchor` / `base_policy`
- `expert_train_only`
- `residual_transport`
- `object_centric_geometric`
- `recovery_tangent`
- `negative_antigoal`
- `learned_generator`

Generator baselines:

- V0 transported residual retrieval copies train-state tangents into the current
  state and lets the learned utility field select among them.
- V1 utility-weighted residual retrieval changes proposal support before
  selection by weighting train tangents with retrieval affinity and measured
  source advantage, `exp(-distance / tau + rho * delta_utility)`.
- The full CIL-Atlas generator should learn positive causal tangent support
  directly, with V0/V1 kept as diagnostic baselines rather than the main novelty.

Learned generator targets:

- `scripts/export_positive_tangent_targets.py` exports `delta_action = action - base`
  for each same-state group.
- The base priority is deployment-clean (`policy`, `policy_residual`, `anchor`,
  `base`) with `expert` as a final fallback for current train-only CIL shards.
- Labels are measured from same-state utility contrast:
  `positive` if `U(action) > U(base) + epsilon`, `negative` if below the base by
  `epsilon`, and `neutral` otherwise.
- Each target includes a low-dimensional `tangent_summary.spline_code` with
  endpoint, midpoint, gripper, lift, and approach-axis summaries. This is the
  first training artifact for V2/V3 CVAE or flow-matching generators in
  object-centric spline tangent space.
- `scripts/eval_positive_tangent_memory.py` is a leakage-checked diagnostic
  baseline: train-only positive tangents become diverse task-level prototypes,
  and heldout groups report PTR proxy, negative-near rate, and whether the
  nearest proposal is closer to a hidden positive than to hidden negatives. It
  should be reported as support evidence, not as the final learned generator.
- `scripts/eval_positive_tangent_local_atlas.py` evaluates local chart-neighborhood
  reuse: for each heldout chart it retrieves train-only positive tangents from
  nearby observation-language-task charts. This is not the final generator, but
  it tests whether positive support is locally organized in the atlas. It can
  also score candidates by distance from local train negatives, which is a
  diagnostic for whether negative boundaries alone can replace local positive
  support. Current K16 support-proxy results say no: pool16 local positives
  reach 23.66% PTR at RMS<=0.20 and 52.69% at RMS<=0.40, while pool64
  negative-margin reranking keeps or lowers strict PTR and does not reduce the
  5.33% strict negative-near rate.
- `scripts/eval_positive_tangent_chart_synthesis.py` is the next local-chart
  diagnostic. It keeps train-only positive chart atoms and adds barycentric
  means over nearby chart neighborhoods, testing whether positive support is
  better expressed as local chart coordinates than as raw prototype replay.
  Current best keeps 15 direct local atoms plus one 16-neighbor chart mean,
  preserving K16 PTR at 23.66% / 52.69% and strict negative-near at 5.33% while
  improving positives-closer-than-negatives to 65.33%. This remains an offline
  support proxy; successful settings should motivate the learned object-centric
  atlas generator rather than become the final deployment method by themselves.
- `scripts/train_positive_tangent_cvae.py` trains a first raw-action CVAE over
  train-only positive tangents. The companion
  `scripts/summarize_positive_tangent_cvae_sweep.py` ranks temperature/beta
  sweeps by heldout PTR and negative-near rates. This baseline is useful for
  falsifying raw action-chunk likelihood as the final geometry.
- `scripts/train_positive_tangent_spline_cvae.py` trains a keyframe-spline CVAE
  over start/midpoint/endpoint residual codes and decodes samples back into
  full action chunks before evaluation. This is the first chart-geometry
  baseline and a bridge toward object-centric flow matching.
- `scripts/train_positive_tangent_spline_flow.py` trains conditional flow
  matching in the same keyframe tangent space. Current results should be read
  as a falsification of unguided density flow: useful as a stepping stone, but
  not the final utility-guided atlas generator.
- `scripts/train_positive_tangent_guided_spline_flow.py` adds the missing
  contrastive part: a train-only positive-vs-negative utility head guides the
  same spline flow during sampling. This tests whether the atlas should be a
  learned local causal utility field over tangent geometry, not just a density
  model over positive examples.

JSONL shards should preserve complete groups. A group may exceed the target shard size, but it
should never be split across shards unless an explicit future streaming mode opts into that tradeoff.