workspace/docs/cil_format.md

Counterfactual Action Atlas / CILBench Format

CILBench stores same-state do-action alternatives. A CILBenchGroup is a local action chart: the simulator state, observation, and instruction are held fixed, then multiple action branches are executed from that identical state.

Core group fields:

• group_id
• task_id
• split_id
• simulator_name
• simulator_state_hash
• random_seed
• instruction
• observation_ref or observation_inline
• scene_metadata
• anchor_policy
• branches
• metadata

Each branch stores:

• branch_id
• action
• action_representation
• branch_family
• outcome
• rollout_trace_ref or rollout_trace_inline
• train_only
• metadata

The outcome vector is:

[success, progress, contact_quality, safety_violation,
 task_stage_quality, smoothness, recovery]

The scalar utility used for ranking is a view over this vector. Reports should publish the components separately, then use the scalar only for training, ranking, CAR, and oracle computations.

Benchmark tracks:

• Ranking: choose among measured branches without seeing outcomes.
• Generation: propose new action tangents from the same observation and instruction.
• Deployment: execute deployment-clean proposals without same-state validation rewards.
• Recovery: evaluate near-miss states and safety/abstention behavior.

Canonical branch families:

• anchor / base_policy
• expert_train_only
• residual_transport
• object_centric_geometric
• recovery_tangent
• negative_antigoal
• learned_generator

Generator baselines:

• V0 transported residual retrieval copies train-state tangents into the current state and lets the learned utility field select among them.
• V1 utility-weighted residual retrieval changes proposal support before selection by weighting train tangents with retrieval affinity and measured source advantage, exp(-distance / tau + rho * delta_utility).
• The full CIL-Atlas generator should learn positive causal tangent support directly, with V0/V1 kept as diagnostic baselines rather than the main novelty.

Learned generator targets:

• scripts/export_positive_tangent_targets.py exports delta_action = action - base for each same-state group.
• The base priority is deployment-clean (policy, policy_residual, anchor, base) with expert as a final fallback for current train-only CIL shards.
• Labels are measured from same-state utility contrast: positive if U(action) > U(base) + epsilon, negative if below the base by epsilon, and neutral otherwise.
• Each target includes a low-dimensional tangent_summary.spline_code with endpoint, midpoint, gripper, lift, and approach-axis summaries. This is the first training artifact for V2/V3 CVAE or flow-matching generators in object-centric spline tangent space.
• scripts/eval_positive_tangent_memory.py is a leakage-checked diagnostic baseline: train-only positive tangents become diverse task-level prototypes, and heldout groups report PTR proxy, negative-near rate, and whether the nearest proposal is closer to a hidden positive than to hidden negatives. It should be reported as support evidence, not as the final learned generator.
• scripts/eval_positive_tangent_local_atlas.py evaluates local chart-neighborhood reuse: for each heldout chart it retrieves train-only positive tangents from nearby observation-language-task charts. This is not the final generator, but it tests whether positive support is locally organized in the atlas. It can also score candidates by distance from local train negatives, which is a diagnostic for whether negative boundaries alone can replace local positive support. Current K16 support-proxy results say no: pool16 local positives reach 23.66% PTR at RMS<=0.20 and 52.69% at RMS<=0.40, while pool64 negative-margin reranking keeps or lowers strict PTR and does not reduce the 5.33% strict negative-near rate.
• scripts/eval_positive_tangent_chart_synthesis.py is the next local-chart diagnostic. It keeps train-only positive chart atoms and adds barycentric means over nearby chart neighborhoods, testing whether positive support is better expressed as local chart coordinates than as raw prototype replay. Current best keeps 15 direct local atoms plus one 16-neighbor chart mean, preserving K16 PTR at 23.66% / 52.69% and strict negative-near at 5.33% while improving positives-closer-than-negatives to 65.33%. This remains an offline support proxy; successful settings should motivate the learned object-centric atlas generator rather than become the final deployment method by themselves.
• scripts/train_positive_tangent_cvae.py trains a first raw-action CVAE over train-only positive tangents. The companion scripts/summarize_positive_tangent_cvae_sweep.py ranks temperature/beta sweeps by heldout PTR and negative-near rates. This baseline is useful for falsifying raw action-chunk likelihood as the final geometry.
• scripts/train_positive_tangent_spline_cvae.py trains a keyframe-spline CVAE over start/midpoint/endpoint residual codes and decodes samples back into full action chunks before evaluation. This is the first chart-geometry baseline and a bridge toward object-centric flow matching.
• scripts/train_positive_tangent_spline_flow.py trains conditional flow matching in the same keyframe tangent space. Current results should be read as a falsification of unguided density flow: useful as a stepping stone, but not the final utility-guided atlas generator.
• scripts/train_positive_tangent_guided_spline_flow.py adds the missing contrastive part: a train-only positive-vs-negative utility head guides the same spline flow during sampling. This tests whether the atlas should be a learned local causal utility field over tangent geometry, not just a density model over positive examples.

JSONL shards should preserve complete groups. A group may exceed the target shard size, but it should never be split across shards unless an explicit future streaming mode opts into that tradeoff.