# VLA / CIL-Atlas / Causal Tangent Transport

This repository is the working codebase for a robot-learning research project
around same-state counterfactual action charts and Causal Tangent Transport
(CTT).

The single spine of the project is:

> Standard VLA training observes one demonstrated action per state. CIL-Atlas
> restores the same state, executes multiple action chunks, and measures which
> local action tangents causally improve, recover, fail, collide, or succeed.
> CTT turns this measured local causal geometry into deployment-clean proposal
> generation by transporting measured positive do-action tangents from
> train-only neighboring charts into the target chart.

The current evidence is intentionally written as a diagnostic method paper, not
as an overclaimed final success. K=16 `env_clip` support is strong on held-out
test (`proposal_oracle_success = 0.5694`, `OutcomePTR@16 = 0.5486`), while the
selector/dominance side remains the bottleneck. The strongest current
train-clean K=16 selector reaches `selected_success = 0.3542` against a
`0.5694` proposal oracle, leaving a `0.2431` success selector gap. The
score-source LCB dominance fallback is safe under action-bound labels but
negative as a selector (`0.2778` auto, `0.2917` tau0). Additional K=8 tanh
and per-dimension trainmax selector sweeps completed on 2026-07-04; the best
tanh selector reaches `0.3819` selected success, but its proposal oracle is
only `0.3819`, so it does not replace the K=16 `env_clip` outcome gate.

All project-authored Markdown files outside vendored/tool caches were removed
and consolidated into this README. The canonical paper is `latex/main.tex` plus
`latex/main.pdf`; experiment evidence lives in JSON, TeX, logs, configs, and
command files under `runs/`.

## Research Goal

The paper target is not "a bigger stack." The target is a clean method story:

1. Same-state CIL charts define local do-action causal geometry.
2. Causal Action Regret decomposes deployment failure into support gap plus
   selector gap.
3. CTT proposes candidates by transporting measured train positive tangents,
   not by Gaussian noise or verifier optimization off support.
4. Utility energy and calibrated dominance decide whether a transported tangent
   should replace the base action.
5. Every main claim must have a method, implemented script/module, metric table,
   leakage audit, and reproducible run log.

Current strategic diagnosis:

| Area | Current status | Meaning |
| --- | --- | --- |
| Same-state chart data | Implemented and leakage-audited | Good scientific primitive |
| Metrics | Implemented with measured/proxy separation | OutcomePTR and PPTC are not confused |
| CTT residual transport | Implemented and measured | K=16 support is real |
| Gated/residual proxy variants | Implemented | Mostly diagnostic |
| `env_clip` execution convention | Implemented | Action-bound-clean current convention |
| Learned dominance selectors | Implemented | Best current selector still leaves large gap |
| LCB calibrated dominance | Implemented | Safe fallback diagnostic, not successful selector |
| Object-layout hand features | Implemented | Negative measured result |
| Theory notes/section | Implemented | Honest support-regret framing |
| Paper | Implemented in LaTeX | Must remain diagnostic until selector improves |

## High-Level Layout

```text
.
|-- cil/                 Core CTT metrics, chart features, and small models.
|-- dovla_cil/           Broader CIL/VLA framework: data, sims, models, eval.
|-- configs/             YAML/JSON configs for baselines, CTT, large jobs, toy jobs.
|-- data/                Exported CIL chart indexes and shards.
|-- latex/               Main paper source, tables, references, and PDF.
|-- paper/               Theory section used by the LaTeX paper.
|-- scripts/             Training, export, audit, rollout, evaluation, HF sync.
|-- manifests/           Job/run manifests and active templates.
|-- runs/                Reproducible experiment artifacts and metrics.
|-- logs/                Cluster/stdout/stderr logs and local sync logs.
|-- outputs/             Scratch-like local outputs and HF sync manifests.
|-- reports/             Legacy non-Markdown report data and CSVs.
|-- results/             Legacy non-Markdown result artifacts, if any remain.
|-- tests/               Unit/regression tests.
|-- Makefile             Convenience command entrypoint.
```

## Folder And File Inventory

### Root

- `README.md`: this file. The only Markdown document kept in the workspace.
- `Makefile`: convenience wrapper for common commands.
- `pyproject.toml`: Python packaging/tooling metadata for tests and local dev.
- `.env.example`: example environment variables; do not store secrets here.
- `.gitignore`: ignored local artifacts, caches, and generated files.
- `.claude/`, `.codex/`, `.agents/`, `.remember/`: local assistant/tool state.
- `.pytest_cache/`, `.ruff_cache/`: local test/lint caches.

### `cil/`

Core research implementation for CTT and canonical metrics.

- `cil/__init__.py`: package marker.
- `cil/chart_features.py`: deployment-visible chart feature construction.
  Feature modes include `base`, `base_context`, `base_context_obs`,
  `base_context_obj`, and `base_context_obs_obj`. This file is important
  because it controls what information the selector/generator may see.
- `cil/metrics.py`: canonical metrics:
  - BranchCAR / branch causal action regret.
  - OutcomePTR@K for measured executed generated candidates.
  - PPTC@K for distance-only proxy positive tangent coverage.
  - SelectorRegret@K.
  - SupportGap.
  - ProxySupportDistance.
  - NegativeNear.
  - PosCloserThanNeg.
  - Pairwise Causal Calibration Error.
  - safety label coverage and unsafe-rate helpers.
- `cil/models/__init__.py`: model package exports.
- `cil/models/chart_encoder.py`: chart encoder used by CTT/utility energy.
- `cil/models/tangent_encoder.py`: tangent encoder and normalization helpers.
- `cil/models/ctt.py`: Causal Tangent Transport model definitions, including
  residual/gated transport variants.
- `cil/models/utility_energy.py`: utility energy scorer used by ranking and
  dominance.

### `dovla_cil/`

General CIL/VLA framework code. This is older and broader than the compact
`cil/` CTT layer.

- `dovla_cil/config/defaults.yaml`: default project config.
- `dovla_cil/config/schema.py`: config schema.
- `dovla_cil/effects/extractors.py`: outcome/effect extraction utilities.
- `dovla_cil/effects/failure_classifier.py`: failure classifier logic.
- `dovla_cil/effects/rewards.py`: scalar reward/utility helpers.
- `dovla_cil/eval/causalstress.py`: causal stress evaluation.
- `dovla_cil/eval/external_vla_baseline.py`: external VLA baseline eval.
- `dovla_cil/eval/lattice_eval.py`: CIL lattice evaluation.
- `dovla_cil/eval/libero_eval.py`: LIBERO eval adapter.
- `dovla_cil/eval/maniskill_eval.py`: ManiSkill eval adapter.
- `dovla_cil/eval/maniskill_policy_rollout.py`: policy rollout harness.
- `dovla_cil/eval/metrics.py`: legacy eval metrics.
- `dovla_cil/eval/simpler_eval.py`: simpler eval adapter.
- `dovla_cil/eval/smolvla_cil_baseline.py`: SmolVLA CIL baseline eval.
- `dovla_cil/eval/smolvla_runtime.py`: SmolVLA runtime wrapper.
- `dovla_cil/experiments/baselines.py`: baseline experiment definitions.
- `dovla_cil/experiments/manifest.py`: manifest execution helpers.
- `dovla_cil/experiments/reports.py`: legacy report generation helpers.
- `dovla_cil/experiments/scaling.py`: scaling experiment helpers.
- `dovla_cil/generation/distributed.py`: distributed generation utilities.
- `dovla_cil/generation/maniskill_lattice.py`: ManiSkill lattice generation.
- `dovla_cil/generation/maniskill_parallel.py`: parallel ManiSkill generation.
- `dovla_cil/generation/maniskill_render.py`: rendering utilities.
- `dovla_cil/generation/pipeline.py`: generation pipeline orchestration.
- `dovla_cil/generation/tangent_chart_synthesis.py`: chart synthesis baseline.
- `dovla_cil/generation/tangent_cvae.py`: raw/positive tangent CVAE baseline.
- `dovla_cil/generation/tangent_local_atlas.py`: local atlas memory baseline.
- `dovla_cil/generation/tangent_memory.py`: tangent memory utilities.
- `dovla_cil/generation/tangent_spline_cvae.py`: spline-CVAE baseline.
- `dovla_cil/generation/tangent_spline_flow.py`: spline flow baseline.
- `dovla_cil/generation/tangent_spline_guided_flow.py`: guided spline flow.
- `dovla_cil/generation/tangent_targets.py`: tangent target construction.
- `dovla_cil/interventions/language_counterfactuals.py`: language CIL changes.
- `dovla_cil/interventions/perturbations.py`: action/scene perturbations.
- `dovla_cil/interventions/physics_counterfactuals.py`: physics interventions.
- `dovla_cil/interventions/samplers.py`: intervention samplers.
- `dovla_cil/interventions/schema.py`: intervention data schema.
- `dovla_cil/models/action_encoder.py`: action encoder.
- `dovla_cil/models/dovla.py`: base DOVLA model.
- `dovla_cil/models/dovla_attention.py`: attention model variant.
- `dovla_cil/models/dovla_attention_enhanced.py`: enhanced attention variant.
- `dovla_cil/models/dovla_hybrid.py`: hybrid model variant.
- `dovla_cil/models/dovla_transformer.py`: transformer model variant.
- `dovla_cil/models/effect_heads.py`: effect prediction heads.
- `dovla_cil/models/openvla_adapter.py`: OpenVLA adapter.
- `dovla_cil/models/policy_heads.py`: policy output heads.
- `dovla_cil/retrieval/embeddings.py`: retrieval embedding utilities.
- `dovla_cil/retrieval/eval.py`: retrieval evaluation.
- `dovla_cil/retrieval/index.py`: retrieval index.
- `dovla_cil/retrieval/prompting.py`: retrieval prompt helpers.
- `dovla_cil/retrieval/retriever.py`: retriever implementation.
- `dovla_cil/sim/base.py`: simulator interface.
- `dovla_cil/sim/genesis_backend.py`: Genesis simulator backend.
- `dovla_cil/sim/maniskill_backend.py`: ManiSkill backend.
- `dovla_cil/sim/registry.py`: simulator registry.
- `dovla_cil/sim/toy_backend.py`: toy backend.
- `dovla_cil/tasks/library.py`: task library.
- `dovla_cil/tasks/predicates.py`: task predicates.
- `dovla_cil/tasks/schema.py`: task schema.
- `dovla_cil/tasks/validators.py`: task validation.
- `dovla_cil/training/collate.py`: data collation.
- `dovla_cil/training/losses.py`: training losses.
- `dovla_cil/training/metrics.py`: training metrics.
- `dovla_cil/training/trainer.py`: trainer implementation.
- `dovla_cil/transfercritic/*`: transfer critic labels, model, training,
  selection, and evaluation.
- `dovla_cil/utils/*`: hashing, IO, logging, seeding, language embeddings,
  and OpenClaude client helpers.
- `dovla_cil/vlm/*`: VLM annotation, prompts, clients, and task generation.
- `dovla_cil/py.typed`: marks the package as typed.

### `configs/`

Configuration files for experiments.

- `configs/ctt/residual_smoke.yaml`: small residual CTT smoke config.
- `configs/ctt/residual_full.yaml`: full residual CTT config.
- `configs/ctt/residual_ot_smoke.yaml`: residual CTT smoke config with
  entropic OT alignment enabled.
- `configs/ctt/residual_ot_full.yaml`: full residual CTT + OT config.
- `configs/ctt/residual_no_cycle_smoke.yaml`: residual CTT smoke config with
  cycle consistency disabled.
- `configs/ctt/residual_no_cycle_full.yaml`: full residual CTT no-cycle
  ablation config.
- `configs/ctt/gated_residual_smoke.yaml`: small gated CTT smoke config.
- `configs/ctt/gated_residual_full.yaml`: full gated CTT config.
- `configs/baselines/*.yaml`: baseline configs for expert-only BC,
  cross-state negatives, random negatives, label-only counterfactuals, and
  world-model auxiliary baselines.
- `configs/external/*.json`: SmolVLA aligned/full/smoke external configs.
- `configs/hpc/nvidia_icd.json`: HPC GPU/ICD runtime config.
- `configs/large/*.yaml`: large-scale generation/training templates.
- `configs/toy/*.yaml`: toy generation/eval/training configs.

### `data/`

Exported CIL chart databases. These are generated artifacts, not source code.

- `data/cil_charts/{train,val,test}/`: original chart indexes/shards.
- `data/cil_charts_rgb_refs/{train,val,test}/`: non-destructive RGB-reference
  chart export with observation refs and deterministic RGB/object features.
- `index.json` inside each split records split hashes, content hashes,
  retrieval permissions, and evaluator-only outcome contracts.
- NPZ shards store chart rows, base actions, branch actions, utility labels,
  outcome vectors, residual action tangents, spline tangent codes, and metadata.

### `latex/`

Paper source and build artifacts.

- `latex/main.tex`: canonical paper draft. This is the single main paper source.
- `latex/main.pdf`: compiled PDF.
- `latex/references.bib`: bibliography.
- `latex/tables/*.tex`: hand-maintained or generated tables used by `main.tex`.
- `latex/main.aux`, `main.bbl`, `main.blg`, `main.fdb_latexmk`, `main.fls`,
  `main.log`, `main.out`: LaTeX build intermediates.

### `paper/`

Paper sections that are included or copied into the LaTeX draft.

- `paper/sections/theory.tex`: formal theory section with same-state causal
  contrast identifiability, CAR decomposition, support/sample-complexity
  arguments, and transport smoothness/support-regret bounds.
- `paper/notes/`: reserved for non-Markdown theory notes if needed. Markdown
  notes were removed to keep this README as the single textual overview.

### `scripts/`

Main executable research pipeline.

Data/chart export and audits:

- `scripts/export_cil_charts.py`: exports train/val/test CIL chart DB.
- `scripts/build_data_accounting.py`: builds data accounting artifacts. Use
  `--no-markdown-report` for README-only runs.
- `scripts/audit_cil_charts.py`: leakage audit for chart indexes and run
  hashes. Use `--no-markdown-report` for README-only runs.
- `scripts/audit_leakage.py`: legacy leakage audit.
- `scripts/audit_action_bounds.py`: action-bound validity audit. Use
  `--no-markdown-report` for README-only runs.
- `scripts/audit_chart_feature_sources.py`: audits feature source availability.
  Use `--no-markdown-report` for README-only runs.
- `scripts/check_tangent_reconstruction.py`: verifies spline tangent
  reconstruction exactly matches stored residuals. Use `--no-markdown-report`
  for README-only runs.
- `scripts/build_action_scale_vector.py`: builds per-dimension action scaling.
  Use `--no-markdown-report` for README-only runs.

CTT training/proxy/rollout:

- `scripts/train_ctt.py`: trains residual or gated residual CTT. It logs
  configured loss weights plus per-epoch component losses for Chamfer positive
  alignment, optional entropic OT alignment, negative boundary, ranking, cycle,
  and diversity terms. Use `--no-markdown-report` for README-only runs.
- `scripts/eval_ctt_proxy.py`: proxy support evaluation with PPTC,
  NegativeNear, PosCloserThanNeg, distance, diversity, and collapse metrics.
  Use `--no-markdown-report` for README-only runs.
- `scripts/eval_ctt_generated_rollout.py`: measured rollout harness that
  restores states, decodes generated tangents, executes candidates, and writes
  measured candidate rows. Use `--no-markdown-report` for README-only runs.
- `scripts/eval_ctt_rollout.py`: measured-output wrapper. Use
  `--no-markdown-report` for README-only runs.
- `scripts/build_ctt_proxy_comparison.py`: proxy comparison/gate table with
  by-task/by-seed JSON outputs. Use `--no-markdown-report` for README-only
  runs.
- `scripts/build_ctt_rollout_comparison.py`: measured rollout aggregation.
  Use `--no-markdown-report` for README-only runs.
- `scripts/build_ctt_outcome_gate.py`: measured outcome acceptance gate that
  combines rollout support and train-clean selector artifacts.
- `scripts/summarize_ctt_runs.py`: global CSV summary. Use
  `--no-markdown-report` so the persistent prose overview remains this README.

Dominance and utility:

- `scripts/train_utility_energy.py`: train-only utility energy model. Use
  `--no-markdown-report` for README-only runs.
- `scripts/calibrate_dominance.py`: conformal-style dominance calibration rule.
  Use `--no-markdown-report` for README-only runs.
- `scripts/eval_dominance_selector.py`: LCB dominance fallback evaluation.
  Reports selected success, coverage, fallback, unsafe execution, PCCE,
  selector regret, and support/selector gaps. Use `--no-markdown-report` for
  README-only runs.
- `scripts/eval_learned_dominance_selector.py`: ridge learned dominance
  selector with basic/context/tangent/source/chart-compat and score-shape
  features. Use `--no-markdown-report` for README-only runs.
- `scripts/eval_nonlinear_dominance_selector.py`: nonlinear selector sweep.
  Use `--no-markdown-report` for README-only runs.
- `scripts/build_selector_diagnostic_sweep.py`: non-cherry-picked selector
  diagnostic summary builder. It reads completed selector `metrics.json`
  artifacts, keeps all candidate rows, selects the best row per action
  convention by held-out selected success, and writes JSON/TeX/log outputs.

Metric and paper artifacts:

- `scripts/eval_metrics.py`: canonical measured/proxy metric evaluator. It
  writes JSON, TeX, config, command, hash, and log artifacts; use
  `--no-markdown-report` for README-only runs.
- `scripts/audit_ctt_paper_artifacts.py`: claim-to-artifact audit for the CTT
  paper. It scans forbidden wording, paper table inputs, implementation paths,
  run artifact contracts, and the README-only Markdown invariant, then writes
  JSON/TeX audit outputs without creating extra persistent Markdown files.
- `scripts/backfill_paper_run_artifacts.py`: transparent non-Markdown
  backfill for paper-referenced run dirs that are missing grouped metric
  placeholders, config metadata, or log stubs. It preserves existing files and
  intentionally does not recreate deleted `report.md` files.
- `scripts/reproduce_v0_report.py`: V0 reproduction artifact.
- `scripts/make_paper_artifacts.py`: generated paper tables/artifacts.
- `scripts/build_paper_analysis.py`: paper analysis builder.
- `scripts/build_paper_table_status.py`: paper table status builder.
- `scripts/report_dataset.py`, `report_eval.py`, `report_hpc_clean_results.py`:
  structured reporting helpers.

Generation and baseline scripts:

- `scripts/generate_cil.py`: CIL generation entrypoint.
- `scripts/generate_cil_distributed.py`: distributed CIL generation.
- `scripts/generate_maniskill_lattice.py`: ManiSkill lattice generator.
- `scripts/generate_metaworld_lattice.py`: MetaWorld lattice generator.
- `scripts/generate_rlbench_lattice.py`: RLBench lattice generator.
- `scripts/generate_12task_collection.py`: larger task collection generator.
- `scripts/make_cil_collection.py`: collection builder.
- `scripts/merge_task_datasets.py`: merge task datasets.
- `scripts/prepare_baseline_dataset.py`: baseline dataset prep.
- `scripts/run_baseline.py`: baseline launcher.
- `scripts/run_external_vla_baseline.py`: external VLA baseline launcher.
- `scripts/run_manifest.py`: manifest executor.
- `scripts/run_master_workflow.sh`: legacy master workflow.

Positive tangent baselines:

- `scripts/export_positive_tangent_targets.py`: exports positive tangent targets.
- `scripts/eval_positive_tangent_memory.py`: memory baseline eval.
- `scripts/eval_positive_tangent_local_atlas.py`: local atlas baseline eval.
- `scripts/eval_positive_tangent_chart_synthesis.py`: chart synthesis eval.
- `scripts/train_positive_tangent_cvae.py`: CVAE baseline training.
- `scripts/train_positive_tangent_spline_cvae.py`: spline-CVAE training.
- `scripts/train_positive_tangent_spline_flow.py`: spline flow training.
- `scripts/train_positive_tangent_guided_spline_flow.py`: guided flow training.
- `scripts/summarize_positive_tangent_*`: sweep summarizers.

Legacy DOVLA training/eval:

- `scripts/train_dovla.py`: base DOVLA training.
- `scripts/train_dovla_attention.py`: attention variant.
- `scripts/train_dovla_enhanced.py`: enhanced variant.
- `scripts/train_dovla_transformer.py`: transformer variant.
- `scripts/train_hybrid_direct.py`: hybrid direct model.
- `scripts/train_transformer_with_language.py`: language transformer training.
- `scripts/eval_*checkpoint.py`: checkpoint evaluators.
- `scripts/evaluate_phase_a*.py`: legacy phase evaluators.

HF sync and operations:

- `scripts/hf_push_once.sh`: one-shot HF upload of workspace/scratch roots.
  It also removes stale remote Markdown by default, keeping only
  `workspace/README.md` unless `HF_SYNC_REMOTE_MARKDOWN_KEEP` is changed.
- `scripts/hf_push_every_15m.sh`: local 15-minute HF sync daemon.
- `scripts/hf_sync_daemon.sh`: alternate daemon wrapper.
- `scripts/auto_sync_hf.py`: legacy auto-sync helper.
- `scripts/check_hf_sync.sh`: HF sync check.
- `scripts/quick_start.sh`, `run_eval.sh`, `run_inference.sh`,
  `run_train_debug.sh`, `smoke_test.sh`: convenience shell entrypoints.

### `scripts/slurm/`

Cluster job templates. Important groups:

- CTT: `train_ctt_proxy_sweep.sbatch`, `train_ctt_feature_proxy.sbatch`,
  `train_ctt_ot_proxy.sbatch`, `train_ctt_no_cycle_proxy.sbatch`,
  `eval_ctt_generated_rollout.sbatch`.
- Dominance: `eval_tanh_train_dominance.sbatch`,
  `eval_perdim_trainmax_dominance.sbatch`,
  `eval_bundle_consensus_dominance.sbatch`,
  `train_utility_energy_selector.sbatch`.
- Rendering/export: `render_six_task_chart_observations.sbatch`,
  `reexport_rgb_ref_cil_charts.sbatch`, `render_maniskill_observations.sbatch`.
- Baselines/generation: `generate_6task_h16.sbatch`,
  `make_maniskill_collection.sbatch`, `train_maniskill_*`.
- Legacy model training/eval: `train_dovla*.sbatch`, `train_transformer*.sbatch`,
  `eval_*`.
- HF sync: `hf_push_daemon.sbatch`.

### `manifests/`

Run manifests and templates.

- `manifests/cil_1b_template.yaml`: active large template opened in the IDE.
- `manifests/cil_160m.yaml`: smaller CIL manifest.
- `manifests/baselines_full.yaml`: full baseline manifest.
- `manifests/scaling_k_sweep.yaml`: scaling/K sweep.
- `manifests/source_score_bonus_pick001_stack005.*`: source-score bonus configs.

### `runs/`

Reproducible experiment artifacts. Each serious run should contain:

```text
config.yaml
command.txt
git_hash.txt
data_hash.txt
split_hash.txt
train.log
eval.log
metrics.json
metrics_by_task.json
metrics_by_seed.json
table.tex
```

Markdown `report.md` files were removed per the current cleanup request. Use
`metrics.json`, `table.tex`, logs, and this README instead.

High-value run directories:

- `runs/data_accounting`: verified data counts.
- `runs/leakage_audit`: original leakage audit.
- `runs/leakage_audit_rgb_refs`: RGB-ref leakage audit.
- `runs/tangent_reconstruction`: original tangent reconstruction check.
- `runs/tangent_reconstruction_rgb_refs`: RGB-ref tangent reconstruction check.
- `runs/action_bound_audit_rgb_refs`: action-bound audit.
- `runs/chart_observation_embeddings_rgb_refs`: RGB-stat embedding export.
- `runs/chart_object_embeddings_rgb_refs`: object-layout embedding export.
- `runs/chart_feature_audit*`: feature-source audits.
- `runs/ctt_residual_full_seed{0,1,2}`: full residual CTT training.
- `runs/ctt_residual_ot_full_seed{0,1,2}`: completed full residual CTT + OT
  ablation from Slurm job `15203117`; it passes the proxy gate but is worse
  than plain residual CTT on PPTC and mean positive distance.
- `runs/ctt_residual_no_cycle_full_seed{0,1,2}`: completed no-cycle residual
  CTT ablation from Slurm job `15204363`; it passes the proxy gate with lower
  NegativeNear@0.20 but slightly weaker positive support than cycle residual
  CTT.
- `runs/ctt_gated_residual_full_seed{0,1,2}`: full gated residual CTT training.
- `runs/ctt_residual_component_smoke`: real-data CTT training smoke with
  `--no-markdown-report`; logs configured loss weights and per-component loss
  means for Chamfer/OT positive alignment, negative boundary, ranking, cycle,
  and diversity terms.
- `runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison`: current
  strongest measured support artifact.
- `runs/ctt_val_proxy_comparison`: CTT-vs-local-atlas proxy support gate.
- `runs/ctt_outcome_acceptance_gate`: measured Part-F outcome acceptance gate.
- `runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison`: train
  calibration rows for K=16 `env_clip`.
- `runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test`: K=16 LCB
  dominance auto threshold, negative selector result.
- `runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test_tau0`: K=16
  LCB tau0 fallback, matches base but does not improve.
- `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`:
  best current train-clean selector diagnostic.
- `runs/ctt_base_context_obs_learned_dominance_score_*_envclip_k16_train_to_test`:
  score-shape selector diagnostics; these did not improve the best selector.
- `runs/ctt_base_context_obs_learned_dominance_*bundle*_envclip_k16_train_to_test`:
  bundle-consensus selector diagnostics. These test whether agreement among
  transported train-positive tangents is a useful deployment-visible confidence
  signal.
- `runs/ctt_base_context_obs_learned_dominance_*_tanh_train_to_test`: K=8
  tanh selector diagnostics from Slurm job `15149814`; best selected success
  is `0.3819`, but proposal oracle is also only `0.3819`.
- `runs/ctt_base_context_obs_learned_dominance_*_perdim_trainmax_train_to_test`:
  per-dimension trainmax selector diagnostics from Slurm job `15149815`; these
  are below the current K=16 `env_clip` support setting.
- `runs/ctt_selector_diagnostic_sweep`: generated selector summary table used
  by the paper to compare K=8 tanh, K=8 per-dim trainmax, K=16 env-clip, and
  K=16 checkpoint utility-energy selector diagnostics without cherry-picking a
  single row.
- `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*`: fixed
  nonlinear selector diagnostics.
- `runs/summary_ctt.csv`: global run summary table.

### `logs/`

Cluster stdout/stderr and daemon logs.

- `logs/*.out` and `logs/*.err`: Slurm job output/error files.
- `logs/auto_sync_hf.log`: legacy HF auto-sync log.
- `logs/auto_sync_hf.pid`: legacy auto-sync PID file.
- `logs/workflow/`: workflow logs.

### `outputs/`

Local generated outputs and HF sync state.

- `outputs/hf_sync/hf_sync.log`: one-shot HF sync log.
- `outputs/hf_sync/hf_sync_daemon.log`: 15-minute daemon log.
- `outputs/hf_sync/last_manifest.json`: latest HF sync manifest.
- `outputs/hpc/`: HPC outputs.
- `outputs/external_vla*`: external VLA export/probe outputs.
- `outputs/manifest_*`: manifest smoke outputs.
- `outputs/phase5_*`: legacy phase-5 outputs.
- `outputs/smoke_*`, `outputs/train_smoke_*`: smoke outputs.
- `outputs/wheels`: local wheel/cache outputs.

### `reports/`

Legacy non-Markdown reports and CSV/JSON summaries.

- `reports/phase_a2_evaluation.json`: legacy Phase-A2 evaluation artifact.
- `reports/phase_a4_results.json`: legacy Phase-A4 result artifact.
- `reports/hpc_clean_results/clean_result_manifest.json`: file manifest for
  the HPC-clean result sweep.
- `reports/hpc_clean_results/clean_result_rows.csv`: row-level clean-result
  table.
- `reports/hpc_clean_results/clean_result_summary.csv`: aggregate
  clean-result summary.
- `reports/hpc_clean_results/excluded_unclean_paths.txt`: paths excluded from
  the clean-result sweep.

### `results/`

Legacy result files. Markdown summaries were removed. Any remaining non-Markdown
files are legacy evidence or machine-readable artifacts. Current CTT evidence
should be read from `runs/`, not `results/`.

### `tests/`

Regression tests. Key tests:

- `tests/test_metrics.py`: canonical metric behavior.
- `tests/test_causal_action_metrics.py`: causal action metric checks.
- `tests/test_ctt.py`: CTT model/training/eval checks.
- `tests/test_chart_features.py`: chart feature leakage invariants.
- `tests/test_dominance_selector.py`: dominance selector, PCCE, safety fields.
- `tests/test_cil_schema.py`, `test_cil_images.py`: CIL data/image schema.
- `tests/test_maniskill_*`: ManiSkill backend/lattice/render/rollout tests.
- `tests/test_tangent_*`: positive tangent generator baselines.
- `tests/test_transfercritic.py`: transfer critic checks.
- `tests/test_slurm_templates.py`: Slurm template sanity.

## Current Best Evidence

### K=16 `env_clip` measured support

Run:

```text
runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison
```

Key held-out test values:

| Metric | Value |
| --- | ---: |
| Rows | 144 |
| Base success | 0.2917 |
| Score-only selected success | 0.2778 |
| Proposal oracle success | 0.5694 |
| Hidden chart oracle success | 0.7292 |
| OutcomePTR@16 | 0.5486 |
| Success support gap | 0.2014 |
| Success selector gap | 0.2917 |
| Action-bound unsafe | 0.0000 |

Interpretation: support is strong; selector is not.

### CTT validation proxy support gate

Run:

```text
runs/ctt_val_proxy_comparison
```

This is a proxy geometry gate, not rollout success. CTT variants pass by
improving mean distance to target positives while staying within the
NegativeNear@0.20 safety slack; they do not beat local-atlas on PPTC thresholds.
The OT variant passes the same proxy gate but does not improve over plain
residual CTT. The no-cycle ablation passes too, but is slightly worse than
cycle residual CTT on PPTC and mean positive distance. The gated residual
variant fails the safety gate.

| Method | PPTC@0.20 | PPTC@0.40 | Neg@0.20 | Pos<Neg | MeanPos | Collapse | Gate |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | --- |
| local-atlas | 0.4058 | 0.6812 | 0.0368 | 0.5998 | 0.7203 | 0.0661 | baseline |
| CTT residual full | 0.1981 | 0.6087 | 0.0296 | 0.7352 | 0.4509 | 0.0681 | pass |
| CTT residual no-cycle full | 0.1884 | 0.6039 | 0.0194 | 0.7391 | 0.4577 | 0.0681 | pass |
| CTT residual+OT full | 0.1787 | 0.5894 | 0.0187 | 0.7284 | 0.4604 | 0.0681 | pass |
| CTT residual base+context+obs | 0.2464 | 0.6425 | 0.0343 | 0.7717 | 0.4347 | 0.0681 | pass |
| CTT gated residual full | 0.2319 | 0.6135 | 0.0527 | 0.7248 | 0.4337 | 0.0681 | fail |

Interpretation: CTT improves support distance geometry, but the story is still
diagnostic until measured rollout and selection close the outcome gap.

### CTT + OT alignment smoke

Runs:

```text
runs/ctt_residual_ot_component_smoke
runs/ctt_residual_ot_component_smoke_val_proxy
```

Purpose: verify the implemented optional entropic OT alignment path for
`train_ctt.py` before promoting it to a full proxy comparison. This is a
code-path smoke, not a method result.

| Item | Value |
| --- | ---: |
| Train charts | 16 |
| Neighbor pairs | 28 |
| OT alignment weight | 0.25 |
| Final OT component | 31.3781 |
| Val proxy rows | 32 |
| PPTC@0.20 | 0.3438 |
| PPTC@0.40 | 0.7188 |
| NegativeNear@0.20 | 0.0423 |
| Pos<Neg | 0.5974 |
| Mean positive distance | 0.6672 |

Full 3-seed validation proxy job completed on 2026-07-04:

```text
15203117 scripts/slurm/train_ctt_ot_proxy.sbatch array=0-2
exit: 0:0 for tasks 0, 1, 2
elapsed: 00:01:27 per task
```

Full validation result averaged over three seeds:

| Metric | Value |
| --- | ---: |
| Rows | 207 |
| PPTC@0.20 | 0.1787 |
| PPTC@0.40 | 0.5894 |
| NegativeNear@0.20 | 0.0187 |
| Pos<Neg | 0.7284 |
| Mean positive distance | 0.4604 |
| Collapse | 0.0681 |

Interpretation: the OT path is implemented, smoke-tested, and evaluated. It is
safer on NegativeNear@0.20 than plain residual CTT, but it is worse on PPTC and
mean positive distance. Treat it as an ablation/negative diagnostic, not the
main CTT variant.

### CTT no-cycle ablation

Purpose: test whether the cycle consistency term in `train_ctt.py` improves
validation proxy support or only adds optimization baggage.

Completed on 2026-07-04:

```text
15204363 scripts/slurm/train_ctt_no_cycle_proxy.sbatch array=0-2
exit: 0:0 for tasks 0, 1, 2
elapsed: 00:01:28 to 00:01:41 per task
```

Full validation result averaged over three seeds:

| Metric | Value |
| --- | ---: |
| Rows | 207 |
| PPTC@0.20 | 0.1884 |
| PPTC@0.40 | 0.6039 |
| NegativeNear@0.20 | 0.0194 |
| Pos<Neg | 0.7391 |
| Mean positive distance | 0.4577 |
| Collapse | 0.0681 |

Interpretation: removing cycle reduces NegativeNear@0.20, but it also slightly
worsens PPTC and mean positive distance versus the cycle residual row. Cycle is
not a breakthrough ingredient, but the measured ablation supports keeping it in
the default CTT support model.

### Best current K=16 train-clean learned selector

Run:

```text
runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test
```

Key values:

| Metric | Value |
| --- | ---: |
| Selected success | 0.3542 |
| Coverage | 0.6597 |
| Proposal oracle success | 0.5694 |
| Success selector gap | 0.2431 |
| Pairwise causal calibration ECE | 0.0150 |
| Pair count | 12,434 |

Interpretation: RGB-stat chart compatibility helps, but the selector gap is
still too large for a deployment-clean method success claim.

### K=16 measured outcome acceptance gate

Run:

```text
runs/ctt_outcome_acceptance_gate
```

This gate combines the measured K=16 rollout support artifact with the best
train-clean K=16 selector. It records which Part-F acceptance bars are met and
which are not.

| Gate | Observed | Required | Status |
| --- | ---: | ---: | --- |
| Selected clean success | 0.3542 | >= 0.4745 | fail |
| Target selected clean success | 0.3542 | >= 0.5000 | fail |
| Proposal oracle success | 0.5694 | >= 0.5000 | pass |
| Success support gap | 0.2014 | <= 0.0700 | fail |
| Success selector gap | 0.2431 | <= 0.0300 | fail |
| OutcomePTR@16 vs V0 | 0.5486 | > 0.4435 | pass |
| Generated unsafe rate | 0.0000 | <= base+slack | pass |
| Three train seeds + bootstrap CI | yes | required | pass |

Interpretation: CTT passes proposal-oracle, measured OutcomePTR, safety, seed,
and CI checks; it fails selected-success, support-gap, and selector-gap bars.
The current paper must remain diagnostic rather than claiming final CTT method
success.

### K=16 score-shape selector diagnostic

New score-relative features were added to test whether visible row-shape
signals can close the selector gap. They did not beat the current best
chart-compat selector.

| Run family | Selected success | Coverage | Success selector gap |
| --- | ---: | ---: | ---: |
| `score_chart_compat`, utility | 0.3264 | 0.6389 | 0.2500 |
| `score_context_chart_compat`, utility | 0.3264 | 0.6458 | 0.2500 |
| `score_chart_compat`, success bonus 2 | 0.3194 | 0.5486 | 0.2500 |
| `score_context_chart_compat`, success bonus 2 | 0.3403 | 0.6111 | 0.2292 |

Interpretation: small deployment-visible selector features are not enough; the
paper should keep emphasizing positive tangent support generation and CTT.

### K=16 bundle-consensus selector diagnostic

Submitted on 2026-07-04:

```text
15205463 scripts/slurm/eval_bundle_consensus_dominance.sbatch
```

Purpose: test whether deployment-visible agreement among transported
train-positive tangents helps the selector. The feature set uses only generated
tangent geometry, row scores, source chart/task identifiers, and train-source
evidence where configured. It does not use target positive/negative sets or
candidate outcomes when choosing. This is a selector diagnostic, not a new main
method contribution. Replace this pending note with measured values after the
job completes.

### K=8 tanh and per-dimension trainmax selector diagnostics

Jobs:

```text
15149814 scripts/slurm/eval_tanh_train_dominance.sbatch
15149815 scripts/slurm/eval_perdim_trainmax_dominance.sbatch
```

Both jobs completed with exit code `0:0` on 2026-07-04 and wrote README-only
artifacts, with no persistent `report.md`. The strongest tanh learned selector
is `context_success` with selected success `0.3819`, but the tanh proposal
oracle is also `0.3819`; this is a selector diagnostic, not a stronger CTT
support result. The best per-dimension trainmax selector reaches `0.3333`
selected success with proposal oracle `0.2222`, which is worse than the K=16
`env_clip` support setting.

| Diagnostic family | Best run | Selected success | Proposal oracle | Success selector gap |
| --- | --- | ---: | ---: | ---: |
| tanh | `context_success` | 0.3819 | 0.3819 | 0.1250 |
| per-dim trainmax | `context` | 0.3333 | 0.2222 | 0.0625 |

Interpretation: these sweeps reinforce the current diagnosis: changing the
execution/action transform or lightweight selector features does not yet
produce a deployment-clean method success.

### K=16 score-source LCB dominance

Runs:

```text
runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test
runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test_tau0
```

Key values:

| Variant | Selected | Coverage | Unsafe exec. | PCCE | Selector gap |
| --- | ---: | ---: | ---: | ---: | ---: |
| auto | 0.2778 | 0.1319 | 0.0000 | 0.1633 | 0.2986 |
| tau0 | 0.2917 | 0.1111 | 0.0000 | 0.1633 | 0.2986 |

Interpretation: the LCB fallback records the requested safety/calibration
fields, but it does not solve dominance. It is a negative Part-G diagnostic.

### Utility-energy selector diagnostic

Submitted and completed on 2026-07-04:

```text
15177546 scripts/slurm/train_utility_energy_selector.sbatch array=0-2
exit: 0:0 for tasks 0, 1, 2
```

Purpose: train three `base_context_obs` utility-energy checkpoints on the
train-only RGB-ref chart database, then evaluate checkpoint-scored calibrated
dominance on the strongest current measured support setting:

```text
calibration: runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison
evaluation:  runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison
K:           16
```

Result:

| Seed | Selected | Coverage | Fallback | Unsafe | PCCE | Selector gap |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| 0 | 0.2708 | 0.1250 | 0.8750 | 0.0000 | 0.0150 | 0.3194 |
| 1 | 0.2708 | 0.0972 | 0.9028 | 0.0000 | 0.0090 | 0.3194 |
| 2 | 0.2778 | 0.1736 | 0.8264 | 0.0000 | 0.0253 | 0.3056 |

Interpretation: checkpoint-scored utility energy does not close the selector
gap. It has low coverage and mostly falls back to the base policy, so it is a
negative Part-G diagnostic and does not replace the existing train-clean K=16
selector.

## How To Run Core Commands

Use the local virtual environment when possible:

```bash
.venv/bin/python -m pytest tests/test_metrics.py tests/test_ctt.py tests/test_dominance_selector.py -q
```

Build the paper:

```bash
cd latex
latexmk -pdf -interaction=nonstopmode main.tex
```

Refresh summary:

```bash
.venv/bin/python scripts/summarize_ctt_runs.py \
  --run-root runs \
  --out-csv runs/summary_ctt.csv \
  --no-markdown-report
```

Run K=16 LCB dominance auto:

```bash
.venv/bin/python scripts/eval_dominance_selector.py \
  --calibration-input runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison/combined_measured_candidates.json \
  --calibration-target-index data/cil_charts_rgb_refs/train/index.json \
  --eval-input runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison/combined_measured_candidates.json \
  --eval-target-index data/cil_charts_rgb_refs/test/index.json \
  --checkpoint-template 'runs/ctt_residual_base_context_obs_seed{seed}/model.pt' \
  --out-dir runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test \
  --k 16 \
  --bootstrap-samples 1000 \
  --no-markdown-report
```

Run leakage audits:

```bash
.venv/bin/python scripts/audit_cil_charts.py \
  --chart-root data/cil_charts \
  --run-root runs \
  --out-dir runs/leakage_audit \
  --no-markdown-report

.venv/bin/python scripts/audit_cil_charts.py \
  --chart-root data/cil_charts_rgb_refs \
  --run-root runs \
  --out-dir runs/leakage_audit_rgb_refs \
  --no-markdown-report
```

Run tangent reconstruction checks:

```bash
.venv/bin/python scripts/check_tangent_reconstruction.py \
  --chart-root data/cil_charts \
  --out-dir runs/tangent_reconstruction \
  --no-markdown-report

.venv/bin/python scripts/check_tangent_reconstruction.py \
  --chart-root data/cil_charts_rgb_refs \
  --out-dir runs/tangent_reconstruction_rgb_refs \
  --no-markdown-report
```

Start HF sync daemon:

```bash
bash scripts/hf_push_every_15m.sh
```

Current local daemon process was previously observed as PID `615094`; verify
with:

```bash
ps -p 615094 -o pid,etimes,cmd
```

## Hugging Face Remote

Repository:

```text
anhtld/vla
```

The workspace is uploaded under:

```text
workspace/
```

Large scratch roots are uploaded under:

```text
scratch/
```

Use:

```bash
.venv/bin/hf auth whoami
.venv/bin/hf upload anhtld/vla <local_path> workspace/<remote_path>
```

Avoid uploading secrets. The sync scripts exclude `.env`, token/secret/key
patterns, virtualenvs, git internals, containers, and native library folders.
They also enforce the README-only Markdown policy on the Hub by deleting
remote `*.md` files except `workspace/README.md` after each successful sync.
Set `HF_SYNC_CLEAN_REMOTE_MARKDOWN=0` only for a deliberate temporary override.

## Development Rules

- Do not claim method success unless the result is implemented, measured,
  leakage-audited, and logged.
- Do not call distance proxy metrics PTR. Use PPTC for proxy support.
- Do not compute OutcomePTR, SelectorRegret, or SupportGap from proxy-only
  candidates.
- Train-only retrieval must use train charts only.
- Validation/test outcomes are evaluator-only.
- Keep V0/V1/V3 as diagnostics/baselines, not final method claims.
- Use K=16 `env_clip` as the current bounded-action diagnostic convention until
  a better action representation is implemented and measured.
- Treat deterministic object-layout features as a negative result unless a new
  measured run proves otherwise.
- The next real method work is a stronger deployment-visible chart/outcome
  representation and dominance model, not more wrapper text.

## Immediate Next Actions

1. Replace hand-built RGB/object descriptors with learned visual-language or
   task/object/contact-stage tokens.
2. Train a stronger train-only utility/dominance model under the K=16
   `env_clip` convention.
3. Re-run measured selection on held-out test after the representation fix.
4. Add actual collision/contact safety labels beyond action-bound validity.
5. Keep the paper honest: support is promising, selector is not solved.