workspace/README.md

VLA / CIL-Atlas / Causal Tangent Transport

This repository is the working codebase for a robot-learning research project around same-state counterfactual action charts and Causal Tangent Transport (CTT).

The single spine of the project is:

Standard VLA training observes one demonstrated action per state. CIL-Atlas restores the same state, executes multiple action chunks, and measures which local action tangents causally improve, recover, fail, collide, or succeed. CTT turns this measured local causal geometry into deployment-clean proposal generation by transporting measured positive do-action tangents from train-only neighboring charts into the target chart.

The current evidence is intentionally written as a diagnostic method paper, not as an overclaimed final success. K=16 env_clip support is strong on held-out test (proposal_oracle_success = 0.5694, OutcomePTR@16 = 0.5486), while the selector/dominance side remains the bottleneck. The strongest current train-clean K=16 selector reaches selected_success = 0.3542 against a 0.5694 proposal oracle, leaving a 0.2431 success selector gap. The score-source LCB dominance fallback is safe under action-bound labels but negative as a selector (0.2778 auto, 0.2917 tau0). Additional K=8 tanh and per-dimension trainmax selector sweeps completed on 2026-07-04; the best tanh selector reaches 0.3819 selected success, but its proposal oracle is only 0.3819, so it does not replace the K=16 env_clip outcome gate.

All project-authored Markdown files outside vendored/tool caches were removed and consolidated into this README. The canonical paper is latex/main.tex plus latex/main.pdf; experiment evidence lives in JSON, TeX, logs, configs, and command files under runs/.

Research Goal

The paper target is not "a bigger stack." The target is a clean method story:

1. Same-state CIL charts define local do-action causal geometry.
2. Causal Action Regret decomposes deployment failure into support gap plus selector gap.
3. CTT proposes candidates by transporting measured train positive tangents, not by Gaussian noise or verifier optimization off support.
4. Utility energy and calibrated dominance decide whether a transported tangent should replace the base action.
5. Every main claim must have a method, implemented script/module, metric table, leakage audit, and reproducible run log.

Current strategic diagnosis:

Area	Current status	Meaning
Same-state chart data	Implemented and leakage-audited	Good scientific primitive
Metrics	Implemented with measured/proxy separation	OutcomePTR and PPTC are not confused
CTT residual transport	Implemented and measured	K=16 support is real
Gated/residual proxy variants	Implemented	Mostly diagnostic
env_clip execution convention	Implemented	Action-bound-clean current convention
Learned dominance selectors	Implemented	Best current selector still leaves large gap
LCB calibrated dominance	Implemented	Safe fallback diagnostic, not successful selector
Object-layout hand features	Implemented	Negative measured result
Theory notes/section	Implemented	Honest support-regret framing
Paper	Implemented in LaTeX	Must remain diagnostic until selector improves

High-Level Layout

.
|-- cil/                 Core CTT metrics, chart features, and small models.
|-- dovla_cil/           Broader CIL/VLA framework: data, sims, models, eval.
|-- configs/             YAML/JSON configs for baselines, CTT, large jobs, toy jobs.
|-- data/                Exported CIL chart indexes and shards.
|-- latex/               Main paper source, tables, references, and PDF.
|-- paper/               Theory section used by the LaTeX paper.
|-- scripts/             Training, export, audit, rollout, evaluation, HF sync.
|-- manifests/           Job/run manifests and active templates.
|-- runs/                Reproducible experiment artifacts and metrics.
|-- logs/                Cluster/stdout/stderr logs and local sync logs.
|-- outputs/             Scratch-like local outputs and HF sync manifests.
|-- reports/             Legacy non-Markdown report data and CSVs.
|-- results/             Legacy non-Markdown result artifacts, if any remain.
|-- tests/               Unit/regression tests.
|-- Makefile             Convenience command entrypoint.

Folder And File Inventory

Root

• README.md: this file. The only Markdown document kept in the workspace.
• Makefile: convenience wrapper for common commands.
• pyproject.toml: Python packaging/tooling metadata for tests and local dev.
• .env.example: example environment variables; do not store secrets here.
• .gitignore: ignored local artifacts, caches, and generated files.
• .claude/, .codex/, .agents/, .remember/: local assistant/tool state.
• .pytest_cache/, .ruff_cache/: local test/lint caches.

cil/

Core research implementation for CTT and canonical metrics.

• cil/__init__.py: package marker.
• cil/chart_features.py: deployment-visible chart feature construction. Feature modes include base, base_context, base_context_obs, base_context_obj, and base_context_obs_obj. This file is important because it controls what information the selector/generator may see.
• cil/metrics.py: canonical metrics:
  • BranchCAR / branch causal action regret.
  • OutcomePTR@K for measured executed generated candidates.
  • PPTC@K for distance-only proxy positive tangent coverage.
  • SelectorRegret@K.
  • SupportGap.
  • ProxySupportDistance.
  • NegativeNear.
  • PosCloserThanNeg.
  • Pairwise Causal Calibration Error.
  • safety label coverage and unsafe-rate helpers.
• cil/models/__init__.py: model package exports.
• cil/models/chart_encoder.py: chart encoder used by CTT/utility energy.
• cil/models/tangent_encoder.py: tangent encoder and normalization helpers.
• cil/models/ctt.py: Causal Tangent Transport model definitions, including residual/gated transport variants.
• cil/models/utility_energy.py: utility energy scorer used by ranking and dominance.

dovla_cil/

General CIL/VLA framework code. This is older and broader than the compact cil/ CTT layer.

• dovla_cil/config/defaults.yaml: default project config.
• dovla_cil/config/schema.py: config schema.
• dovla_cil/effects/extractors.py: outcome/effect extraction utilities.
• dovla_cil/effects/failure_classifier.py: failure classifier logic.
• dovla_cil/effects/rewards.py: scalar reward/utility helpers.
• dovla_cil/eval/causalstress.py: causal stress evaluation.
• dovla_cil/eval/external_vla_baseline.py: external VLA baseline eval.
• dovla_cil/eval/lattice_eval.py: CIL lattice evaluation.
• dovla_cil/eval/libero_eval.py: LIBERO eval adapter.
• dovla_cil/eval/maniskill_eval.py: ManiSkill eval adapter.
• dovla_cil/eval/maniskill_policy_rollout.py: policy rollout harness.
• dovla_cil/eval/metrics.py: legacy eval metrics.
• dovla_cil/eval/simpler_eval.py: simpler eval adapter.
• dovla_cil/eval/smolvla_cil_baseline.py: SmolVLA CIL baseline eval.
• dovla_cil/eval/smolvla_runtime.py: SmolVLA runtime wrapper.
• dovla_cil/experiments/baselines.py: baseline experiment definitions.
• dovla_cil/experiments/manifest.py: manifest execution helpers.
• dovla_cil/experiments/reports.py: legacy report generation helpers.
• dovla_cil/experiments/scaling.py: scaling experiment helpers.
• dovla_cil/generation/distributed.py: distributed generation utilities.
• dovla_cil/generation/maniskill_lattice.py: ManiSkill lattice generation.
• dovla_cil/generation/maniskill_parallel.py: parallel ManiSkill generation.
• dovla_cil/generation/maniskill_render.py: rendering utilities.
• dovla_cil/generation/pipeline.py: generation pipeline orchestration.
• dovla_cil/generation/tangent_chart_synthesis.py: chart synthesis baseline.
• dovla_cil/generation/tangent_cvae.py: raw/positive tangent CVAE baseline.
• dovla_cil/generation/tangent_local_atlas.py: local atlas memory baseline.
• dovla_cil/generation/tangent_memory.py: tangent memory utilities.
• dovla_cil/generation/tangent_spline_cvae.py: spline-CVAE baseline.
• dovla_cil/generation/tangent_spline_flow.py: spline flow baseline.
• dovla_cil/generation/tangent_spline_guided_flow.py: guided spline flow.
• dovla_cil/generation/tangent_targets.py: tangent target construction.
• dovla_cil/interventions/language_counterfactuals.py: language CIL changes.
• dovla_cil/interventions/perturbations.py: action/scene perturbations.
• dovla_cil/interventions/physics_counterfactuals.py: physics interventions.
• dovla_cil/interventions/samplers.py: intervention samplers.
• dovla_cil/interventions/schema.py: intervention data schema.
• dovla_cil/models/action_encoder.py: action encoder.
• dovla_cil/models/dovla.py: base DOVLA model.
• dovla_cil/models/dovla_attention.py: attention model variant.
• dovla_cil/models/dovla_attention_enhanced.py: enhanced attention variant.
• dovla_cil/models/dovla_hybrid.py: hybrid model variant.
• dovla_cil/models/dovla_transformer.py: transformer model variant.
• dovla_cil/models/effect_heads.py: effect prediction heads.
• dovla_cil/models/openvla_adapter.py: OpenVLA adapter.
• dovla_cil/models/policy_heads.py: policy output heads.
• dovla_cil/retrieval/embeddings.py: retrieval embedding utilities.
• dovla_cil/retrieval/eval.py: retrieval evaluation.
• dovla_cil/retrieval/index.py: retrieval index.
• dovla_cil/retrieval/prompting.py: retrieval prompt helpers.
• dovla_cil/retrieval/retriever.py: retriever implementation.
• dovla_cil/sim/base.py: simulator interface.
• dovla_cil/sim/genesis_backend.py: Genesis simulator backend.
• dovla_cil/sim/maniskill_backend.py: ManiSkill backend.
• dovla_cil/sim/registry.py: simulator registry.
• dovla_cil/sim/toy_backend.py: toy backend.
• dovla_cil/tasks/library.py: task library.
• dovla_cil/tasks/predicates.py: task predicates.
• dovla_cil/tasks/schema.py: task schema.
• dovla_cil/tasks/validators.py: task validation.
• dovla_cil/training/collate.py: data collation.
• dovla_cil/training/losses.py: training losses.
• dovla_cil/training/metrics.py: training metrics.
• dovla_cil/training/trainer.py: trainer implementation.
• dovla_cil/transfercritic/*: transfer critic labels, model, training, selection, and evaluation.
• dovla_cil/utils/*: hashing, IO, logging, seeding, language embeddings, and OpenClaude client helpers.
• dovla_cil/vlm/*: VLM annotation, prompts, clients, and task generation.
• dovla_cil/py.typed: marks the package as typed.

configs/

Configuration files for experiments.

• configs/ctt/residual_smoke.yaml: small residual CTT smoke config.
• configs/ctt/residual_full.yaml: full residual CTT config.
• configs/ctt/residual_ot_smoke.yaml: residual CTT smoke config with entropic OT alignment enabled.
• configs/ctt/residual_ot_full.yaml: full residual CTT + OT config.
• configs/ctt/residual_no_cycle_smoke.yaml: residual CTT smoke config with cycle consistency disabled.
• configs/ctt/residual_no_cycle_full.yaml: full residual CTT no-cycle ablation config.
• configs/ctt/gated_residual_smoke.yaml: small gated CTT smoke config.
• configs/ctt/gated_residual_full.yaml: full gated CTT config.
• configs/baselines/*.yaml: baseline configs for expert-only BC, cross-state negatives, random negatives, label-only counterfactuals, and world-model auxiliary baselines.
• configs/external/*.json: SmolVLA aligned/full/smoke external configs.
• configs/hpc/nvidia_icd.json: HPC GPU/ICD runtime config.
• configs/large/*.yaml: large-scale generation/training templates.
• configs/toy/*.yaml: toy generation/eval/training configs.

data/

Exported CIL chart databases. These are generated artifacts, not source code.

• data/cil_charts/{train,val,test}/: original chart indexes/shards.
• data/cil_charts_rgb_refs/{train,val,test}/: non-destructive RGB-reference chart export with observation refs and deterministic RGB/object features.
• index.json inside each split records split hashes, content hashes, retrieval permissions, and evaluator-only outcome contracts.
• NPZ shards store chart rows, base actions, branch actions, utility labels, outcome vectors, residual action tangents, spline tangent codes, and metadata.

latex/

Paper source and build artifacts.

• latex/main.tex: canonical paper draft. This is the single main paper source.
• latex/main.pdf: compiled PDF.
• latex/references.bib: bibliography.
• latex/tables/*.tex: hand-maintained or generated tables used by main.tex.
• latex/main.aux, main.bbl, main.blg, main.fdb_latexmk, main.fls, main.log, main.out: LaTeX build intermediates.

paper/

Paper sections that are included or copied into the LaTeX draft.

• paper/sections/theory.tex: formal theory section with same-state causal contrast identifiability, CAR decomposition, support/sample-complexity arguments, and transport smoothness/support-regret bounds.
• paper/notes/: reserved for non-Markdown theory notes if needed. Markdown notes were removed to keep this README as the single textual overview.

scripts/

Main executable research pipeline.

Data/chart export and audits:

• scripts/export_cil_charts.py: exports train/val/test CIL chart DB.
• scripts/build_data_accounting.py: builds data accounting artifacts. Use --no-markdown-report for README-only runs.
• scripts/audit_cil_charts.py: leakage audit for chart indexes and run hashes. Use --no-markdown-report for README-only runs.
• scripts/audit_leakage.py: legacy leakage audit.
• scripts/audit_action_bounds.py: action-bound validity audit. Use --no-markdown-report for README-only runs.
• scripts/audit_chart_feature_sources.py: audits feature source availability. Use --no-markdown-report for README-only runs.
• scripts/check_tangent_reconstruction.py: verifies spline tangent reconstruction exactly matches stored residuals. Use --no-markdown-report for README-only runs.
• scripts/build_action_scale_vector.py: builds per-dimension action scaling. Use --no-markdown-report for README-only runs.

CTT training/proxy/rollout:

• scripts/train_ctt.py: trains residual or gated residual CTT. It logs configured loss weights plus per-epoch component losses for Chamfer positive alignment, optional entropic OT alignment, negative boundary, ranking, cycle, and diversity terms. Use --no-markdown-report for README-only runs.
• scripts/eval_ctt_proxy.py: proxy support evaluation with PPTC, NegativeNear, PosCloserThanNeg, distance, diversity, and collapse metrics. Use --no-markdown-report for README-only runs.
• scripts/eval_ctt_generated_rollout.py: measured rollout harness that restores states, decodes generated tangents, executes candidates, and writes measured candidate rows. Use --no-markdown-report for README-only runs.
• scripts/eval_ctt_rollout.py: measured-output wrapper. Use --no-markdown-report for README-only runs.
• scripts/build_ctt_proxy_comparison.py: proxy comparison/gate table with by-task/by-seed JSON outputs. Use --no-markdown-report for README-only runs.
• scripts/build_ctt_rollout_comparison.py: measured rollout aggregation. Use --no-markdown-report for README-only runs.
• scripts/build_ctt_outcome_gate.py: measured outcome acceptance gate that combines rollout support and train-clean selector artifacts.
• scripts/summarize_ctt_runs.py: global CSV summary. Use --no-markdown-report so the persistent prose overview remains this README.

Dominance and utility:

• scripts/train_utility_energy.py: train-only utility energy model. Use --no-markdown-report for README-only runs.
• scripts/calibrate_dominance.py: conformal-style dominance calibration rule. Use --no-markdown-report for README-only runs.
• scripts/eval_dominance_selector.py: LCB dominance fallback evaluation. Reports selected success, coverage, fallback, unsafe execution, PCCE, selector regret, and support/selector gaps. Use --no-markdown-report for README-only runs.
• scripts/eval_learned_dominance_selector.py: ridge learned dominance selector with basic/context/tangent/source/chart-compat and score-shape features. Use --no-markdown-report for README-only runs.
• scripts/eval_nonlinear_dominance_selector.py: nonlinear selector sweep. Use --no-markdown-report for README-only runs.
• scripts/build_selector_diagnostic_sweep.py: non-cherry-picked selector diagnostic summary builder. It reads completed selector metrics.json artifacts, keeps all candidate rows, selects the best row per action convention by held-out selected success, and writes JSON/TeX/log outputs.

Metric and paper artifacts:

• scripts/eval_metrics.py: canonical measured/proxy metric evaluator. It writes JSON, TeX, config, command, hash, and log artifacts; use --no-markdown-report for README-only runs.
• scripts/audit_ctt_paper_artifacts.py: claim-to-artifact audit for the CTT paper. It scans forbidden wording, paper table inputs, implementation paths, run artifact contracts, and the README-only Markdown invariant, then writes JSON/TeX audit outputs without creating extra persistent Markdown files.
• scripts/backfill_paper_run_artifacts.py: transparent non-Markdown backfill for paper-referenced run dirs that are missing grouped metric placeholders, config metadata, or log stubs. It preserves existing files and intentionally does not recreate deleted report.md files.
• scripts/reproduce_v0_report.py: V0 reproduction artifact.
• scripts/make_paper_artifacts.py: generated paper tables/artifacts.
• scripts/build_paper_analysis.py: paper analysis builder.
• scripts/build_paper_table_status.py: paper table status builder.
• scripts/report_dataset.py, report_eval.py, report_hpc_clean_results.py: structured reporting helpers.

Generation and baseline scripts:

• scripts/generate_cil.py: CIL generation entrypoint.
• scripts/generate_cil_distributed.py: distributed CIL generation.
• scripts/generate_maniskill_lattice.py: ManiSkill lattice generator.
• scripts/generate_metaworld_lattice.py: MetaWorld lattice generator.
• scripts/generate_rlbench_lattice.py: RLBench lattice generator.
• scripts/generate_12task_collection.py: larger task collection generator.
• scripts/make_cil_collection.py: collection builder.
• scripts/merge_task_datasets.py: merge task datasets.
• scripts/prepare_baseline_dataset.py: baseline dataset prep.
• scripts/run_baseline.py: baseline launcher.
• scripts/run_external_vla_baseline.py: external VLA baseline launcher.
• scripts/run_manifest.py: manifest executor.
• scripts/run_master_workflow.sh: legacy master workflow.

Positive tangent baselines:

• scripts/export_positive_tangent_targets.py: exports positive tangent targets.
• scripts/eval_positive_tangent_memory.py: memory baseline eval.
• scripts/eval_positive_tangent_local_atlas.py: local atlas baseline eval.
• scripts/eval_positive_tangent_chart_synthesis.py: chart synthesis eval.
• scripts/train_positive_tangent_cvae.py: CVAE baseline training.
• scripts/train_positive_tangent_spline_cvae.py: spline-CVAE training.
• scripts/train_positive_tangent_spline_flow.py: spline flow training.
• scripts/train_positive_tangent_guided_spline_flow.py: guided flow training.
• scripts/summarize_positive_tangent_*: sweep summarizers.

Legacy DOVLA training/eval:

• scripts/train_dovla.py: base DOVLA training.
• scripts/train_dovla_attention.py: attention variant.
• scripts/train_dovla_enhanced.py: enhanced variant.
• scripts/train_dovla_transformer.py: transformer variant.
• scripts/train_hybrid_direct.py: hybrid direct model.
• scripts/train_transformer_with_language.py: language transformer training.
• scripts/eval_*checkpoint.py: checkpoint evaluators.
• scripts/evaluate_phase_a*.py: legacy phase evaluators.

HF sync and operations:

• scripts/hf_push_once.sh: one-shot HF upload of workspace/scratch roots. It also removes stale remote Markdown by default, keeping only workspace/README.md unless HF_SYNC_REMOTE_MARKDOWN_KEEP is changed.
• scripts/hf_push_every_15m.sh: local 15-minute HF sync daemon.
• scripts/hf_sync_daemon.sh: alternate daemon wrapper.
• scripts/auto_sync_hf.py: legacy auto-sync helper.
• scripts/check_hf_sync.sh: HF sync check.
• scripts/quick_start.sh, run_eval.sh, run_inference.sh, run_train_debug.sh, smoke_test.sh: convenience shell entrypoints.

scripts/slurm/

Cluster job templates. Important groups:

• CTT: train_ctt_proxy_sweep.sbatch, train_ctt_feature_proxy.sbatch, train_ctt_ot_proxy.sbatch, train_ctt_no_cycle_proxy.sbatch, eval_ctt_generated_rollout.sbatch.
• Dominance: eval_tanh_train_dominance.sbatch, eval_perdim_trainmax_dominance.sbatch, eval_bundle_consensus_dominance.sbatch, train_utility_energy_selector.sbatch.
• Rendering/export: render_six_task_chart_observations.sbatch, reexport_rgb_ref_cil_charts.sbatch, render_maniskill_observations.sbatch.
• Baselines/generation: generate_6task_h16.sbatch, make_maniskill_collection.sbatch, train_maniskill_*.
• Legacy model training/eval: train_dovla*.sbatch, train_transformer*.sbatch, eval_*.
• HF sync: hf_push_daemon.sbatch.

manifests/

Run manifests and templates.

• manifests/cil_1b_template.yaml: active large template opened in the IDE.
• manifests/cil_160m.yaml: smaller CIL manifest.
• manifests/baselines_full.yaml: full baseline manifest.
• manifests/scaling_k_sweep.yaml: scaling/K sweep.
• manifests/source_score_bonus_pick001_stack005.*: source-score bonus configs.

runs/

Reproducible experiment artifacts. Each serious run should contain:

config.yaml
command.txt
git_hash.txt
data_hash.txt
split_hash.txt
train.log
eval.log
metrics.json
metrics_by_task.json
metrics_by_seed.json
table.tex

Markdown report.md files were removed per the current cleanup request. Use metrics.json, table.tex, logs, and this README instead.

High-value run directories:

• runs/data_accounting: verified data counts.
• runs/leakage_audit: original leakage audit.
• runs/leakage_audit_rgb_refs: RGB-ref leakage audit.
• runs/tangent_reconstruction: original tangent reconstruction check.
• runs/tangent_reconstruction_rgb_refs: RGB-ref tangent reconstruction check.
• runs/action_bound_audit_rgb_refs: action-bound audit.
• runs/chart_observation_embeddings_rgb_refs: RGB-stat embedding export.
• runs/chart_object_embeddings_rgb_refs: object-layout embedding export.
• runs/chart_feature_audit*: feature-source audits.
• runs/ctt_residual_full_seed{0,1,2}: full residual CTT training.
• runs/ctt_residual_ot_full_seed{0,1,2}: completed full residual CTT + OT ablation from Slurm job 15203117; it passes the proxy gate but is worse than plain residual CTT on PPTC and mean positive distance.
• runs/ctt_residual_no_cycle_full_seed{0,1,2}: completed no-cycle residual CTT ablation from Slurm job 15204363; it passes the proxy gate with lower NegativeNear@0.20 but slightly weaker positive support than cycle residual CTT.
• runs/ctt_gated_residual_full_seed{0,1,2}: full gated residual CTT training.
• runs/ctt_residual_component_smoke: real-data CTT training smoke with --no-markdown-report; logs configured loss weights and per-component loss means for Chamfer/OT positive alignment, negative boundary, ranking, cycle, and diversity terms.
• runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison: current strongest measured support artifact.
• runs/ctt_val_proxy_comparison: CTT-vs-local-atlas proxy support gate.
• runs/ctt_outcome_acceptance_gate: measured Part-F outcome acceptance gate.
• runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison: train calibration rows for K=16 env_clip.
• runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test: K=16 LCB dominance auto threshold, negative selector result.
• runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test_tau0: K=16 LCB tau0 fallback, matches base but does not improve.
• runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test: best current train-clean selector diagnostic.
• runs/ctt_base_context_obs_learned_dominance_score_*_envclip_k16_train_to_test: score-shape selector diagnostics; these did not improve the best selector.
• runs/ctt_base_context_obs_learned_dominance_*bundle*_envclip_k16_train_to_test: bundle-consensus selector diagnostics. These test whether agreement among transported train-positive tangents is a useful deployment-visible confidence signal.
• runs/ctt_base_context_obs_learned_dominance_*_tanh_train_to_test: K=8 tanh selector diagnostics from Slurm job 15149814; best selected success is 0.3819, but proposal oracle is also only 0.3819.
• runs/ctt_base_context_obs_learned_dominance_*_perdim_trainmax_train_to_test: per-dimension trainmax selector diagnostics from Slurm job 15149815; these are below the current K=16 env_clip support setting.
• runs/ctt_selector_diagnostic_sweep: generated selector summary table used by the paper to compare K=8 tanh, K=8 per-dim trainmax, K=16 env-clip, and K=16 checkpoint utility-energy selector diagnostics without cherry-picking a single row.
• runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*: fixed nonlinear selector diagnostics.
• runs/summary_ctt.csv: global run summary table.

logs/

Cluster stdout/stderr and daemon logs.

• logs/*.out and logs/*.err: Slurm job output/error files.
• logs/auto_sync_hf.log: legacy HF auto-sync log.
• logs/auto_sync_hf.pid: legacy auto-sync PID file.
• logs/workflow/: workflow logs.

outputs/

Local generated outputs and HF sync state.

• outputs/hf_sync/hf_sync.log: one-shot HF sync log.
• outputs/hf_sync/hf_sync_daemon.log: 15-minute daemon log.
• outputs/hf_sync/last_manifest.json: latest HF sync manifest.
• outputs/hpc/: HPC outputs.
• outputs/external_vla*: external VLA export/probe outputs.
• outputs/manifest_*: manifest smoke outputs.
• outputs/phase5_*: legacy phase-5 outputs.
• outputs/smoke_*, outputs/train_smoke_*: smoke outputs.
• outputs/wheels: local wheel/cache outputs.

reports/

Legacy non-Markdown reports and CSV/JSON summaries.

• reports/phase_a2_evaluation.json: legacy Phase-A2 evaluation artifact.
• reports/phase_a4_results.json: legacy Phase-A4 result artifact.
• reports/hpc_clean_results/clean_result_manifest.json: file manifest for the HPC-clean result sweep.
• reports/hpc_clean_results/clean_result_rows.csv: row-level clean-result table.
• reports/hpc_clean_results/clean_result_summary.csv: aggregate clean-result summary.
• reports/hpc_clean_results/excluded_unclean_paths.txt: paths excluded from the clean-result sweep.

results/

Legacy result files. Markdown summaries were removed. Any remaining non-Markdown files are legacy evidence or machine-readable artifacts. Current CTT evidence should be read from runs/, not results/.

tests/

Regression tests. Key tests:

• tests/test_metrics.py: canonical metric behavior.
• tests/test_causal_action_metrics.py: causal action metric checks.
• tests/test_ctt.py: CTT model/training/eval checks.
• tests/test_chart_features.py: chart feature leakage invariants.
• tests/test_dominance_selector.py: dominance selector, PCCE, safety fields.
• tests/test_cil_schema.py, test_cil_images.py: CIL data/image schema.
• tests/test_maniskill_*: ManiSkill backend/lattice/render/rollout tests.
• tests/test_tangent_*: positive tangent generator baselines.
• tests/test_transfercritic.py: transfer critic checks.
• tests/test_slurm_templates.py: Slurm template sanity.

Current Best Evidence

K=16 env_clip measured support

Run:

runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison

Key held-out test values:

Metric	Value
Rows	144
Base success	0.2917
Score-only selected success	0.2778
Proposal oracle success	0.5694
Hidden chart oracle success	0.7292
OutcomePTR@16	0.5486
Success support gap	0.2014
Success selector gap	0.2917
Action-bound unsafe	0.0000

Interpretation: support is strong; selector is not.

CTT validation proxy support gate

Run:

runs/ctt_val_proxy_comparison

This is a proxy geometry gate, not rollout success. CTT variants pass by improving mean distance to target positives while staying within the NegativeNear@0.20 safety slack; they do not beat local-atlas on PPTC thresholds. The OT variant passes the same proxy gate but does not improve over plain residual CTT. The no-cycle ablation passes too, but is slightly worse than cycle residual CTT on PPTC and mean positive distance. The gated residual variant fails the safety gate.

Method	PPTC@0.20	PPTC@0.40	Neg@0.20	Pos<Neg	MeanPos	Collapse	Gate
local-atlas	0.4058	0.6812	0.0368	0.5998	0.7203	0.0661	baseline
CTT residual full	0.1981	0.6087	0.0296	0.7352	0.4509	0.0681	pass
CTT residual no-cycle full	0.1884	0.6039	0.0194	0.7391	0.4577	0.0681	pass
CTT residual+OT full	0.1787	0.5894	0.0187	0.7284	0.4604	0.0681	pass
CTT residual base+context+obs	0.2464	0.6425	0.0343	0.7717	0.4347	0.0681	pass
CTT gated residual full	0.2319	0.6135	0.0527	0.7248	0.4337	0.0681	fail

Interpretation: CTT improves support distance geometry, but the story is still diagnostic until measured rollout and selection close the outcome gap.

CTT + OT alignment smoke

Runs:

runs/ctt_residual_ot_component_smoke
runs/ctt_residual_ot_component_smoke_val_proxy

Purpose: verify the implemented optional entropic OT alignment path for train_ctt.py before promoting it to a full proxy comparison. This is a code-path smoke, not a method result.

Item	Value
Train charts	16
Neighbor pairs	28
OT alignment weight	0.25
Final OT component	31.3781
Val proxy rows	32
PPTC@0.20	0.3438
PPTC@0.40	0.7188
NegativeNear@0.20	0.0423
Pos<Neg	0.5974
Mean positive distance	0.6672

Full 3-seed validation proxy job completed on 2026-07-04:

15203117 scripts/slurm/train_ctt_ot_proxy.sbatch array=0-2
exit: 0:0 for tasks 0, 1, 2
elapsed: 00:01:27 per task

Full validation result averaged over three seeds:

Metric	Value
Rows	207
PPTC@0.20	0.1787
PPTC@0.40	0.5894
NegativeNear@0.20	0.0187
Pos<Neg	0.7284
Mean positive distance	0.4604
Collapse	0.0681

Interpretation: the OT path is implemented, smoke-tested, and evaluated. It is safer on NegativeNear@0.20 than plain residual CTT, but it is worse on PPTC and mean positive distance. Treat it as an ablation/negative diagnostic, not the main CTT variant.

CTT no-cycle ablation

Purpose: test whether the cycle consistency term in train_ctt.py improves validation proxy support or only adds optimization baggage.

Completed on 2026-07-04:

15204363 scripts/slurm/train_ctt_no_cycle_proxy.sbatch array=0-2
exit: 0:0 for tasks 0, 1, 2
elapsed: 00:01:28 to 00:01:41 per task

Full validation result averaged over three seeds:

Metric	Value
Rows	207
PPTC@0.20	0.1884
PPTC@0.40	0.6039
NegativeNear@0.20	0.0194
Pos<Neg	0.7391
Mean positive distance	0.4577
Collapse	0.0681

Interpretation: removing cycle reduces NegativeNear@0.20, but it also slightly worsens PPTC and mean positive distance versus the cycle residual row. Cycle is not a breakthrough ingredient, but the measured ablation supports keeping it in the default CTT support model.

Best current K=16 train-clean learned selector

Run:

runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test

Key values:

Metric	Value
Selected success	0.3542
Coverage	0.6597
Proposal oracle success	0.5694
Success selector gap	0.2431
Pairwise causal calibration ECE	0.0150
Pair count	12,434

Interpretation: RGB-stat chart compatibility helps, but the selector gap is still too large for a deployment-clean method success claim.

K=16 measured outcome acceptance gate

Run:

runs/ctt_outcome_acceptance_gate

This gate combines the measured K=16 rollout support artifact with the best train-clean K=16 selector. It records which Part-F acceptance bars are met and which are not.

Gate	Observed	Required	Status
Selected clean success	0.3542	>= 0.4745	fail
Target selected clean success	0.3542	>= 0.5000	fail
Proposal oracle success	0.5694	>= 0.5000	pass
Success support gap	0.2014	<= 0.0700	fail
Success selector gap	0.2431	<= 0.0300	fail
OutcomePTR@16 vs V0	0.5486	> 0.4435	pass
Generated unsafe rate	0.0000	<= base+slack	pass
Three train seeds + bootstrap CI	yes	required	pass

Interpretation: CTT passes proposal-oracle, measured OutcomePTR, safety, seed, and CI checks; it fails selected-success, support-gap, and selector-gap bars. The current paper must remain diagnostic rather than claiming final CTT method success.

K=16 score-shape selector diagnostic

New score-relative features were added to test whether visible row-shape signals can close the selector gap. They did not beat the current best chart-compat selector.

Run family	Selected success	Coverage	Success selector gap
score_chart_compat, utility	0.3264	0.6389	0.2500
score_context_chart_compat, utility	0.3264	0.6458	0.2500
score_chart_compat, success bonus 2	0.3194	0.5486	0.2500
score_context_chart_compat, success bonus 2	0.3403	0.6111	0.2292

Interpretation: small deployment-visible selector features are not enough; the paper should keep emphasizing positive tangent support generation and CTT.

K=16 bundle-consensus selector diagnostic

Submitted on 2026-07-04:

15205463 scripts/slurm/eval_bundle_consensus_dominance.sbatch

Purpose: test whether deployment-visible agreement among transported train-positive tangents helps the selector. The feature set uses only generated tangent geometry, row scores, source chart/task identifiers, and train-source evidence where configured. It does not use target positive/negative sets or candidate outcomes when choosing. This is a selector diagnostic, not a new main method contribution. Replace this pending note with measured values after the job completes.

K=8 tanh and per-dimension trainmax selector diagnostics

Jobs:

15149814 scripts/slurm/eval_tanh_train_dominance.sbatch
15149815 scripts/slurm/eval_perdim_trainmax_dominance.sbatch

Both jobs completed with exit code 0:0 on 2026-07-04 and wrote README-only artifacts, with no persistent report.md. The strongest tanh learned selector is context_success with selected success 0.3819, but the tanh proposal oracle is also 0.3819; this is a selector diagnostic, not a stronger CTT support result. The best per-dimension trainmax selector reaches 0.3333 selected success with proposal oracle 0.2222, which is worse than the K=16 env_clip support setting.

Diagnostic family	Best run	Selected success	Proposal oracle	Success selector gap
tanh	context_success	0.3819	0.3819	0.1250
per-dim trainmax	context	0.3333	0.2222	0.0625

Interpretation: these sweeps reinforce the current diagnosis: changing the execution/action transform or lightweight selector features does not yet produce a deployment-clean method success.

K=16 score-source LCB dominance

Runs:

runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test
runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test_tau0

Key values:

Variant	Selected	Coverage	Unsafe exec.	PCCE	Selector gap
auto	0.2778	0.1319	0.0000	0.1633	0.2986
tau0	0.2917	0.1111	0.0000	0.1633	0.2986

Interpretation: the LCB fallback records the requested safety/calibration fields, but it does not solve dominance. It is a negative Part-G diagnostic.

Utility-energy selector diagnostic

Submitted and completed on 2026-07-04:

15177546 scripts/slurm/train_utility_energy_selector.sbatch array=0-2
exit: 0:0 for tasks 0, 1, 2

Purpose: train three base_context_obs utility-energy checkpoints on the train-only RGB-ref chart database, then evaluate checkpoint-scored calibrated dominance on the strongest current measured support setting:

calibration: runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison
evaluation:  runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison
K:           16

Result:

Seed	Selected	Coverage	Fallback	Unsafe	PCCE	Selector gap
0	0.2708	0.1250	0.8750	0.0000	0.0150	0.3194
1	0.2708	0.0972	0.9028	0.0000	0.0090	0.3194
2	0.2778	0.1736	0.8264	0.0000	0.0253	0.3056

Interpretation: checkpoint-scored utility energy does not close the selector gap. It has low coverage and mostly falls back to the base policy, so it is a negative Part-G diagnostic and does not replace the existing train-clean K=16 selector.

How To Run Core Commands

Use the local virtual environment when possible:

.venv/bin/python -m pytest tests/test_metrics.py tests/test_ctt.py tests/test_dominance_selector.py -q

Build the paper:

cd latex
latexmk -pdf -interaction=nonstopmode main.tex

Refresh summary:

.venv/bin/python scripts/summarize_ctt_runs.py \
  --run-root runs \
  --out-csv runs/summary_ctt.csv \
  --no-markdown-report

Run K=16 LCB dominance auto:

.venv/bin/python scripts/eval_dominance_selector.py \
  --calibration-input runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison/combined_measured_candidates.json \
  --calibration-target-index data/cil_charts_rgb_refs/train/index.json \
  --eval-input runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison/combined_measured_candidates.json \
  --eval-target-index data/cil_charts_rgb_refs/test/index.json \
  --checkpoint-template 'runs/ctt_residual_base_context_obs_seed{seed}/model.pt' \
  --out-dir runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test \
  --k 16 \
  --bootstrap-samples 1000 \
  --no-markdown-report

Run leakage audits:

.venv/bin/python scripts/audit_cil_charts.py \
  --chart-root data/cil_charts \
  --run-root runs \
  --out-dir runs/leakage_audit \
  --no-markdown-report

.venv/bin/python scripts/audit_cil_charts.py \
  --chart-root data/cil_charts_rgb_refs \
  --run-root runs \
  --out-dir runs/leakage_audit_rgb_refs \
  --no-markdown-report

Run tangent reconstruction checks:

.venv/bin/python scripts/check_tangent_reconstruction.py \
  --chart-root data/cil_charts \
  --out-dir runs/tangent_reconstruction \
  --no-markdown-report

.venv/bin/python scripts/check_tangent_reconstruction.py \
  --chart-root data/cil_charts_rgb_refs \
  --out-dir runs/tangent_reconstruction_rgb_refs \
  --no-markdown-report

Start HF sync daemon:

bash scripts/hf_push_every_15m.sh

Current local daemon process was previously observed as PID 615094; verify with:

ps -p 615094 -o pid,etimes,cmd

Hugging Face Remote

Repository:

anhtld/vla

The workspace is uploaded under:

workspace/

Large scratch roots are uploaded under:

scratch/

Use:

.venv/bin/hf auth whoami
.venv/bin/hf upload anhtld/vla <local_path> workspace/<remote_path>

Avoid uploading secrets. The sync scripts exclude .env, token/secret/key patterns, virtualenvs, git internals, containers, and native library folders. They also enforce the README-only Markdown policy on the Hub by deleting remote *.md files except workspace/README.md after each successful sync. Set HF_SYNC_CLEAN_REMOTE_MARKDOWN=0 only for a deliberate temporary override.

Development Rules

• Do not claim method success unless the result is implemented, measured, leakage-audited, and logged.
• Do not call distance proxy metrics PTR. Use PPTC for proxy support.
• Do not compute OutcomePTR, SelectorRegret, or SupportGap from proxy-only candidates.
• Train-only retrieval must use train charts only.
• Validation/test outcomes are evaluator-only.
• Keep V0/V1/V3 as diagnostics/baselines, not final method claims.
• Use K=16 env_clip as the current bounded-action diagnostic convention until a better action representation is implemented and measured.
• Treat deterministic object-layout features as a negative result unless a new measured run proves otherwise.
• The next real method work is a stronger deployment-visible chart/outcome representation and dominance model, not more wrapper text.

Immediate Next Actions

1. Replace hand-built RGB/object descriptors with learned visual-language or task/object/contact-stage tokens.
2. Train a stronger train-only utility/dominance model under the K=16 env_clip convention.
3. Re-run measured selection on held-out test after the representation fix.
4. Add actual collision/contact safety labels beyond action-bound validity.
5. Keep the paper honest: support is promising, selector is not solved.