Auto-sync: 2026-06-28 01:00:39 (part 3)

results/paper_story_memo.md

@@ -17,9 +17,10 @@ when queried on proposal geometry that matches those local counterfactuals.
 | Conservative same-state result is large | no-expert lattice is 56.99% vs 29.74% policy | Main result |
 | Full lattice gives upper result | full lattice is 69.33%, oracle is 86.78% | Strong but label expert proposal clearly |
 | Deployment-clean proposal is currently a bottleneck | best clean proposal+field sweep is 32.93%, far below 56.99% | Supported |
-| Gradient-based field optimization can solve the clean proposal gap | `field_optim` jobs are pending | Not yet known |
-| A broader non-expert proposal target can reduce the proposal gap | `nonexpert_policy_bc5` jobs are pending; target census shows 888/2873 states have best non-expert action outside `near_miss` | Not yet known |
-| Counterfactual residuals transfer better than absolute retrieved actions | `retrieval_residual` jobs are pending; absolute retrieval was 28.93% | Not yet known |
+| Gradient-based field optimization does not solve the clean proposal gap | `field_optim` best observed result is 25.39% | Negative diagnostic |
+| A broader non-expert proposal target does not reduce the proposal gap | direct broad non-expert policy is 27.88%; with field scoring it is 26.49% | Negative diagnostic |
+| Counterfactual residuals transfer better than absolute retrieved actions | nearest residual retrieval is 32.12% vs absolute retrieval 28.93%; KNN4 residual drops to 29.91% | Supported as a clean bridge |
+| Field-teacher distillation may turn the same-state rule into a policy | train-split and aligned-validation target-map jobs are running/pending | Pending |
 
 ## Main Table Candidate
 
@@ -31,10 +32,13 @@ clean proposal result, the intended main rows are:
 2. Gaussian field search: 29.10%
 3. Retrieval lattice, no expert: 27.13%
 4. Near-miss proposal + field, BC x5 field checkpoint: 32.93%
-5. Lattice, near-miss only: 55.94%
-6. Lattice, no expert: 56.99%
-7. Lattice, full: 69.33%
-8. Oracle ceiling: 86.78%
+5. Trust-region field optimization: 25.39%
+6. Broad non-expert proposal + field: 26.49%
+7. Train-state residual retrieval: 32.12%
+8. Lattice, near-miss only: 55.94%
+9. Lattice, no expert: 56.99%
+10. Lattice, full: 69.33%
+11. Oracle ceiling: 86.78%
 
 ## Novelty Framing
 
@@ -56,57 +60,33 @@ test-time search. The cleaner novelty is:
 |---|---|---|
 | Same-state lattice is not deployment-clean | show no-expert lattice and near-miss-only mechanism; show retrieval/Gaussian failures | improve clean proposal route |
 | Full lattice includes expert proposal | label as upper deployment/ceiling, not main conservative result | keep no-expert row as main |
-| Gains are from candidate leakage, not learning | selection never reads candidate rewards; no-expert and near-miss-only isolate mechanism | add field_optim/proposal model evidence |
+| Gains are from candidate leakage, not learning | selection never reads candidate rewards; no-expert and near-miss-only isolate mechanism; field_optim and broad proposal BC fail | add field-teacher distillation evidence |
 | Method is just a bundle of tricks | use mechanism ablations to show one central idea: local counterfactual field | avoid presenting unrelated variants as core |
 | Not SOTA enough | current clean deploy result is modest | need external baselines and stronger proposal generator before claiming SOTA |
 
 ## Active Jobs
 
-Last checked: `2026-06-28 04:51 UTC`.
-
-- `14842523`: GPU smoke for `selection_mode=field_optim`.
-- `14842557`: low-resource CPU unit smoke for the pure action-optimization helper.
-- `14842528`: 4-config x 3-seed `field_optim` sweep, dependent on `14842523`.
-- `14842529`: sweep summary, dependent on `14842528`.
-- `14842551`: after-any fallback summary for partial sweep results.
-- `14842574`: train `nonexpert_policy_bc5` on best non-expert local interventions.
-- `14842575`: direct rollout eval for `nonexpert_policy_bc5`.
-- `14842616`: direct rollout summary.
-- `14842577`: field-selection sweep for `nonexpert_policy_bc5`.
-- `14842617`: field-selection summary.
-- `14842596`: smoke-test `retrieval_residual`, which translates nearest
-  train-state counterfactual residuals around the policy mean.
-- `14842597`: full `retrieval_residual` rollout after smoke.
-- `14842618`: `retrieval_residual` summary.
-- `14842609`: smoke-test `retrieval_residual` with `RETRIEVAL_NEIGHBORS=4`.
-- `14842610`: full KNN4 `retrieval_residual` rollout after smoke.
-- `14842619`: KNN4 `retrieval_residual` summary.
-- `14842646`: CPU unit smoke for the KNN residual helper. Completed.
-- `14857111`: fixed nearest-1 `retrieval_residual` smoke.
-- `14857112`: fixed nearest-1 `retrieval_residual` full rollout. Completed.
-- `14857113`: fixed nearest-1 `retrieval_residual` summary. Completed.
-- `14857114`: fixed KNN4 `retrieval_residual` smoke. Completed.
-- `14857115`: fixed KNN4 `retrieval_residual` full rollout. Completed.
-- `14857116`: fixed KNN4 `retrieval_residual` summary. Completed.
-- `14857117`: rebuild `paper_table_status.*` after fixed residual summaries. Completed.
-- `14857692`: smoke nearest-1 transferred near-miss residual retrieval. Completed.
-- `14857693`: full nearest-1 transferred near-miss residual retrieval. Canceled.
-- `14857694`: summary nearest-1 transferred near-miss residual retrieval. Canceled.
-- `14857695`: smoke KNN4 transferred near-miss residual retrieval. Completed.
-- `14857696`: full KNN4 transferred near-miss residual retrieval. Canceled.
-- `14857697`: summary KNN4 transferred near-miss residual retrieval. Canceled.
-- `14857698`: rebuild `paper_table_status.*` after near-miss residual summaries. Canceled.
-
-Current scheduler state: no tracked jobs are active. `field_optim`,
-`nonexpert_policy_bc5`, and residual v2 jobs completed. Residual nearest-1 is a
-positive clean bridge at 32.12%, KNN4 residual is 29.91%, and near-miss-only
-residual smoke was weak enough to cancel its full jobs.
-
-## Decision Rule For Field Optim Jobs
-
-- If `field_optim` beats 32.93% but remains below 40%, keep it as a better
-  deployment-clean positive control, not the main result.
-- If `field_optim` reaches 40-50%, promote it to the main clean-deployment bridge
-  and frame same-state lattice as mechanistic supervision/upper bound.
-- If `field_optim` fails or stays near 30%, keep it as a negative ablation and
-  prioritize training a proposal model on successful non-expert lattice candidates.
+Last checked: `2026-06-28 05:03 UTC`.
+
+- `14858328`: running 3 seeds of `field_selected_noexpert_bc5` using the
+  train-split field-selected no-expert target map.
+- `14858329`/`14858330`: direct rollout evaluation and summary for that student.
+- `14858331`/`14858332`: field-guided rollout sweep and summary for that student.
+- `14858333`: rebuild `paper_table_status.*` after those summaries.
+- `14858449`: completed export of an all-split target map for aligned validation
+  checkpoint selection and seed-invariant student train coverage.
+- `14858450`: pending 3-seed `field_selected_noexpert_bc5_allmap` training.
+- `14858451`/`14858452`: direct rollout evaluation and summary for allmap.
+- `14858453`/`14858454`: field-guided rollout sweep and summary for allmap.
+- `14858455`: rebuild `paper_table_status.*` after allmap summaries.
+
+## Decision Rule For Field-Teacher Jobs
+
+- If allmap field-teacher distillation beats 32.93%, promote it as the best
+  deployment-clean bridge and keep same-state lattice as the mechanism result.
+- If it lands near residual retrieval, present residual retrieval and
+  field-teacher distillation as complementary evidence for transferable local
+  counterfactual geometry.
+- If it fails, keep the central paper story focused on the same-state mechanism
+  and the clean-proposal bottleneck, with residual retrieval as the strongest
+  deployment-clean bridge.

results/paper_table_status.json

@@ -113,7 +113,7 @@
       "clean_deployment": "yes",
       "same_state_proposals": "no",
       "expert_proposal": "no",
-      "story_role": "pending broader non-expert proposal model",
+      "story_role": "broader non-expert proposal-model ablation",
       "fallback_success": null,
       "pending_job": "14842574/14842575/14842616",
       "path_exists": true,
@@ -132,7 +132,7 @@
       "clean_deployment": "yes",
       "same_state_proposals": "no",
       "expert_proposal": "no",
-      "story_role": "pending broader proposal-field bridge",
+      "story_role": "broader proposal-field ablation",
       "fallback_success": null,
       "pending_job": "14842574/14842577/14842617",
       "path_exists": true,
@@ -148,6 +148,82 @@
       "best_config": "k64_sigma0.50",
       "gain_vs_h16_policy": -0.03246376811594204
     },
+    {
+      "key": "field_selected_noexpert_policy",
+      "label": "Field-selected no-expert distillation policy",
+      "path": "h16_policy_ckpt_field_selected_noexpert_bc5_summary.json",
+      "clean_deployment": "yes",
+      "same_state_proposals": "no",
+      "expert_proposal": "no",
+      "story_role": "student of field-on-lattice teacher",
+      "fallback_success": null,
+      "pending_job": "14858327/14858328/14858329/14858330",
+      "path_exists": false,
+      "status": "pending",
+      "success": null,
+      "std_success": null,
+      "completed_seeds": null,
+      "num_completed": null,
+      "best_config": null,
+      "gain_vs_h16_policy": null
+    },
+    {
+      "key": "field_selected_noexpert_policy_field",
+      "label": "Field-selected no-expert distillation + field",
+      "path": "h16_policy_ckpt_field_selected_noexpert_bc5_bestpt_field_sweep_summary.json",
+      "clean_deployment": "yes",
+      "same_state_proposals": "no",
+      "expert_proposal": "no",
+      "story_role": "student proposal with field scoring",
+      "fallback_success": null,
+      "pending_job": "14858327/14858328/14858331/14858332",
+      "path_exists": false,
+      "status": "pending",
+      "success": null,
+      "std_success": null,
+      "completed_seeds": null,
+      "num_completed": null,
+      "best_config": null,
+      "gain_vs_h16_policy": null
+    },
+    {
+      "key": "field_selected_noexpert_policy_allmap",
+      "label": "Field-selected no-expert distillation policy, aligned validation",
+      "path": "h16_policy_ckpt_field_selected_noexpert_bc5_allmap_summary.json",
+      "clean_deployment": "yes",
+      "same_state_proposals": "no",
+      "expert_proposal": "no",
+      "story_role": "field-teacher student with aligned checkpoint selection",
+      "fallback_success": null,
+      "pending_job": "14858449/14858450/14858451/14858452",
+      "path_exists": false,
+      "status": "pending",
+      "success": null,
+      "std_success": null,
+      "completed_seeds": null,
+      "num_completed": null,
+      "best_config": null,
+      "gain_vs_h16_policy": null
+    },
+    {
+      "key": "field_selected_noexpert_policy_allmap_field",
+      "label": "Field-selected no-expert distillation + field, aligned validation",
+      "path": "h16_policy_ckpt_field_selected_noexpert_bc5_allmap_bestpt_field_sweep_summary.json",
+      "clean_deployment": "yes",
+      "same_state_proposals": "no",
+      "expert_proposal": "no",
+      "story_role": "aligned field-teacher student with field scoring",
+      "fallback_success": null,
+      "pending_job": "14858449/14858450/14858453/14858454",
+      "path_exists": false,
+      "status": "pending",
+      "success": null,
+      "std_success": null,
+      "completed_seeds": null,
+      "num_completed": null,
+      "best_config": null,
+      "gain_vs_h16_policy": null
+    },
     {
       "key": "retrieval_residual",
       "label": "Train-state counterfactual residual retrieval",
@@ -155,7 +231,7 @@
       "clean_deployment": "yes",
       "same_state_proposals": "no",
       "expert_proposal": "no",
-      "story_role": "pending transferable local tangent proposal",
+      "story_role": "transferable local tangent proposal",
       "fallback_success": null,
       "pending_job": "14857111/14857112/14857113",
       "path_exists": true,
@@ -174,7 +250,7 @@
       "clean_deployment": "yes",
       "same_state_proposals": "no",
       "expert_proposal": "no",
-      "story_role": "pending KNN tangent proposal",
+      "story_role": "KNN tangent proposal ablation",
       "fallback_success": null,
       "pending_job": "14857114/14857115/14857116",
       "path_exists": true,

results/paper_table_status.md

@@ -9,10 +9,14 @@ Baseline h=16 policy: 29.74%
 | retrieval_lattice_no_expert | Nearest train-state lattice, no expert | complete | 27.13% | -2.61 pp | yes | no | no | negative generic action-library ablation |
 | near_miss_policy_bc5_field | Near-miss proposal policy + field | complete k64_sigma0.50 | 32.93% | +3.19 pp | yes | no | no | current best clean deployment bridge |
 | field_optim | Trust-region field optimization | complete k32_sigma0.50 | 25.39% | -4.35 pp | yes | no | no | differentiable field-ascent diagnostic |
-| nonexpert_policy_bc5 | Best non-expert proposal policy | complete | 27.88% | -1.86 pp | yes | no | no | pending broader non-expert proposal model |
-| nonexpert_policy_bc5_field | Best non-expert proposal policy + field | complete k64_sigma0.50 | 26.49% | -3.25 pp | yes | no | no | pending broader proposal-field bridge |
-| retrieval_residual | Train-state counterfactual residual retrieval | complete | 32.12% | +2.38 pp | yes | no | no | pending transferable local tangent proposal |
-| retrieval_residual_knn4 | KNN counterfactual residual retrieval | complete | 29.91% | +0.17 pp | yes | no | no | pending KNN tangent proposal |
+| nonexpert_policy_bc5 | Best non-expert proposal policy | complete | 27.88% | -1.86 pp | yes | no | no | broader non-expert proposal-model ablation |
+| nonexpert_policy_bc5_field | Best non-expert proposal policy + field | complete k64_sigma0.50 | 26.49% | -3.25 pp | yes | no | no | broader proposal-field ablation |
+| field_selected_noexpert_policy | Field-selected no-expert distillation policy | pending 14858327/14858328/14858329/14858330 | pending | pending | yes | no | no | student of field-on-lattice teacher |
+| field_selected_noexpert_policy_field | Field-selected no-expert distillation + field | pending 14858327/14858328/14858331/14858332 | pending | pending | yes | no | no | student proposal with field scoring |
+| field_selected_noexpert_policy_allmap | Field-selected no-expert distillation policy, aligned validation | pending 14858449/14858450/14858451/14858452 | pending | pending | yes | no | no | field-teacher student with aligned checkpoint selection |
+| field_selected_noexpert_policy_allmap_field | Field-selected no-expert distillation + field, aligned validation | pending 14858449/14858450/14858453/14858454 | pending | pending | yes | no | no | aligned field-teacher student with field scoring |
+| retrieval_residual | Train-state counterfactual residual retrieval | complete | 32.12% | +2.38 pp | yes | no | no | transferable local tangent proposal |
+| retrieval_residual_knn4 | KNN counterfactual residual retrieval | complete | 29.91% | +0.17 pp | yes | no | no | KNN tangent proposal ablation |
 | near_miss_only_lattice | Same-state lattice, near-miss only | complete | 55.94% | +26.20 pp | no | yes | no | minimal mechanism result |
 | no_expert_lattice | Same-state lattice, no expert | complete | 56.99% | +27.25 pp | no | yes | no | main conservative mechanism result |
 | no_near_miss_no_expert_lattice | Same-state lattice, no expert/no near-miss | complete | 25.57% | -4.17 pp | no | yes | no | mechanism knockout |

scripts/build_paper_table_status.py

@@ -82,7 +82,7 @@ SPECS = [
         clean_deployment="yes",
         same_state_proposals="no",
         expert_proposal="no",
-        story_role="pending broader non-expert proposal model",
+        story_role="broader non-expert proposal-model ablation",
         pending_job="14842574/14842575/14842616",
     ),
     ResultSpec(
@@ -92,7 +92,7 @@ SPECS = [
         clean_deployment="yes",
         same_state_proposals="no",
         expert_proposal="no",
-        story_role="pending broader proposal-field bridge",
+        story_role="broader proposal-field ablation",
         pending_job="14842574/14842577/14842617",
     ),
     ResultSpec(
@@ -102,7 +102,7 @@ SPECS = [
         clean_deployment="yes",
         same_state_proposals="no",
         expert_proposal="no",
-        story_role="pending student of field-on-lattice teacher",
+        story_role="student of field-on-lattice teacher",
         pending_job="14858327/14858328/14858329/14858330",
     ),
     ResultSpec(
@@ -112,9 +112,29 @@ SPECS = [
         clean_deployment="yes",
         same_state_proposals="no",
         expert_proposal="no",
-        story_role="pending student proposal with field scoring",
+        story_role="student proposal with field scoring",
         pending_job="14858327/14858328/14858331/14858332",
     ),
+    ResultSpec(
+        key="field_selected_noexpert_policy_allmap",
+        label="Field-selected no-expert distillation policy, aligned validation",
+        path="h16_policy_ckpt_field_selected_noexpert_bc5_allmap_summary.json",
+        clean_deployment="yes",
+        same_state_proposals="no",
+        expert_proposal="no",
+        story_role="field-teacher student with aligned checkpoint selection",
+        pending_job="14858449/14858450/14858451/14858452",
+    ),
+    ResultSpec(
+        key="field_selected_noexpert_policy_allmap_field",
+        label="Field-selected no-expert distillation + field, aligned validation",
+        path="h16_policy_ckpt_field_selected_noexpert_bc5_allmap_bestpt_field_sweep_summary.json",
+        clean_deployment="yes",
+        same_state_proposals="no",
+        expert_proposal="no",
+        story_role="aligned field-teacher student with field scoring",
+        pending_job="14858449/14858450/14858453/14858454",
+    ),
     ResultSpec(
         key="retrieval_residual",
         label="Train-state counterfactual residual retrieval",
@@ -122,7 +142,7 @@ SPECS = [
         clean_deployment="yes",
         same_state_proposals="no",
         expert_proposal="no",
-        story_role="pending transferable local tangent proposal",
+        story_role="transferable local tangent proposal",
         pending_job="14857111/14857112/14857113",
     ),
     ResultSpec(
@@ -132,7 +152,7 @@ SPECS = [
         clean_deployment="yes",
         same_state_proposals="no",
         expert_proposal="no",
-        story_role="pending KNN tangent proposal",
+        story_role="KNN tangent proposal ablation",
         pending_job="14857114/14857115/14857116",
     ),
     ResultSpec(

scripts/slurm/export_field_selected_policy_targets.sbatch

@@ -0,0 +1,53 @@
+#!/bin/bash
+#SBATCH --job-name=export_field_targets
+#SBATCH --account=def-yalda_gpu
+#SBATCH --nodes=1
+#SBATCH --ntasks=1
+#SBATCH --cpus-per-task=2
+#SBATCH --gres=gpu:nvidia_h100_80gb_hbm3_1g.10gb:1
+#SBATCH --mem=12G
+#SBATCH --time=00:30:00
+#SBATCH --output=outputs/hpc/logs/%x_%j.out
+#SBATCH --error=outputs/hpc/logs/%x_%j.err
+
+set -euo pipefail
+
+PROJECT_DIR="${PROJECT_DIR:-$SLURM_SUBMIT_DIR}"
+SCRATCH_ROOT="/scratch/$USER/dovla"
+SIF="${SIF:-$SCRATCH_ROOT/containers/pytorch_2.7.1_cuda12.8.sif}"
+PYTHON="${PYTHON:-$SCRATCH_ROOT/envs/maniskill/bin/python}"
+DATASET="${DATASET:-$SCRATCH_ROOT/experiments/six_task_h16_collection}"
+CHECKPOINT="${CHECKPOINT:?Set CHECKPOINT to a trained DoVLA checkpoint}"
+OUT="${OUT:?Set OUT to the target-map JSON path}"
+SPLIT="${SPLIT:-train}"
+EXCLUDE_TYPES="${EXCLUDE_TYPES:-expert}"
+BATCH_GROUPS="${BATCH_GROUPS:-32}"
+MAX_GROUPS="${MAX_GROUPS:-}"
+
+module load StdEnv/2023 apptainer/1.4.5
+cd "$PROJECT_DIR"
+mkdir -p outputs/hpc/logs "$(dirname "$OUT")"
+
+export OMP_NUM_THREADS=1
+export OPENBLAS_NUM_THREADS=1
+export MKL_NUM_THREADS=1
+export DOVLA_TORCH_THREADS=1
+
+EXTRA_ARGS=()
+if [[ -n "$MAX_GROUPS" ]]; then
+  EXTRA_ARGS+=(--max-groups "$MAX_GROUPS")
+fi
+
+apptainer exec --nv \
+  --env "OMP_NUM_THREADS=1,OPENBLAS_NUM_THREADS=1,MKL_NUM_THREADS=1,DOVLA_TORCH_THREADS=1,PYTHONDONTWRITEBYTECODE=1" \
+  -B "$PROJECT_DIR:$PROJECT_DIR" \
+  -B "/scratch/$USER:/scratch/$USER" \
+  "$SIF" "$PYTHON" scripts/export_field_selected_policy_targets.py \
+    --checkpoint "$CHECKPOINT" \
+    --dataset "$DATASET" \
+    --out "$OUT" \
+    --device cuda \
+    --split "$SPLIT" \
+    --exclude-types "$EXCLUDE_TYPES" \
+    --batch-groups "$BATCH_GROUPS" \
+    "${EXTRA_ARGS[@]}"