Auto-sync: 2026-06-28 01:59:41 (part 2)

results/paper_story_memo.md

@@ -25,6 +25,7 @@ when queried on proposal geometry that matches those local counterfactuals.
 | Seed-0 train-split field-teacher distillation does not solve the proposal gap | direct student is 26.84%; with field scoring it is 27.65% | Negative diagnostic |
 | All-split field-teacher distillation does not fix checkpointing/coverage | allmap direct is 28.00%; field-guided best is 26.49% despite 100% target coverage | Negative diagnostic |
 | Residual family consistency is the next hypothesis | field-selected random/wrong-direction residuals have low rollout success; masked residual jobs are active | Active |
+| Retrieval metric locality is the next hypothesis | z-score train-bank retrieval jobs are active after unit smoke `14859165` passed | Active |
 
 ## Main Table Candidate
 
@@ -90,15 +91,21 @@ Last checked: `2026-06-28 05:47 UTC`.
 - `14859041`: completed CPU Apptainer unit smoke for hybrid residual+Gaussian selection.
 - `14859042`-`14859046`: completed hybrid residual+Gaussian jobs; K32 reaches
   31.30% and K64 reaches 30.90%, both below residual-only transport.
-- `14859141`/`14859142`: active masked residual eval/summary, scale `0.50`,
+- `14859188`/`14859189`: active masked residual eval/summary, scale `0.50`,
   excluding `residual_random_negative`.
-- `14859143`/`14859144`: active masked residual eval/summary, scale `0.50`,
+- `14859191`/`14859192`: active masked residual eval/summary, scale `0.50`,
   excluding `residual_random_negative` and `residual_wrong_direction`.
-- `14859145`/`14859146`: active masked residual eval/summary, scale `0.25`,
+- `14859193`/`14859194`: active masked residual eval/summary, scale `0.25`,
   excluding `residual_random_negative` and `residual_wrong_direction`.
-- `14859147`/`14859148`: active typed residual eval/summary, scale `0.50`,
+- `14859195`/`14859196`: active typed residual eval/summary, scale `0.50`,
   keeping policy/no-op/wrong-gripper residual families.
-- `14859149`: rebuild `paper_table_status.*` after masked residual summaries.
+- `14859203`: rebuild `paper_table_status.*` after all masked and z-score summaries.
+- `14859165`: completed Apptainer unit smoke for z-score retrieval metric.
+- `14859197`/`14859198`: active z-score retrieval eval/summary, scale `0.50`.
+- `14859199`/`14859200`: active z-score retrieval eval/summary, scale `0.50`,
+  excluding `residual_random_negative` and `residual_wrong_direction`.
+- `14859201`/`14859202`: active z-score retrieval eval/summary, scale `0.25`,
+  excluding `residual_random_negative` and `residual_wrong_direction`.
 
 ## Decision Rule For Masked Residual Jobs
 
@@ -109,3 +116,13 @@ Last checked: `2026-06-28 05:47 UTC`.
   residual result and present masking as a diagnostic of field over-selection.
 - If masks fail, keep the story focused on residual scale calibration and the
   larger same-state counterfactual mechanism.
+
+## Decision Rule For Z-Score Retrieval Jobs
+
+- If z-score retrieval beats 33.33%, promote state-normalized tangent retrieval
+  as the best deployment-clean bridge.
+- If z-score masks only help with the anti-goal residual exclusions, frame
+  retrieval locality and residual family consistency as two sides of the same
+  tangent-transport bottleneck.
+- If z-score retrieval fails, keep the raw nearest-state residual result as the
+  clean bridge and treat metric normalization as a negative ablation.

results/paper_table_status.json

@@ -298,7 +298,7 @@
       "expert_proposal": "no",
       "story_role": "state-normalized tangent retrieval ablation",
       "fallback_success": null,
-      "pending_job": "",
+      "pending_job": "14859166/14859167",
       "path_exists": false,
       "status": "pending",
       "success": null,
@@ -317,7 +317,7 @@
       "expert_proposal": "no",
       "story_role": "state-normalized typed tangent retrieval ablation",
       "fallback_success": null,
-      "pending_job": "",
+      "pending_job": "14859168/14859169",
       "path_exists": false,
       "status": "pending",
       "success": null,
@@ -336,7 +336,7 @@
       "expert_proposal": "no",
       "story_role": "state-normalized typed tangent retrieval ablation",
       "fallback_success": null,
-      "pending_job": "",
+      "pending_job": "14859170/14859171",
       "path_exists": false,
       "status": "pending",
       "success": null,

results/paper_table_status.md

@@ -18,9 +18,9 @@ Baseline h=16 policy: 29.74%
 | retrieval_residual | Train-state counterfactual residual retrieval | complete | 32.12% | +2.38 pp | yes | no | no | transferable local tangent proposal |
 | retrieval_residual_scale025 | Train-state residual retrieval, scale 0.25 | complete | 32.93% | +3.19 pp | yes | no | no | tangent transport scale ablation |
 | retrieval_residual_scale050 | Train-state residual retrieval, scale 0.50 | complete | 33.33% | +3.59 pp | yes | no | no | tangent transport scale ablation |
-| retrieval_residual_scale050_zscore | Train-state residual retrieval, scale 0.50, z-score retrieval | pending | pending | pending | yes | no | no | state-normalized tangent retrieval ablation |
-| retrieval_residual_scale050_zscore_no_random_wrongdir | Train-state residual retrieval, scale 0.50, z-score retrieval, no random/wrong-direction residuals | pending | pending | pending | yes | no | no | state-normalized typed tangent retrieval ablation |
-| retrieval_residual_scale025_zscore_no_random_wrongdir | Train-state residual retrieval, scale 0.25, z-score retrieval, no random/wrong-direction residuals | pending | pending | pending | yes | no | no | state-normalized typed tangent retrieval ablation |
+| retrieval_residual_scale050_zscore | Train-state residual retrieval, scale 0.50, z-score retrieval | pending 14859166/14859167 | pending | pending | yes | no | no | state-normalized tangent retrieval ablation |
+| retrieval_residual_scale050_zscore_no_random_wrongdir | Train-state residual retrieval, scale 0.50, z-score retrieval, no random/wrong-direction residuals | pending 14859168/14859169 | pending | pending | yes | no | no | state-normalized typed tangent retrieval ablation |
+| retrieval_residual_scale025_zscore_no_random_wrongdir | Train-state residual retrieval, scale 0.25, z-score retrieval, no random/wrong-direction residuals | pending 14859170/14859171 | pending | pending | yes | no | no | state-normalized typed tangent retrieval ablation |
 | retrieval_residual_scale050_no_random | Train-state residual retrieval, scale 0.50, no random residuals | pending 14859141/14859142 | pending | pending | yes | no | no | anti-goal residual family mask ablation |
 | retrieval_residual_scale050_no_random_wrongdir | Train-state residual retrieval, scale 0.50, no random/wrong-direction residuals | pending 14859143/14859144 | pending | pending | yes | no | no | anti-goal residual family mask ablation |
 | retrieval_residual_scale025_no_random_wrongdir | Train-state residual retrieval, scale 0.25, no random/wrong-direction residuals | pending 14859145/14859146 | pending | pending | yes | no | no | anti-goal residual family mask ablation |

scripts/build_paper_table_status.py

@@ -173,6 +173,7 @@ SPECS = [
         same_state_proposals="no",
         expert_proposal="no",
         story_role="state-normalized tangent retrieval ablation",
+        pending_job="14859197/14859198",
     ),
     ResultSpec(
         key="retrieval_residual_scale050_zscore_no_random_wrongdir",
@@ -182,6 +183,7 @@ SPECS = [
         same_state_proposals="no",
         expert_proposal="no",
         story_role="state-normalized typed tangent retrieval ablation",
+        pending_job="14859199/14859200",
     ),
     ResultSpec(
         key="retrieval_residual_scale025_zscore_no_random_wrongdir",
@@ -191,6 +193,7 @@ SPECS = [
         same_state_proposals="no",
         expert_proposal="no",
         story_role="state-normalized typed tangent retrieval ablation",
+        pending_job="14859201/14859202",
     ),
     ResultSpec(
         key="retrieval_residual_scale050_no_random",
@@ -200,7 +203,7 @@ SPECS = [
         same_state_proposals="no",
         expert_proposal="no",
         story_role="anti-goal residual family mask ablation",
-        pending_job="14859141/14859142",
+        pending_job="14859188/14859189",
     ),
     ResultSpec(
         key="retrieval_residual_scale050_no_random_wrongdir",
@@ -210,7 +213,7 @@ SPECS = [
         same_state_proposals="no",
         expert_proposal="no",
         story_role="anti-goal residual family mask ablation",
-        pending_job="14859143/14859144",
+        pending_job="14859191/14859192",
     ),
     ResultSpec(
         key="retrieval_residual_scale025_no_random_wrongdir",
@@ -220,7 +223,7 @@ SPECS = [
         same_state_proposals="no",
         expert_proposal="no",
         story_role="anti-goal residual family mask ablation",
-        pending_job="14859145/14859146",
+        pending_job="14859193/14859194",
     ),
     ResultSpec(
         key="retrieval_residual_scale050_safe_types",
@@ -230,7 +233,7 @@ SPECS = [
         same_state_proposals="no",
         expert_proposal="no",
         story_role="typed tangent-family mask ablation",
-        pending_job="14859147/14859148",
+        pending_job="14859195/14859196",
     ),
     ResultSpec(
         key="retrieval_residual_scale075",

scripts/slurm/smoke_retrieval_metric_unit.sbatch

@@ -0,0 +1,123 @@
+#!/bin/bash
+#SBATCH --job-name=smoke_retrieval_metric
+#SBATCH --account=def-yalda
+#SBATCH --nodes=1
+#SBATCH --ntasks=1
+#SBATCH --cpus-per-task=1
+#SBATCH --mem=1G
+#SBATCH --time=00:05:00
+#SBATCH --output=outputs/hpc/logs/%x_%j.out
+#SBATCH --error=outputs/hpc/logs/%x_%j.err
+
+set -euo pipefail
+
+PROJECT_DIR="${PROJECT_DIR:-$SLURM_SUBMIT_DIR}"
+SCRATCH_ROOT="/scratch/$USER/dovla"
+SIF="${SIF:-$SCRATCH_ROOT/containers/pytorch_2.7.1_cuda12.8.sif}"
+PYTHON="${PYTHON:-$SCRATCH_ROOT/envs/maniskill/bin/python}"
+
+module load StdEnv/2023 apptainer/1.4.5
+cd "$PROJECT_DIR"
+mkdir -p outputs/hpc/logs
+
+export OMP_NUM_THREADS=1
+export OPENBLAS_NUM_THREADS=1
+export MKL_NUM_THREADS=1
+export DOVLA_TORCH_THREADS=1
+
+apptainer exec \
+  --env "OMP_NUM_THREADS=1,OPENBLAS_NUM_THREADS=1,MKL_NUM_THREADS=1,DOVLA_TORCH_THREADS=1,PYTHONDONTWRITEBYTECODE=1" \
+  -B "$PROJECT_DIR:$PROJECT_DIR" \
+  -B "/scratch/$USER:/scratch/$USER" \
+  "$SIF" "$PYTHON" - <<'PY'
+from types import SimpleNamespace
+from pathlib import Path
+
+import numpy as np
+
+from dovla_cil.data.schema import ActionChunk
+from dovla_cil.eval.maniskill_policy_rollout import (
+    _RolloutCase,
+    _attach_retrieved_residual_candidates,
+)
+
+
+def record(group_id, candidate_type, action_value, feature):
+    return SimpleNamespace(
+        group_id=group_id,
+        task_id="PickCube-v1",
+        candidate_type=candidate_type,
+        record_id=f"{group_id}-{candidate_type}-{action_value}",
+        observation_inline={"features": feature},
+        action_chunk=ActionChunk(
+            representation="continuous",
+            horizon=1,
+            values=[[action_value, 0.0]],
+        ),
+    )
+
+
+groups = {
+    "train_a": [
+        record("train_a", "expert", 1.0, [0.0, 0.0]),
+        record("train_a", "near_miss", 1.1, [0.0, 0.0]),
+    ],
+    "train_b": [
+        record("train_b", "expert", 2.0, [10.0, 1.0]),
+        record("train_b", "near_miss", 2.2, [10.0, 1.0]),
+    ],
+    "train_c": [
+        record("train_c", "expert", 3.0, [11.0, 1.0]),
+        record("train_c", "near_miss", 3.3, [11.0, 1.0]),
+    ],
+    "heldout": [
+        record("heldout", "expert", 9.0, [0.0, 1.0]),
+        record("heldout", "near_miss", 9.9, [0.0, 1.0]),
+    ],
+}
+dataset = SimpleNamespace(
+    group_ids=list(groups),
+    get_group=lambda group_id: groups[group_id],
+)
+case = _RolloutCase(
+    group_id="heldout",
+    task_id="PickCube-v1",
+    source_dataset=Path("."),
+    state={},
+    observation={"features": [0.0, 1.0]},
+    instruction="pick",
+    oracle_score=1.0,
+    oracle_success=True,
+    expert_score=1.0,
+    expert_success=True,
+    best_action_values=[[9.9, 0.0]],
+    candidate_action_values=[],
+    candidate_types=[],
+)
+
+[raw_attached] = _attach_retrieved_residual_candidates(
+    dataset,
+    [case],
+    heldout_group_ids=["heldout"],
+    obs_dim=2,
+    observation_mode="state",
+    retrieval_neighbors=1,
+    retrieval_metric="raw",
+)
+[zscore_attached] = _attach_retrieved_residual_candidates(
+    dataset,
+    [case],
+    heldout_group_ids=["heldout"],
+    obs_dim=2,
+    observation_mode="state",
+    retrieval_neighbors=1,
+    retrieval_metric="zscore",
+)
+
+assert raw_attached.candidate_source_group_id == "train_a", raw_attached
+assert zscore_attached.candidate_source_group_id == "train_b", zscore_attached
+expected = np.asarray([[[0.0, 0.0]], [[0.2, 0.0]]], dtype=np.float32)
+actual = np.asarray(zscore_attached.candidate_action_values, dtype=np.float32)
+assert np.allclose(actual, expected), actual
+print({"status": "ok", "raw": raw_attached.candidate_source_group_id, "zscore": zscore_attached.candidate_source_group_id})
+PY