sync paper source after proxy gate refresh
Browse files- latex/main.tex +571 -62
latex/main.tex
CHANGED
|
@@ -9,12 +9,13 @@
|
|
| 9 |
\usepackage{xcolor}
|
| 10 |
\usepackage{hyperref}
|
| 11 |
\hypersetup{
|
| 12 |
-
pdftitle={
|
| 13 |
pdfauthor={DoVLA-CIL Working Draft},
|
| 14 |
-
pdfsubject={
|
| 15 |
}
|
| 16 |
|
| 17 |
-
\title{
|
|
|
|
| 18 |
\author{DoVLA-CIL Working Draft}
|
| 19 |
\date{\today}
|
| 20 |
|
|
@@ -25,6 +26,7 @@
|
|
| 25 |
\newcommand{\dovla}{\textsc{DoVLA}}
|
| 26 |
\newcommand{\pptc}{\ensuremath{\mathrm{PPTC}}}
|
| 27 |
\newcommand{\outcomeptr}{\ensuremath{\mathrm{OutcomePTR}}}
|
|
|
|
| 28 |
|
| 29 |
\newtheorem{definition}{Definition}
|
| 30 |
\newtheorem{theorem}{Theorem}
|
|
@@ -36,13 +38,14 @@
|
|
| 36 |
\begin{abstract}
|
| 37 |
Vision-language-action policies are usually trained from demonstrations that
|
| 38 |
show what the robot did, but not what would have happened under nearby actions.
|
| 39 |
-
We
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
\bench{} chart restores the identical state and
|
| 43 |
-
action chunks, and
|
| 44 |
-
collision, or success. The current six-task
|
| 45 |
-
matters: direct h=16 behavior cloning reaches
|
|
|
|
| 46 |
deployment-clean residual transport reaches 38.90\%, its top-8 proposal oracle
|
| 47 |
reaches 44.35\%, and the hidden same-state no-expert chart reaches 56.99\%.
|
| 48 |
This decomposes failure into proposal-support and selector gaps. The current
|
|
@@ -53,9 +56,22 @@ do-action tangent from a nearby source chart into the current target chart.
|
|
| 53 |
Measured validation and test rollouts confirm the support story but not yet a
|
| 54 |
final deployment method: with the visual-stat chart token, the held-out test
|
| 55 |
proposal oracle reaches 51.39\% success and \outcomeptr{}@8 reaches 53.47\%.
|
| 56 |
-
However, the score-only selected action remains below the measured base action
|
| 57 |
-
|
| 58 |
-
test success at 25.69\% coverage.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
\outcomeptr{} is reported only after generated candidates are rolled out, while
|
| 60 |
distance-only support diagnostics are \pptc{}. These runs are therefore
|
| 61 |
diagnostic evidence for the Atlas thesis and the next selector/generator gate,
|
|
@@ -171,7 +187,10 @@ outcome vector
|
|
| 171 |
\]
|
| 172 |
for terminal success, dense progress, contact quality, safety violation,
|
| 173 |
task-stage quality, smoothness, and recovery. A scalar utility $U(y_i)$ is used
|
| 174 |
-
for ranking and CAR
|
|
|
|
|
|
|
|
|
|
| 175 |
|
| 176 |
\subsection{Splits and Leakage Contract}
|
| 177 |
|
|
@@ -201,6 +220,27 @@ The current leakage audit over \texttt{data/cil\_charts/\{train,val,test\}}
|
|
| 201 |
passes with zero violations. The train index contains 2,044 charts and 32,704
|
| 202 |
rows; the complete split export contains 2,873 charts and 45,968 rows.
|
| 203 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
\section{Metrics}
|
| 205 |
|
| 206 |
For one chart, let $a_b$ be the base action and let $U(\cdot)$ be measured
|
|
@@ -210,6 +250,16 @@ utility when outcomes are available. Branch Causal Action Regret is
|
|
| 210 |
\qquad
|
| 211 |
a^*_{\mathcal A}=\arg\max_{a\in\mathcal A} U(a).
|
| 212 |
\]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 213 |
|
| 214 |
\paragraph{Measured proposal metrics.}
|
| 215 |
Generated candidates may be called successful only after they are rolled out or
|
|
@@ -231,6 +281,16 @@ Support Gap is
|
|
| 231 |
\]
|
| 232 |
These metrics are invalid for distance-only candidates.
|
| 233 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 234 |
\paragraph{Proxy geometry metrics.}
|
| 235 |
When generated candidates have not been rolled out, this draft reports support
|
| 236 |
geometry as \pptc{}:
|
|
@@ -285,20 +345,20 @@ a negative boundary penalty, and a cycle consistency term:
|
|
| 285 |
+ \lambda_-\max(0,m-d(\hat\xi_t,\Xi_t^-))
|
| 286 |
+ \lambda_c\|T_{\phi}(z_t,z_s,\hat\xi_t)-\xi_s^+\|_2^2.
|
| 287 |
\]
|
| 288 |
-
The first implementation uses exported base-action summaries
|
| 289 |
-
This is an engineering limitation, not
|
| 290 |
-
|
| 291 |
-
|
| 292 |
-
|
| 293 |
-
|
| 294 |
-
|
| 295 |
-
|
| 296 |
-
|
| 297 |
-
action. The current \ctt{} implementation retrieves nearby train positive source
|
| 298 |
-
tangents, transports them into the current chart, and decodes the public 21D
|
| 299 |
tangent summary as three residual keyframes with linear interpolation into an
|
| 300 |
-
action chunk
|
| 301 |
-
|
|
|
|
|
|
|
| 302 |
|
| 303 |
\input{../paper/sections/theory}
|
| 304 |
|
|
@@ -326,8 +386,8 @@ reranking, barycentric chart synthesis, CVAE generators, and flow generators are
|
|
| 326 |
diagnostic baselines. V1 does not beat V0. Negative-margin reranking does not
|
| 327 |
replace local positive support. Raw-action CVAE and spline flow variants can be
|
| 328 |
safe under NegativeNear, but collapse strict positive support. These failures
|
| 329 |
-
motivate \ctt{}: positive support
|
| 330 |
-
|
| 331 |
from ambient noise.
|
| 332 |
|
| 333 |
Previous distance-only memory and local-atlas diagnostics must be read as
|
|
@@ -345,7 +405,7 @@ success.
|
|
| 345 |
\caption{First residual \ctt{} proxy smoke on train self-target charts. This
|
| 346 |
table is an artifact check, not validation/test performance and not
|
| 347 |
\outcomeptr{}. The 0.20 NegativeNear value exceeds the current safety gate, so
|
| 348 |
-
the run
|
| 349 |
\label{tab:ctt-smoke}
|
| 350 |
\scriptsize
|
| 351 |
\input{../runs/ctt_residual_smoke_proxy/table}
|
|
@@ -420,6 +480,209 @@ candidates underperform the measured base action on that chart. This negative
|
|
| 420 |
smoke is useful because it protects the paper from reclassifying \pptc{} proxy
|
| 421 |
evidence as outcome success.
|
| 422 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 423 |
\begin{table}[t]
|
| 424 |
\centering
|
| 425 |
\caption{Measured residual \ctt{} rollout on 69 validation positive-support
|
|
@@ -461,6 +724,10 @@ points. The selector failure remains: selected success is 24.15\%, below the
|
|
| 461 |
27.54\% base success, and the success selector gap grows to 16.43 points. Thus
|
| 462 |
deterministic visual statistics help proposal support, but they do not solve
|
| 463 |
deployment-clean action selection.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 464 |
|
| 465 |
\begin{table}[t]
|
| 466 |
\centering
|
|
@@ -473,7 +740,7 @@ validation row.}
|
|
| 473 |
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obj_val_rollout_comparison/table}}
|
| 474 |
\end{table}
|
| 475 |
|
| 476 |
-
Table~\ref{tab:ctt-base-context-obj-val-rollout}
|
| 477 |
object-layout proxy diagnostic. Despite passing the proxy gate, the measured
|
| 478 |
proposal oracle is only 38.16\%, below the RGB-stat row's 40.58\%, while
|
| 479 |
\outcomeptr{}@8 remains 50.24\% and selected success drops to 20.29\%. This is
|
|
@@ -483,9 +750,9 @@ missing visual-language or object-centric causal chart token.
|
|
| 483 |
\begin{table}[t]
|
| 484 |
\centering
|
| 485 |
\caption{Measured residual \ctt{} rollout on 48 test positive-support charts
|
| 486 |
-
across three train seeds, K=8. The generated proposal oracle
|
| 487 |
-
|
| 488 |
-
|
| 489 |
\label{tab:ctt-test-rollout}
|
| 490 |
\scriptsize
|
| 491 |
\resizebox{\linewidth}{!}{\input{../runs/ctt_test_rollout_comparison/table}}
|
|
@@ -547,8 +814,8 @@ does not transfer as a reliable dominance certificate.
|
|
| 547 |
\caption{Learned dominance fallback trained on validation measured rows and
|
| 548 |
evaluated on held-out test rows. Features are deployment-visible candidate
|
| 549 |
features: utility-energy scores, score margins to base, rank, and tangent
|
| 550 |
-
norms. This improves over the base test success but remains
|
| 551 |
-
|
| 552 |
\label{tab:ctt-learned-dominance}
|
| 553 |
\scriptsize
|
| 554 |
\resizebox{\linewidth}{!}{\input{../runs/ctt_learned_dominance_val_to_test/table}}
|
|
@@ -559,7 +826,7 @@ can repair the selector without new rollouts. A ridge dominance model trained
|
|
| 559 |
only on validation measured rows reaches 30.56\% held-out test selected success
|
| 560 |
at 24.31\% coverage, improving over the 28.47\% base policy and the 22.22\%
|
| 561 |
score-only selector. This is a real selector improvement, but it is still far
|
| 562 |
-
from
|
| 563 |
evidence therefore narrows the bottleneck: transported proposals contain useful
|
| 564 |
actions, and lightweight dominance helps, but the final method needs a stronger
|
| 565 |
train-only utility-energy model and richer visual/object-centric chart tokens.
|
|
@@ -576,7 +843,8 @@ checkpoints are negative across three seeds: when
|
|
| 576 |
|
| 577 |
\begin{table}[t]
|
| 578 |
\centering
|
| 579 |
-
\caption{Best validation-calibrated dominance diagnostic
|
|
|
|
| 580 |
context dominance over the measured \texttt{base\_context\_obs} visual-stat
|
| 581 |
rollout rows. The calibrator is fit on validation measured rows only and
|
| 582 |
evaluated on held-out test rows.}
|
|
@@ -594,6 +862,198 @@ dominance row. The gain is real but not sufficient: proposal-oracle success is
|
|
| 594 |
remains 24.31 points. The remaining problem is still reliable deployment-clean
|
| 595 |
dominance, not merely generating more nearby tangents.
|
| 596 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 597 |
The feature-source audits turn this negative result into a concrete engineering
|
| 598 |
target. In the original \texttt{data/cil\_charts} indexes, scene ids and
|
| 599 |
instructions are present, but observation embeddings and raw observation
|
|
@@ -626,9 +1086,8 @@ train-calibrated learned selector reaches 31.25\% selected success at 25.69\%
|
|
| 626 |
coverage, compared with 29.17\% measured base success and 51.39\% proposal
|
| 627 |
oracle success. This is a small clean improvement over base, but it is lower
|
| 628 |
than the validation-calibrated context diagnostic and leaves a 25.69-point
|
| 629 |
-
success selector gap.
|
| 630 |
-
|
| 631 |
-
method.
|
| 632 |
|
| 633 |
\begin{table}[t]
|
| 634 |
\centering
|
|
@@ -679,20 +1138,34 @@ The current draft is backed by explicit artifacts:
|
|
| 679 |
\texttt{scripts/slurm/eval\_ctt\_generated\_rollout.sbatch} implement the
|
| 680 |
measured generated-candidate rollout path, including a self-source exclusion
|
| 681 |
flag for train-split calibration and metadata loading for deployment-visible
|
| 682 |
-
chart features such as \texttt{base\_context\_obs}.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 683 |
\item \texttt{scripts/build\_ctt\_rollout\_comparison.py} aggregates
|
| 684 |
measured validation/test rollouts and reports selected success, proposal
|
| 685 |
-
oracle success, hidden chart oracle success, success support gap,
|
| 686 |
-
selector gap.
|
| 687 |
\item \texttt{scripts/eval\_dominance\_selector.py} calibrates a dominance
|
| 688 |
-
fallback rule on
|
| 689 |
measured rows; it can use rollout row scores or recompute scores from a
|
| 690 |
-
train-only utility-energy checkpoint
|
|
|
|
|
|
|
| 691 |
\item \texttt{scripts/eval\_learned\_dominance\_selector.py} trains a small
|
| 692 |
validation- or train-calibrated dominance model over deployment-visible
|
| 693 |
candidate features and evaluates it on held-out measured test rows; it also
|
| 694 |
-
logs feature/target ablations,
|
| 695 |
-
|
|
|
|
| 696 |
\item \texttt{scripts/eval\_nonlinear\_dominance\_selector.py} runs the
|
| 697 |
train-calibrated nonlinear selector diagnostics in
|
| 698 |
\texttt{runs/ctt\_base\_context\_obs\_nonlinear\_dominance\_*}.
|
|
@@ -700,22 +1173,47 @@ The current draft is backed by explicit artifacts:
|
|
| 700 |
\texttt{scripts/build\_ctt\_proxy\_comparison.py} generate the local-atlas
|
| 701 |
baseline and validation proxy gate table.
|
| 702 |
\item \texttt{scripts/check\_tangent\_reconstruction.py} verifies that the
|
| 703 |
-
exported 21D tangent codes are deterministic summaries of
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 704 |
\item \texttt{scripts/train\_utility\_energy.py} and
|
| 705 |
\texttt{scripts/calibrate\_dominance.py} implement the utility/scoring and
|
| 706 |
dominance-calibration path.
|
| 707 |
\item \texttt{scripts/summarize\_ctt\_runs.py} generates
|
| 708 |
-
\texttt{runs/summary\_ctt.csv}
|
| 709 |
-
|
| 710 |
-
\texttt{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 711 |
\end{itemize}
|
| 712 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 713 |
\section{Limitations and Next Steps}
|
| 714 |
|
| 715 |
This draft is not yet a final deployment method. The current \ctt{} evidence
|
| 716 |
includes validation and test measured generated-candidate rollouts, but the
|
| 717 |
-
selected action
|
| 718 |
-
passes 50\%. The missing method component is calibrated dominance and a better
|
| 719 |
utility selector, not another proxy-only support plot. The first standalone
|
| 720 |
train-only utility-energy checkpoints and context-metadata ridge variants do
|
| 721 |
not solve this transfer problem, so the chart token must move beyond base-action
|
|
@@ -723,17 +1221,28 @@ summaries and coarse task metadata toward exported visual-language and
|
|
| 723 |
object-centric geometry. The RGB-reference export now provides a leakage-audited
|
| 724 |
visual-stat token and improves measured support plus validation-calibrated
|
| 725 |
selected success, but this does not qualify as the needed learned object-centric
|
| 726 |
-
representation because the best
|
| 727 |
-
the selector gap remains
|
| 728 |
-
|
| 729 |
-
|
| 730 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 731 |
|
| 732 |
The next experimental step is concrete: replace the current weak train-only
|
| 733 |
utility-energy selector with visual-language/object-centric chart features,
|
| 734 |
-
rerun held-out measured dominance selection, and
|
| 735 |
-
fallback
|
| 736 |
-
replace rollout measurement.
|
| 737 |
|
| 738 |
\section{Conclusion}
|
| 739 |
|
|
|
|
| 9 |
\usepackage{xcolor}
|
| 10 |
\usepackage{hyperref}
|
| 11 |
\hypersetup{
|
| 12 |
+
pdftitle={Causal Tangent Transport: Learning Positive Do-Action Geometry from Same-State Counterfactual Charts},
|
| 13 |
pdfauthor={DoVLA-CIL Working Draft},
|
| 14 |
+
pdfsubject={Causal Tangent Transport with same-state counterfactual chart evidence},
|
| 15 |
}
|
| 16 |
|
| 17 |
+
\title{Causal Tangent Transport:\\
|
| 18 |
+
Learning Positive Do-Action Geometry from Same-State Counterfactual Charts}
|
| 19 |
\author{DoVLA-CIL Working Draft}
|
| 20 |
\date{\today}
|
| 21 |
|
|
|
|
| 26 |
\newcommand{\dovla}{\textsc{DoVLA}}
|
| 27 |
\newcommand{\pptc}{\ensuremath{\mathrm{PPTC}}}
|
| 28 |
\newcommand{\outcomeptr}{\ensuremath{\mathrm{OutcomePTR}}}
|
| 29 |
+
\newcommand{\ncar}{\ensuremath{\mathrm{NCAR}}}
|
| 30 |
|
| 31 |
\newtheorem{definition}{Definition}
|
| 32 |
\newtheorem{theorem}{Theorem}
|
|
|
|
| 38 |
\begin{abstract}
|
| 39 |
Vision-language-action policies are usually trained from demonstrations that
|
| 40 |
show what the robot did, but not what would have happened under nearby actions.
|
| 41 |
+
We study Causal Tangent Transport (\ctt{}), a deployment-clean generator that
|
| 42 |
+
starts from measured train positive do-action tangents and transports them into
|
| 43 |
+
a target same-state chart. The supporting \atlas{} framework measures the
|
| 44 |
+
local causal geometry: a \bench{} chart restores the identical state and
|
| 45 |
+
instruction, executes multiple action chunks, and records which tangents cause
|
| 46 |
+
recovery, progress, failure, collision, or success. The current six-task
|
| 47 |
+
diagnostic shows why this object matters: direct h=16 behavior cloning reaches
|
| 48 |
+
29.74\% success,
|
| 49 |
deployment-clean residual transport reaches 38.90\%, its top-8 proposal oracle
|
| 50 |
reaches 44.35\%, and the hidden same-state no-expert chart reaches 56.99\%.
|
| 51 |
This decomposes failure into proposal-support and selector gaps. The current
|
|
|
|
| 56 |
Measured validation and test rollouts confirm the support story but not yet a
|
| 57 |
final deployment method: with the visual-stat chart token, the held-out test
|
| 58 |
proposal oracle reaches 51.39\% success and \outcomeptr{}@8 reaches 53.47\%.
|
| 59 |
+
However, the score-only selected action remains below the measured base action.
|
| 60 |
+
Under the clipped convention, the best train-calibrated learned dominance
|
| 61 |
+
selector reaches 31.25\% test success at 25.69\% coverage. A bounded
|
| 62 |
+
\texttt{tanh} execution diagnostic removes action-bound violations, and a
|
| 63 |
+
train-calibrated context selector reaches 38.19\% selected test success, matching
|
| 64 |
+
the tanh proposal oracle in micro success. The row-level selector-gap metric is
|
| 65 |
+
still 12.50 points, and the proposal support itself is lower than the clipped
|
| 66 |
+
visual-stat row. An explicit \texttt{env\_clip} diagnostic is action-bound-clean
|
| 67 |
+
and preserves more proposal support. Increasing transported candidates from
|
| 68 |
+
K=8 to K=16 raises the held-out test proposal oracle to 56.94\% and
|
| 69 |
+
\outcomeptr{}@16 to 54.86\%, with zero action-bound unsafe rate, crossing the
|
| 70 |
+
current support gate. The score-only selected action still reaches only
|
| 71 |
+
27.78\%, below the 29.17\% base action. The best train-calibrated K=16
|
| 72 |
+
selector reaches 35.42\% selected test success using train-only target--source
|
| 73 |
+
RGB-stat chart compatibility, while still leaving a 24.31-point selector gap.
|
| 74 |
+
We keep the metric boundary explicit:
|
| 75 |
\outcomeptr{} is reported only after generated candidates are rolled out, while
|
| 76 |
distance-only support diagnostics are \pptc{}. These runs are therefore
|
| 77 |
diagnostic evidence for the Atlas thesis and the next selector/generator gate,
|
|
|
|
| 187 |
\]
|
| 188 |
for terminal success, dense progress, contact quality, safety violation,
|
| 189 |
task-stage quality, smoothness, and recovery. A scalar utility $U(y_i)$ is used
|
| 190 |
+
for ranking and CAR; reports keep the components visible.
|
| 191 |
+
Metric scripts also report safety-label coverage. Missing or null
|
| 192 |
+
\texttt{safety\_violation} fields are treated as unknown rather than safe, so an
|
| 193 |
+
unsafe-rate number is reported only when safety labels are actually present.
|
| 194 |
|
| 195 |
\subsection{Splits and Leakage Contract}
|
| 196 |
|
|
|
|
| 220 |
passes with zero violations. The train index contains 2,044 charts and 32,704
|
| 221 |
rows; the complete split export contains 2,873 charts and 45,968 rows.
|
| 222 |
|
| 223 |
+
\subsection{Benchmark Tracks and Branch Families}
|
| 224 |
+
|
| 225 |
+
\bench{} is intended as a benchmark object, not only a private training set.
|
| 226 |
+
It exposes four evaluator modes that all share the same chart primitive:
|
| 227 |
+
Ranking Track gives participants an observation, instruction, and candidate
|
| 228 |
+
actions and evaluates their ordering against hidden outcomes; Generation Track
|
| 229 |
+
accepts generated action chunks and executes them from restored states;
|
| 230 |
+
Deployment Track evaluates a closed-loop policy from initial task states; and
|
| 231 |
+
Recovery Track starts from near-miss states and evaluates whether one proposed
|
| 232 |
+
chunk causally recovers progress. Only the evaluator may read validation/test
|
| 233 |
+
outcomes in all four modes.
|
| 234 |
+
|
| 235 |
+
The current six-task export already records branch families such as base,
|
| 236 |
+
expert, residual transport, random negative, wrong-direction, wrong-gripper,
|
| 237 |
+
near-miss, and no-op branches. The intended \bench{}-Core schema generalizes
|
| 238 |
+
these into base/stochastic-anchor branches, train-only expert branches,
|
| 239 |
+
residual tangent branches, object-centric geometric branches, recovery
|
| 240 |
+
tangents, negative anti-goal branches, and learned-generator branches. The
|
| 241 |
+
method does not depend on these family labels at deployment; they define
|
| 242 |
+
interventional coverage and ablation axes for learning local causal geometry.
|
| 243 |
+
|
| 244 |
\section{Metrics}
|
| 245 |
|
| 246 |
For one chart, let $a_b$ be the base action and let $U(\cdot)$ be measured
|
|
|
|
| 250 |
\qquad
|
| 251 |
a^*_{\mathcal A}=\arg\max_{a\in\mathcal A} U(a).
|
| 252 |
\]
|
| 253 |
+
The normalized form reports the fraction of the base-to-oracle gap that remains
|
| 254 |
+
unclosed:
|
| 255 |
+
\[
|
| 256 |
+
\ncar =
|
| 257 |
+
\frac{U(a^*)-U(a_m)}
|
| 258 |
+
{|U(a^*)-U(a_b)|+\epsilon}.
|
| 259 |
+
\]
|
| 260 |
+
Evaluator artifacts report \ncar{} only on rows where the base-to-oracle gap is
|
| 261 |
+
numerically meaningful; otherwise a tiny denominator can turn an uninformative
|
| 262 |
+
chart into a misleadingly large normalized regret.
|
| 263 |
|
| 264 |
\paragraph{Measured proposal metrics.}
|
| 265 |
Generated candidates may be called successful only after they are rolled out or
|
|
|
|
| 281 |
\]
|
| 282 |
These metrics are invalid for distance-only candidates.
|
| 283 |
|
| 284 |
+
\paragraph{Safety coverage.}
|
| 285 |
+
Measured rollout rows may include \texttt{safety\_violation} inside
|
| 286 |
+
\texttt{base\_outcome} and \texttt{candidate\_outcomes}. The evaluator reports
|
| 287 |
+
generated safety-label coverage, selected/base label-known rates, and unsafe
|
| 288 |
+
rates over known labels only. The current implemented safety source is
|
| 289 |
+
action-space validity: if a decoded chunk lies outside the ManiSkill action
|
| 290 |
+
space before optional clipping, the row records
|
| 291 |
+
\texttt{safety\_violation\_source=action\_bounds}. This is a deployment
|
| 292 |
+
validity audit, not a collision/contact audit.
|
| 293 |
+
|
| 294 |
\paragraph{Proxy geometry metrics.}
|
| 295 |
When generated candidates have not been rolled out, this draft reports support
|
| 296 |
geometry as \pptc{}:
|
|
|
|
| 345 |
+ \lambda_-\max(0,m-d(\hat\xi_t,\Xi_t^-))
|
| 346 |
+ \lambda_c\|T_{\phi}(z_t,z_s,\hat\xi_t)-\xi_s^+\|_2^2.
|
| 347 |
\]
|
| 348 |
+
The first implementation uses exported base-action summaries plus deterministic
|
| 349 |
+
RGB/object statistics as chart features. This is an engineering limitation, not
|
| 350 |
+
the full Atlas representation: the missing representation components are
|
| 351 |
+
visual-language tokens, target/distractor object tokens, robot/contact-region
|
| 352 |
+
tokens, and object-centric tangent frames.
|
| 353 |
+
|
| 354 |
+
\paragraph{Deployment algorithm.}
|
| 355 |
+
At test time, the implemented \ctt{} pipeline retrieves nearby train positive
|
| 356 |
+
source tangents, transports them into the current chart, decodes the public 21D
|
|
|
|
|
|
|
| 357 |
tangent summary as three residual keyframes with linear interpolation into an
|
| 358 |
+
action chunk, scores the candidates with the current utility field, and applies
|
| 359 |
+
the calibrated dominance fallback when that rule is enabled. The decoder is an
|
| 360 |
+
auditable engineering decoder, not a lossless reconstruction of the hidden
|
| 361 |
+
branch action.
|
| 362 |
|
| 363 |
\input{../paper/sections/theory}
|
| 364 |
|
|
|
|
| 386 |
diagnostic baselines. V1 does not beat V0. Negative-margin reranking does not
|
| 387 |
replace local positive support. Raw-action CVAE and spline flow variants can be
|
| 388 |
safe under NegativeNear, but collapse strict positive support. These failures
|
| 389 |
+
motivate \ctt{}: positive support is transported from measured positive chart
|
| 390 |
+
tangents, with negative tangents defining boundaries, rather than sampled
|
| 391 |
from ambient noise.
|
| 392 |
|
| 393 |
Previous distance-only memory and local-atlas diagnostics must be read as
|
|
|
|
| 405 |
\caption{First residual \ctt{} proxy smoke on train self-target charts. This
|
| 406 |
table is an artifact check, not validation/test performance and not
|
| 407 |
\outcomeptr{}. The 0.20 NegativeNear value exceeds the current safety gate, so
|
| 408 |
+
the run is not claimed as method success.}
|
| 409 |
\label{tab:ctt-smoke}
|
| 410 |
\scriptsize
|
| 411 |
\input{../runs/ctt_residual_smoke_proxy/table}
|
|
|
|
| 480 |
smoke is useful because it protects the paper from reclassifying \pptc{} proxy
|
| 481 |
evidence as outcome success.
|
| 482 |
|
| 483 |
+
\begin{table}[t]
|
| 484 |
+
\centering
|
| 485 |
+
\caption{Action-bound audit for the leakage-audited RGB-reference chart DB.
|
| 486 |
+
The bound is the ManiSkill \texttt{pd\_ee\_delta\_pose} action space,
|
| 487 |
+
$[-1,1]^7$. This is an action-representation validity audit, not a
|
| 488 |
+
collision/contact safety audit.}
|
| 489 |
+
\label{tab:action-bound-audit}
|
| 490 |
+
\scriptsize
|
| 491 |
+
\input{../runs/action_bound_audit_rgb_refs/table}
|
| 492 |
+
\end{table}
|
| 493 |
+
|
| 494 |
+
Table~\ref{tab:action-bound-audit} exposes a previously hidden rollout
|
| 495 |
+
confound: every base branch in the exported RGB-reference chart DB exceeds the
|
| 496 |
+
nominal ManiSkill action bounds, and about 30.8\% of branch actions exceed the
|
| 497 |
+
same bounds. The per-dimension audit shows the largest violations in the
|
| 498 |
+
rotation/gripper-like dimensions, with a global max-fit scale of about 0.215
|
| 499 |
+
for train/validation/test. The measured rollout evaluator now logs these
|
| 500 |
+
pre-clipping violations and can run with \texttt{--disable-action-clipping} to
|
| 501 |
+
compare faithful raw-action replay against clipped deployment execution. It also
|
| 502 |
+
supports an explicit \texttt{--execution-action-scale} diagnostic so scaled
|
| 503 |
+
raw-action replay is a separate, logged convention rather than an implicit fix.
|
| 504 |
+
A one-chart \texttt{base\_context\_obs} smoke at scale 0.215 confirms 100\%
|
| 505 |
+
action-bound label coverage and zero action-bound violations for base and
|
| 506 |
+
generated candidates. Full scaled validation/test refresh jobs completed and
|
| 507 |
+
are aggregated in
|
| 508 |
+
Tables~\ref{tab:ctt-base-context-obs-val-scaled-rollout}
|
| 509 |
+
and~\ref{tab:ctt-base-context-obs-test-scaled-rollout}. The global scale
|
| 510 |
+
diagnostic rejects a naive ``scale everything'' fix: it removes base action-bound
|
| 511 |
+
violations, but support and selected success drop sharply and about 20\% of
|
| 512 |
+
generated candidates still violate action bounds. The rollout evaluator now also
|
| 513 |
+
supports a bounded \texttt{--execution-action-transform tanh} decoder
|
| 514 |
+
diagnostic, which maps decoded controls smoothly into finite action bounds
|
| 515 |
+
before validity checks. Full tanh validation/test refresh jobs completed and
|
| 516 |
+
are aggregated in
|
| 517 |
+
Tables~\ref{tab:ctt-base-context-obs-val-tanh-rollout}
|
| 518 |
+
and~\ref{tab:ctt-base-context-obs-test-tanh-rollout}; they have zero
|
| 519 |
+
base/generated/selected action-bound violations under known labels, but lower
|
| 520 |
+
proposal support than the clipped/no-clipping visual-stat rows. The full
|
| 521 |
+
no-clipping
|
| 522 |
+
validation/test refresh jobs completed and are aggregated in
|
| 523 |
+
Tables~\ref{tab:ctt-base-context-obs-val-noclip-rollout}
|
| 524 |
+
and~\ref{tab:ctt-base-context-obs-test-noclip-rollout}; the clipped rollout
|
| 525 |
+
tables below therefore remain support/selector diagnostics under the current
|
| 526 |
+
deployment clipping policy, not final safety evidence.
|
| 527 |
+
|
| 528 |
+
\begin{table}[t]
|
| 529 |
+
\centering
|
| 530 |
+
\caption{No-clipping validation refresh for \texttt{base\_context\_obs}, K=8.
|
| 531 |
+
This replays decoded raw actions from restored states and records action-bound
|
| 532 |
+
validity labels before any clipping. The labels are action-space violations, not
|
| 533 |
+
collision/contact outcomes.}
|
| 534 |
+
\label{tab:ctt-base-context-obs-val-noclip-rollout}
|
| 535 |
+
\scriptsize
|
| 536 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_val_noclip_rollout_comparison/table}}
|
| 537 |
+
\end{table}
|
| 538 |
+
|
| 539 |
+
\begin{table}[t]
|
| 540 |
+
\centering
|
| 541 |
+
\caption{No-clipping held-out test refresh for \texttt{base\_context\_obs},
|
| 542 |
+
K=8. The proposal oracle remains nonzero, but selected success does not improve
|
| 543 |
+
over the raw-replay base and almost all known labels indicate action-space
|
| 544 |
+
bound violations.}
|
| 545 |
+
\label{tab:ctt-base-context-obs-test-noclip-rollout}
|
| 546 |
+
\scriptsize
|
| 547 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_test_noclip_rollout_comparison/table}}
|
| 548 |
+
\end{table}
|
| 549 |
+
|
| 550 |
+
The no-clipping refresh changes the measured action execution distribution but
|
| 551 |
+
does not rescue the deployment claim. On validation, \outcomeptr{}@8 is 45.41\%,
|
| 552 |
+
proposal-oracle success is 41.06\%, selected success is 22.71\%, and base
|
| 553 |
+
success is 28.99\%. On held-out test, \outcomeptr{}@8 is 54.17\% and
|
| 554 |
+
proposal-oracle success is 50.00\%, but selected success equals the raw-replay
|
| 555 |
+
base at 25.00\%. Known action-bound violation rates are 97.58\%--100.00\% for
|
| 556 |
+
selected/base rows and 98.13\%--98.44\% for generated rows. Thus raw-action
|
| 557 |
+
replay confirms proposal support remains measurable while action scaling and
|
| 558 |
+
selection remain blockers.
|
| 559 |
+
|
| 560 |
+
\begin{table}[t]
|
| 561 |
+
\centering
|
| 562 |
+
\caption{Scaled raw-action validation refresh for
|
| 563 |
+
\texttt{base\_context\_obs}, K=8, with \texttt{--execution-action-scale 0.215}
|
| 564 |
+
and clipping disabled. This is a diagnostic convention suggested by the
|
| 565 |
+
action-bound audit, not a final action representation.}
|
| 566 |
+
\label{tab:ctt-base-context-obs-val-scaled-rollout}
|
| 567 |
+
\scriptsize
|
| 568 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_val_scaled0215_rollout_comparison/table}}
|
| 569 |
+
\end{table}
|
| 570 |
+
|
| 571 |
+
\begin{table}[t]
|
| 572 |
+
\centering
|
| 573 |
+
\caption{Scaled raw-action held-out test refresh for
|
| 574 |
+
\texttt{base\_context\_obs}, K=8, with \texttt{--execution-action-scale 0.215}
|
| 575 |
+
and clipping disabled. The global scale improves base action-bound validity but
|
| 576 |
+
does not preserve proposal support.}
|
| 577 |
+
\label{tab:ctt-base-context-obs-test-scaled-rollout}
|
| 578 |
+
\scriptsize
|
| 579 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_test_scaled0215_rollout_comparison/table}}
|
| 580 |
+
\end{table}
|
| 581 |
+
|
| 582 |
+
The scaled refresh is a negative diagnostic. On validation, base action-bound
|
| 583 |
+
violations fall to zero, but \outcomeptr{}@8 drops to 27.54\%,
|
| 584 |
+
proposal-oracle success to 12.08\%, selected success to 6.76\%, and generated
|
| 585 |
+
action-bound violations remain 19.20\% over known labels. On held-out test,
|
| 586 |
+
\outcomeptr{}@8 is 37.50\%, proposal-oracle success is 16.67\%, selected
|
| 587 |
+
success is 9.03\%, base success is 22.92\%, and generated action-bound
|
| 588 |
+
violations are 19.97\%. Therefore the next fix must be an action-representation
|
| 589 |
+
or decoder correction, not a global post-hoc scale.
|
| 590 |
+
|
| 591 |
+
\begin{table}[t]
|
| 592 |
+
\centering
|
| 593 |
+
\caption{Per-dimension train-max scaled validation refresh for
|
| 594 |
+
\texttt{base\_context\_obs}, K=8, with clipping disabled. The scale vector is
|
| 595 |
+
fit only from the train split action-bound audit.}
|
| 596 |
+
\label{tab:ctt-base-context-obs-val-perdim-rollout}
|
| 597 |
+
\scriptsize
|
| 598 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_val_perdim_trainmax_rollout_comparison/table}}
|
| 599 |
+
\end{table}
|
| 600 |
+
|
| 601 |
+
\begin{table}[t]
|
| 602 |
+
\centering
|
| 603 |
+
\caption{Per-dimension train-max scaled held-out test refresh for
|
| 604 |
+
\texttt{base\_context\_obs}, K=8, with clipping disabled. This diagnostic tests
|
| 605 |
+
whether per-dimension max-fit scaling preserves more support than the global
|
| 606 |
+
scale.}
|
| 607 |
+
\label{tab:ctt-base-context-obs-test-perdim-rollout}
|
| 608 |
+
\scriptsize
|
| 609 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_test_perdim_trainmax_rollout_comparison/table}}
|
| 610 |
+
\end{table}
|
| 611 |
+
|
| 612 |
+
The per-dimension train-max scale is also negative. It is fit from train
|
| 613 |
+
base-branch action bounds only and keeps the measured base action bound-valid,
|
| 614 |
+
but generated candidates still violate action bounds on 27.42\% of validation
|
| 615 |
+
rows and 30.30\% of held-out test rows. Support collapses further than under
|
| 616 |
+
\texttt{tanh}: validation proposal-oracle success is 16.91\%, selected success
|
| 617 |
+
is 9.18\%, and \outcomeptr{}@8 is 30.43\%; held-out test proposal-oracle
|
| 618 |
+
success is 22.22\%, selected success is 11.81\%, and \outcomeptr{}@8 is
|
| 619 |
+
36.11\%. This rejects another simple post-processing convention and leaves the
|
| 620 |
+
decoder/action representation itself as the blocker.
|
| 621 |
+
|
| 622 |
+
\begin{table}[t]
|
| 623 |
+
\centering
|
| 624 |
+
\caption{Explicit \texttt{env\_clip} validation refresh for
|
| 625 |
+
\texttt{base\_context\_obs}, K=8, with clipping disabled. The declared decoder
|
| 626 |
+
convention clips decoded controls to action-space bounds before validity checks,
|
| 627 |
+
so action-bound labels measure the declared convention rather than silent
|
| 628 |
+
simulator clipping.}
|
| 629 |
+
\label{tab:ctt-base-context-obs-val-envclip-rollout}
|
| 630 |
+
\scriptsize
|
| 631 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_val_envclip_rollout_comparison/table}}
|
| 632 |
+
\end{table}
|
| 633 |
+
|
| 634 |
+
\begin{table}[t]
|
| 635 |
+
\centering
|
| 636 |
+
\caption{Explicit \texttt{env\_clip} held-out test refresh for
|
| 637 |
+
\texttt{base\_context\_obs}, K=8, with clipping disabled. This convention is
|
| 638 |
+
action-bound-clean and preserves more proposal support than bounded
|
| 639 |
+
\texttt{tanh}, but score-only selection remains below base.}
|
| 640 |
+
\label{tab:ctt-base-context-obs-test-envclip-rollout}
|
| 641 |
+
\scriptsize
|
| 642 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_test_envclip_rollout_comparison/table}}
|
| 643 |
+
\end{table}
|
| 644 |
+
|
| 645 |
+
The explicit \texttt{env\_clip} refresh is the strongest bounded-action support
|
| 646 |
+
diagnostic so far. Generated, selected, and base action-bound violation rates
|
| 647 |
+
are all 0.00\% with 100\% known labels. On validation, \outcomeptr{}@8 is
|
| 648 |
+
50.24\%, proposal-oracle success is 41.55\%, selected success is 22.22\%, and
|
| 649 |
+
base success is 27.54\%. On held-out test, \outcomeptr{}@8 is 50.69\%,
|
| 650 |
+
proposal-oracle success is 47.92\%, selected success is 23.61\%, and base
|
| 651 |
+
success is 29.17\%. Thus \texttt{env\_clip} largely removes the action-bound
|
| 652 |
+
validity confound without the severe support collapse of per-dimension scaling
|
| 653 |
+
or bounded \texttt{tanh}, but it exposes the remaining selector/dominance
|
| 654 |
+
failure.
|
| 655 |
+
|
| 656 |
+
\begin{table}[t]
|
| 657 |
+
\centering
|
| 658 |
+
\caption{Bounded \texttt{tanh} validation refresh for
|
| 659 |
+
\texttt{base\_context\_obs}, K=8, with clipping disabled. This action convention
|
| 660 |
+
maps decoded controls into finite action bounds before validity checks.}
|
| 661 |
+
\label{tab:ctt-base-context-obs-val-tanh-rollout}
|
| 662 |
+
\scriptsize
|
| 663 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_val_tanh_rollout_comparison/table}}
|
| 664 |
+
\end{table}
|
| 665 |
+
|
| 666 |
+
\begin{table}[t]
|
| 667 |
+
\centering
|
| 668 |
+
\caption{Bounded \texttt{tanh} held-out test refresh for
|
| 669 |
+
\texttt{base\_context\_obs}, K=8, with clipping disabled. The convention is
|
| 670 |
+
action-bound-clean but selected success remains below the tanh base action.}
|
| 671 |
+
\label{tab:ctt-base-context-obs-test-tanh-rollout}
|
| 672 |
+
\scriptsize
|
| 673 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_test_tanh_rollout_comparison/table}}
|
| 674 |
+
\end{table}
|
| 675 |
+
|
| 676 |
+
The bounded tanh refresh removes the action-bound validity confound in this
|
| 677 |
+
diagnostic convention: generated, selected, and base action-bound violation
|
| 678 |
+
rates are all 0.00\% with 100\% known labels on validation and held-out test.
|
| 679 |
+
It does not by itself solve \ctt{} deployment. On validation,
|
| 680 |
+
\outcomeptr{}@8 is 42.51\%, proposal-oracle success is 36.71\%, selected
|
| 681 |
+
success is 21.26\%, and base success is 27.54\%. On held-out test,
|
| 682 |
+
\outcomeptr{}@8 is 41.67\%, proposal-oracle success is 38.19\%, selected
|
| 683 |
+
success is 25.00\%, and base success is 33.33\%. This makes tanh useful as an
|
| 684 |
+
action-bound-clean diagnostic, but not as a support-preserving decoder fix.
|
| 685 |
+
|
| 686 |
\begin{table}[t]
|
| 687 |
\centering
|
| 688 |
\caption{Measured residual \ctt{} rollout on 69 validation positive-support
|
|
|
|
| 724 |
27.54\% base success, and the success selector gap grows to 16.43 points. Thus
|
| 725 |
deterministic visual statistics help proposal support, but they do not solve
|
| 726 |
deployment-clean action selection.
|
| 727 |
+
The regenerated measured-rollout tables expose safety-label fields only for
|
| 728 |
+
artifacts generated after the action-bound audit landed. Older aggregate tables
|
| 729 |
+
therefore remain valid for success/support comparisons, but not for
|
| 730 |
+
unsafe-contact measurement.
|
| 731 |
|
| 732 |
\begin{table}[t]
|
| 733 |
\centering
|
|
|
|
| 740 |
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obj_val_rollout_comparison/table}}
|
| 741 |
\end{table}
|
| 742 |
|
| 743 |
+
Table~\ref{tab:ctt-base-context-obj-val-rollout} completes the
|
| 744 |
object-layout proxy diagnostic. Despite passing the proxy gate, the measured
|
| 745 |
proposal oracle is only 38.16\%, below the RGB-stat row's 40.58\%, while
|
| 746 |
\outcomeptr{}@8 remains 50.24\% and selected success drops to 20.29\%. This is
|
|
|
|
| 750 |
\begin{table}[t]
|
| 751 |
\centering
|
| 752 |
\caption{Measured residual \ctt{} rollout on 48 test positive-support charts
|
| 753 |
+
across three train seeds, K=8. The generated proposal oracle exceeds 50\%, but
|
| 754 |
+
the selected action fails because the current score/dominance rule chooses poor
|
| 755 |
+
candidates.}
|
| 756 |
\label{tab:ctt-test-rollout}
|
| 757 |
\scriptsize
|
| 758 |
\resizebox{\linewidth}{!}{\input{../runs/ctt_test_rollout_comparison/table}}
|
|
|
|
| 814 |
\caption{Learned dominance fallback trained on validation measured rows and
|
| 815 |
evaluated on held-out test rows. Features are deployment-visible candidate
|
| 816 |
features: utility-energy scores, score margins to base, rank, and tangent
|
| 817 |
+
norms. This improves over the base test success but remains a partial selector
|
| 818 |
+
diagnostic.}
|
| 819 |
\label{tab:ctt-learned-dominance}
|
| 820 |
\scriptsize
|
| 821 |
\resizebox{\linewidth}{!}{\input{../runs/ctt_learned_dominance_val_to_test/table}}
|
|
|
|
| 826 |
only on validation measured rows reaches 30.56\% held-out test selected success
|
| 827 |
at 24.31\% coverage, improving over the 28.47\% base policy and the 22.22\%
|
| 828 |
score-only selector. This is a real selector improvement, but it is still far
|
| 829 |
+
from a method-success claim and leaves a 25.69-point success selector gap. The
|
| 830 |
evidence therefore narrows the bottleneck: transported proposals contain useful
|
| 831 |
actions, and lightweight dominance helps, but the final method needs a stronger
|
| 832 |
train-only utility-energy model and richer visual/object-centric chart tokens.
|
|
|
|
| 843 |
|
| 844 |
\begin{table}[t]
|
| 845 |
\centering
|
| 846 |
+
\caption{Best clipped-convention validation-calibrated dominance diagnostic:
|
| 847 |
+
learned
|
| 848 |
context dominance over the measured \texttt{base\_context\_obs} visual-stat
|
| 849 |
rollout rows. The calibrator is fit on validation measured rows only and
|
| 850 |
evaluated on held-out test rows.}
|
|
|
|
| 862 |
remains 24.31 points. The remaining problem is still reliable deployment-clean
|
| 863 |
dominance, not merely generating more nearby tangents.
|
| 864 |
|
| 865 |
+
\begin{table}[t]
|
| 866 |
+
\centering
|
| 867 |
+
\caption{Bounded-tanh selector diagnostic over already measured candidates.
|
| 868 |
+
The learned context+tangent selector is fit on validation tanh rows and
|
| 869 |
+
evaluated once on held-out test tanh rows. It is action-bound-clean, but not a
|
| 870 |
+
train-clean deployment selector.}
|
| 871 |
+
\label{tab:ctt-base-context-obs-tanh-learned-dominance}
|
| 872 |
+
\scriptsize
|
| 873 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_learned_dominance_context_tangent_success_tanh_val_to_test/table}}
|
| 874 |
+
\end{table}
|
| 875 |
+
|
| 876 |
+
Table~\ref{tab:ctt-base-context-obs-tanh-learned-dominance} asks whether the
|
| 877 |
+
bounded tanh action convention can be paired with a better selector. The
|
| 878 |
+
context+tangent success calibrator reaches 35.42\% selected held-out test
|
| 879 |
+
success at 10.42\% coverage, above the tanh base action at 33.33\% and the
|
| 880 |
+
tanh score-only selector at 25.00\%. The result is useful but narrow: proposal
|
| 881 |
+
oracle success is only 38.19\%, hidden chart oracle success is 72.92\%, and the
|
| 882 |
+
success support gap is 36.81 points. It is therefore a bounded-action selector
|
| 883 |
+
diagnostic, not the final Atlas method.
|
| 884 |
+
|
| 885 |
+
\begin{table}[t]
|
| 886 |
+
\centering
|
| 887 |
+
\caption{Train-calibrated bounded-tanh selector diagnostic. Calibration uses
|
| 888 |
+
train-split tanh measured rollout rows with same-chart and same-state source
|
| 889 |
+
retrieval excluded, then evaluates once on held-out test tanh rows.}
|
| 890 |
+
\label{tab:ctt-base-context-obs-tanh-learned-train-dominance}
|
| 891 |
+
\scriptsize
|
| 892 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_learned_dominance_context_success_tanh_train_to_test/table}}
|
| 893 |
+
\end{table}
|
| 894 |
+
|
| 895 |
+
Table~\ref{tab:ctt-base-context-obs-tanh-learned-train-dominance} removes the
|
| 896 |
+
validation-calibration crutch. The train-calibrated context-success selector
|
| 897 |
+
uses 432 train measured tanh rows for calibration and reaches 38.19\% selected
|
| 898 |
+
success on held-out test at 18.06\% coverage. This matches the bounded-tanh
|
| 899 |
+
proposal oracle and exceeds the tanh base action at 33.33\%. The interpretation
|
| 900 |
+
is precise: under this bounded action convention, the current lightweight
|
| 901 |
+
selector can match the proposal set in micro success, but the row-level
|
| 902 |
+
success selector gap remains 12.50 points and CTT no longer generates enough
|
| 903 |
+
successful proposals. Support, with residual selection error, remains the main
|
| 904 |
+
blocker for this convention.
|
| 905 |
+
|
| 906 |
+
\begin{table}[t]
|
| 907 |
+
\centering
|
| 908 |
+
\caption{Train-calibrated \texttt{env\_clip} selector diagnostic. Calibration
|
| 909 |
+
uses train-split \texttt{env\_clip} measured rollout rows with same-chart and
|
| 910 |
+
same-state source retrieval excluded, then evaluates once on held-out test
|
| 911 |
+
\texttt{env\_clip} rows.}
|
| 912 |
+
\label{tab:ctt-base-context-obs-envclip-learned-train-dominance}
|
| 913 |
+
\scriptsize
|
| 914 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_learned_dominance_basic_envclip_train_to_test/table}}
|
| 915 |
+
\end{table}
|
| 916 |
+
|
| 917 |
+
Table~\ref{tab:ctt-base-context-obs-envclip-learned-train-dominance} tests
|
| 918 |
+
whether the better bounded proposal support translates through train-clean
|
| 919 |
+
selection. It does not yet. The best current env-clip selector is the basic
|
| 920 |
+
utility-margin ridge row: selected held-out test success reaches 31.94\% at
|
| 921 |
+
22.22\% coverage, above the env-clip base action at 29.17\% but far below the
|
| 922 |
+
47.92\% proposal oracle. The success selector gap remains 21.53 points. The
|
| 923 |
+
method evidence therefore separates the action-convention fix from the
|
| 924 |
+
remaining dominance problem: \texttt{env\_clip} preserves bounded proposal
|
| 925 |
+
support, but current selector features do not exploit it.
|
| 926 |
+
A train-clean nonlinear dominance sweep under the same \texttt{env\_clip}
|
| 927 |
+
rows is also negative: the best row reaches 30.56\% selected held-out test
|
| 928 |
+
success, below the ridge row. This suggests the current deployment-visible
|
| 929 |
+
features, not only linear model capacity, are the selector bottleneck.
|
| 930 |
+
|
| 931 |
+
\begin{table}[t]
|
| 932 |
+
\centering
|
| 933 |
+
\caption{Train-calibrated \texttt{env\_clip} source-evidence selector
|
| 934 |
+
diagnostic. This selector may read train-only source-chart positive/negative
|
| 935 |
+
tangent statistics because \ctt{} proposals are transported from measured train
|
| 936 |
+
positive source tangents; it does not read validation/test outcomes.}
|
| 937 |
+
\label{tab:ctt-base-context-obs-envclip-source-evidence-dominance}
|
| 938 |
+
\scriptsize
|
| 939 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_learned_dominance_source_envclip_train_to_test/table}}
|
| 940 |
+
\end{table}
|
| 941 |
+
|
| 942 |
+
Table~\ref{tab:ctt-base-context-obs-envclip-source-evidence-dominance} adds a
|
| 943 |
+
CTT-specific source-evidence feature family: source positive count, source
|
| 944 |
+
positive utility statistics, and the transported tangent's distance back to the
|
| 945 |
+
train source chart's positive/negative tangent sets. This is deployment-clean
|
| 946 |
+
because train chart outcomes are available to the CTT source memory, while
|
| 947 |
+
target validation/test outcomes remain evaluator-only. The row improves
|
| 948 |
+
selected held-out test success to 33.33\% at 23.61\% coverage, above the
|
| 949 |
+
env-clip base action at 29.17\% and the prior env-clip ridge selector at
|
| 950 |
+
31.94\%. The selector gap remains 20.14 points, so source evidence is useful
|
| 951 |
+
but not enough; the remaining failure still calls for richer chart
|
| 952 |
+
representation and dominance features.
|
| 953 |
+
|
| 954 |
+
\paragraph{K=16 support extension.}
|
| 955 |
+
The advisor's decomposition says the main scientific gap is proposal support,
|
| 956 |
+
not only selector calibration. We therefore reran the same declared
|
| 957 |
+
\texttt{env\_clip} convention with K=16 transported candidates. The train
|
| 958 |
+
calibration, validation, and held-out test arrays all completed with three
|
| 959 |
+
seeds and action-bound labels known for every base, selected, and generated
|
| 960 |
+
candidate. The K=16 validation split reaches 46.38\% proposal-oracle success
|
| 961 |
+
and \outcomeptr{}@16 of 53.62\%. The train-calibration split reaches 53.70\%
|
| 962 |
+
proposal-oracle success and \outcomeptr{}@16 of 54.63\%. The held-out test row
|
| 963 |
+
is the important support result: proposal-oracle success rises to 56.94\%,
|
| 964 |
+
\outcomeptr{}@16 is 54.86\%, the success support gap falls to 20.14 points, and
|
| 965 |
+
the generated action-bound unsafe rate remains 0.00\%.
|
| 966 |
+
|
| 967 |
+
\begin{table}[t]
|
| 968 |
+
\centering
|
| 969 |
+
\caption{Held-out test \texttt{env\_clip} measured rollout at K=16.
|
| 970 |
+
The table is generated by \texttt{scripts/eval\_metrics.py}; candidates are
|
| 971 |
+
actually rolled out, so \outcomeptr{}, SupportGap, and SelectorRegret are
|
| 972 |
+
measured rather than proxy quantities.}
|
| 973 |
+
\label{tab:ctt-base-context-obs-envclip-k16-test-rollout}
|
| 974 |
+
\scriptsize
|
| 975 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison/table}}
|
| 976 |
+
\end{table}
|
| 977 |
+
|
| 978 |
+
Table~\ref{tab:ctt-base-context-obs-envclip-k16-test-rollout} is a stronger
|
| 979 |
+
support result than the K=8 \texttt{env\_clip} row: the proposal oracle moves
|
| 980 |
+
from 47.92\% to 56.94\% on the same held-out test charts while preserving
|
| 981 |
+
action-bound validity. The score-only selector moves only from 23.61\% to
|
| 982 |
+
27.78\%, still below the measured base action. This is exactly the failure mode
|
| 983 |
+
the Causal Action Regret decomposition is meant to expose: CTT now finds many
|
| 984 |
+
positive measured tangents, but the present utility selector often chooses the
|
| 985 |
+
wrong one.
|
| 986 |
+
|
| 987 |
+
\begin{table}[t]
|
| 988 |
+
\centering
|
| 989 |
+
\caption{Train-calibrated lower-confidence dominance fallback on the same
|
| 990 |
+
held-out K=16 \texttt{env\_clip} measured rows. The conformal residual
|
| 991 |
+
quantile and threshold are fit on train-calibration rows only. The artifact now
|
| 992 |
+
reports action-bound unsafe execution and within-chart pairwise causal
|
| 993 |
+
calibration error.}
|
| 994 |
+
\label{tab:ctt-base-context-obs-envclip-k16-lcb-dominance}
|
| 995 |
+
\scriptsize
|
| 996 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test/table}}
|
| 997 |
+
\end{table}
|
| 998 |
+
|
| 999 |
+
Table~\ref{tab:ctt-base-context-obs-envclip-k16-lcb-dominance} is the clean
|
| 1000 |
+
Part-G decision-rule diagnostic for the K=16 support result. It executes a
|
| 1001 |
+
generated candidate only when a calibrated lower confidence bound on
|
| 1002 |
+
$F(a)-F(a_b)$ exceeds the learned threshold. The rule is action-bound clean
|
| 1003 |
+
under the available labels: safety label coverage is complete and unsafe
|
| 1004 |
+
execution rate is 0.00\%. It does not solve selection. The auto threshold
|
| 1005 |
+
covers 13.19\% of rows and reaches 27.78\% selected success; the fixed
|
| 1006 |
+
$\tau=0$ rerun falls back 88.89\% of the time and reaches the 29.17\% base
|
| 1007 |
+
rate. Both retain a 29.86-point success selector gap to the 56.94\% proposal
|
| 1008 |
+
oracle. The within-chart pairwise causal calibration error is 0.1633 over
|
| 1009 |
+
12{,}434 measured pairs, so this score-source LCB is a safety fallback
|
| 1010 |
+
diagnostic, not a reliable dominance certificate.
|
| 1011 |
+
|
| 1012 |
+
\begin{table}[t]
|
| 1013 |
+
\centering
|
| 1014 |
+
\caption{Best current train-calibrated \texttt{env\_clip} K=16 selector
|
| 1015 |
+
diagnostic. Calibration uses only train-split K=16 measured rollout rows with
|
| 1016 |
+
same-chart and same-state source retrieval excluded, then evaluates once on
|
| 1017 |
+
held-out test K=16 rows.}
|
| 1018 |
+
\label{tab:ctt-base-context-obs-envclip-k16-learned-train-dominance}
|
| 1019 |
+
\scriptsize
|
| 1020 |
+
\resizebox{\linewidth}{!}{\input{../runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test/table}}
|
| 1021 |
+
\end{table}
|
| 1022 |
+
|
| 1023 |
+
Table~\ref{tab:ctt-base-context-obs-envclip-k16-learned-train-dominance}
|
| 1024 |
+
tests whether a train-clean selector can exploit the improved K=16 support. It
|
| 1025 |
+
cannot yet, but the representation ablation now identifies a useful selector
|
| 1026 |
+
signal. The best row is a ridge selector over deployment-visible target--source
|
| 1027 |
+
RGB-stat chart compatibility, trained on utility-margin targets with Mondrian
|
| 1028 |
+
task-specific thresholds: 35.42\% selected success at 65.97\% coverage,
|
| 1029 |
+
compared with a 29.17\% base action and 56.94\% proposal oracle. The same
|
| 1030 |
+
artifact now reports pairwise causal calibration error over within-chart
|
| 1031 |
+
candidate contrasts; the held-out test error is 0.0150 over 12{,}434 measured
|
| 1032 |
+
pairs. Thus the remaining 24.31-point selector gap is not explained away by
|
| 1033 |
+
omitting the requested calibration metric. The previous basic
|
| 1034 |
+
success-weighted task-threshold row reached 32.64\%. RGB-stat
|
| 1035 |
+
compatibility with a success-weighted target reaches 33.33\%; object-layout
|
| 1036 |
+
compatibility reaches 31.25\%; and the combined RGB-stat/object-layout
|
| 1037 |
+
compatibility row ties 32.64\%. Source-evidence and nonlinear K=16 variants do
|
| 1038 |
+
not improve this: source utility-margin reaches 30.56\%, source success reaches
|
| 1039 |
+
30.56\%, context-source nonlinear ties 31.94\%, and task-scoped context/source
|
| 1040 |
+
variants are lower. We also tested a
|
| 1041 |
+
train-clean within-chart pairwise causal-ranking objective for the same ridge
|
| 1042 |
+
selector. Pairwise-only reaches 29.17\%, while hybrid pointwise-plus-pairwise
|
| 1043 |
+
basic, tangent, and source-evidence variants reach 31.25\%, 30.56\%, and
|
| 1044 |
+
29.17\%, respectively. Thus replacing the pointwise loss by a pairwise chart
|
| 1045 |
+
ranking loss is insufficient with the current features. We also fixed the
|
| 1046 |
+
nonlinear dominance diagnostic so chart-compatibility feature sets load the
|
| 1047 |
+
same deployment-visible target/source chart maps instead of zero-valued
|
| 1048 |
+
compatibility vectors. This does not improve the result: HGB classifier
|
| 1049 |
+
variants over RGB-stat compatibility reach 28.47\%, source+RGB-stat
|
| 1050 |
+
compatibility reaches 26.39\%, and the best regressor-only RGB-stat
|
| 1051 |
+
compatibility row reaches 32.64\% with pairwise calibration errors between
|
| 1052 |
+
0.0086 and 0.0222 for the rerun regressor diagnostics. The
|
| 1053 |
+
conclusion is sharper than the K=8 result: support is now strong enough to make
|
| 1054 |
+
the paper story credible, train-only visual chart compatibility is useful, but
|
| 1055 |
+
selector/utility energy is still the dominant remaining bottleneck.
|
| 1056 |
+
|
| 1057 |
The feature-source audits turn this negative result into a concrete engineering
|
| 1058 |
target. In the original \texttt{data/cil\_charts} indexes, scene ids and
|
| 1059 |
instructions are present, but observation embeddings and raw observation
|
|
|
|
| 1086 |
coverage, compared with 29.17\% measured base success and 51.39\% proposal
|
| 1087 |
oracle success. This is a small clean improvement over base, but it is lower
|
| 1088 |
than the validation-calibrated context diagnostic and leaves a 25.69-point
|
| 1089 |
+
success selector gap. We therefore treat train-only dominance as evidence that
|
| 1090 |
+
the current selector is underpowered, not as a solved deployment method.
|
|
|
|
| 1091 |
|
| 1092 |
\begin{table}[t]
|
| 1093 |
\centering
|
|
|
|
| 1138 |
\texttt{scripts/slurm/eval\_ctt\_generated\_rollout.sbatch} implement the
|
| 1139 |
measured generated-candidate rollout path, including a self-source exclusion
|
| 1140 |
flag for train-split calibration and metadata loading for deployment-visible
|
| 1141 |
+
chart features such as \texttt{base\_context\_obs}. They also record
|
| 1142 |
+
action-bound validity labels and support raw-action replay through
|
| 1143 |
+
\texttt{--disable-action-clipping} plus explicit scaled replay through
|
| 1144 |
+
\texttt{--execution-action-scale}; the same path logs bounded execution
|
| 1145 |
+
diagnostics such as \texttt{--execution-action-transform tanh}.
|
| 1146 |
+
\item \texttt{scripts/eval\_nonlinear\_dominance\_selector.py} shares the
|
| 1147 |
+
chart-compatibility feature-loading path with the ridge selector, including
|
| 1148 |
+
selector chart-feature mode and source split hashes, so nonlinear selector
|
| 1149 |
+
diagnostics are leakage-audited rather than zero-feature fallbacks.
|
| 1150 |
+
\item \texttt{scripts/audit\_action\_bounds.py} produces
|
| 1151 |
+
\texttt{runs/action\_bound\_audit\_rgb\_refs}, which audits whether exported
|
| 1152 |
+
chart actions lie inside the deployment action space before measured rollout.
|
| 1153 |
\item \texttt{scripts/build\_ctt\_rollout\_comparison.py} aggregates
|
| 1154 |
measured validation/test rollouts and reports selected success, proposal
|
| 1155 |
+
oracle success, hidden chart oracle success, \ncar{}, success support gap,
|
| 1156 |
+
and success selector gap.
|
| 1157 |
\item \texttt{scripts/eval\_dominance\_selector.py} calibrates a dominance
|
| 1158 |
+
fallback rule on calibration measured rows and evaluates it on held-out
|
| 1159 |
measured rows; it can use rollout row scores or recompute scores from a
|
| 1160 |
+
train-only utility-energy checkpoint, and its current artifacts report
|
| 1161 |
+
action-bound unsafe execution, pairwise causal calibration ECE, and
|
| 1162 |
+
coverage/fallback rates for the LCB decision rule.
|
| 1163 |
\item \texttt{scripts/eval\_learned\_dominance\_selector.py} trains a small
|
| 1164 |
validation- or train-calibrated dominance model over deployment-visible
|
| 1165 |
candidate features and evaluates it on held-out measured test rows; it also
|
| 1166 |
+
logs feature/target ablations, context metadata, tangent-code variants,
|
| 1167 |
+
Mondrian task thresholds, and within-chart pairwise causal-ranking variants
|
| 1168 |
+
for selector diagnostics.
|
| 1169 |
\item \texttt{scripts/eval\_nonlinear\_dominance\_selector.py} runs the
|
| 1170 |
train-calibrated nonlinear selector diagnostics in
|
| 1171 |
\texttt{runs/ctt\_base\_context\_obs\_nonlinear\_dominance\_*}.
|
|
|
|
| 1173 |
\texttt{scripts/build\_ctt\_proxy\_comparison.py} generate the local-atlas
|
| 1174 |
baseline and validation proxy gate table.
|
| 1175 |
\item \texttt{scripts/check\_tangent\_reconstruction.py} verifies that the
|
| 1176 |
+
exported 21D tangent codes are deterministic summaries of
|
| 1177 |
+
\texttt{delta\_action}; \texttt{runs/tangent\_reconstruction} and
|
| 1178 |
+
\texttt{runs/tangent\_reconstruction\_rgb\_refs} each check 45{,}968 rows
|
| 1179 |
+
with zero failures and log chart data/split hashes.
|
| 1180 |
+
\item \texttt{scripts/audit\_cil\_charts.py} writes the leakage reports
|
| 1181 |
+
\texttt{runs/leakage\_audit} and
|
| 1182 |
+
\texttt{runs/leakage\_audit\_rgb\_refs}, both currently passing with no
|
| 1183 |
+
violations or hash-artifact warnings.
|
| 1184 |
\item \texttt{scripts/train\_utility\_energy.py} and
|
| 1185 |
\texttt{scripts/calibrate\_dominance.py} implement the utility/scoring and
|
| 1186 |
dominance-calibration path.
|
| 1187 |
\item \texttt{scripts/summarize\_ctt\_runs.py} generates
|
| 1188 |
+
\texttt{runs/summary\_ctt.csv}; the persistent prose overview is consolidated
|
| 1189 |
+
in \texttt{README.md}.
|
| 1190 |
+
\item \texttt{scripts/audit\_ctt\_paper\_artifacts.py} audits forbidden
|
| 1191 |
+
wording, paper table inputs, required implementation paths, and run artifact
|
| 1192 |
+
contracts, writing \texttt{runs/paper\_ctt\_audit/audit.json} and a TeX table.
|
| 1193 |
+
\item \texttt{scripts/backfill\_paper\_run\_artifacts.py} transparently
|
| 1194 |
+
backfills non-Markdown run metadata such as missing grouped-metric placeholders,
|
| 1195 |
+
config stubs, and log stubs for paper-referenced run dirs while preserving
|
| 1196 |
+
existing files and not recreating deleted Markdown reports.
|
| 1197 |
+
\item \texttt{paper/sections/theory.tex} states the theory obligations.
|
| 1198 |
\end{itemize}
|
| 1199 |
|
| 1200 |
+
\begin{table}[t]
|
| 1201 |
+
\centering
|
| 1202 |
+
\caption{Claim-to-artifact audit for this draft. The audit is generated by
|
| 1203 |
+
\texttt{scripts/audit\_ctt\_paper\_artifacts.py}. Warnings track the advisor's
|
| 1204 |
+
full run-contract fields such as per-run Markdown reports and logs; the current
|
| 1205 |
+
workspace policy keeps persistent prose consolidated in \texttt{README.md}.}
|
| 1206 |
+
\label{tab:paper-ctt-artifact-audit}
|
| 1207 |
+
\scriptsize
|
| 1208 |
+
\input{../runs/paper_ctt_audit/table}
|
| 1209 |
+
\end{table}
|
| 1210 |
+
|
| 1211 |
\section{Limitations and Next Steps}
|
| 1212 |
|
| 1213 |
This draft is not yet a final deployment method. The current \ctt{} evidence
|
| 1214 |
includes validation and test measured generated-candidate rollouts, but the
|
| 1215 |
+
selected action remains weak even when the clipped-convention test proposal
|
| 1216 |
+
oracle passes 50\%. The missing method component is calibrated dominance and a better
|
| 1217 |
utility selector, not another proxy-only support plot. The first standalone
|
| 1218 |
train-only utility-energy checkpoints and context-metadata ridge variants do
|
| 1219 |
not solve this transfer problem, so the chart token must move beyond base-action
|
|
|
|
| 1221 |
object-centric geometry. The RGB-reference export now provides a leakage-audited
|
| 1222 |
visual-stat token and improves measured support plus validation-calibrated
|
| 1223 |
selected success, but this does not qualify as the needed learned object-centric
|
| 1224 |
+
representation because the best K=16 \texttt{env\_clip} train-clean selected
|
| 1225 |
+
success is still only 35.42\% and the selector gap remains 24.31 points. The
|
| 1226 |
+
bounded tanh diagnostic now
|
| 1227 |
+
has a train-calibrated selector reaching 38.19\% selected success, but with lower
|
| 1228 |
+
proposal support than the clipped visual-stat row.
|
| 1229 |
+
The no-clipping refresh further shows that most
|
| 1230 |
+
decoded actions violate the nominal deployment action bounds before clipping, so
|
| 1231 |
+
action scaling/decoding must be resolved before safety or deployment claims are
|
| 1232 |
+
made. A global max-fit execution scale of 0.215 fixes base action-bound
|
| 1233 |
+
violations but collapses proposal support and leaves nonzero generated
|
| 1234 |
+
action-bound violations, so the next fix must be representational rather than a
|
| 1235 |
+
post-hoc scalar. The bounded tanh decoder is action-bound-clean on full
|
| 1236 |
+
validation/test, but it sacrifices too much support to be promoted as the
|
| 1237 |
+
decoder fix. Related work experiments, external benchmarks, real robot
|
| 1238 |
+
near-miss recovery, unsafe-contact measurement, and a stronger train-clean
|
| 1239 |
+
dominance rule remain to be completed before submission.
|
| 1240 |
|
| 1241 |
The next experimental step is concrete: replace the current weak train-only
|
| 1242 |
utility-energy selector with visual-language/object-centric chart features,
|
| 1243 |
+
rerun held-out measured dominance selection, and collect actual unsafe-contact
|
| 1244 |
+
labels for the safety/fallback metrics now present in the evaluator. Proxy
|
| 1245 |
+
evidence may open the rollout gate; it cannot replace rollout measurement.
|
| 1246 |
|
| 1247 |
\section{Conclusion}
|
| 1248 |
|