auto-sync 2026-07-04T03:59:09Z workspace
Browse files- workspace/README.md +50 -3
- workspace/latex/main.fdb_latexmk +7 -7
- workspace/latex/main.log +3 -3
- workspace/latex/main.pdf +1 -1
workspace/README.md
CHANGED
|
@@ -259,12 +259,15 @@ CTT training/proxy/rollout:
|
|
| 259 |
|
| 260 |
- `scripts/train_ctt.py`: trains residual or gated residual CTT.
|
| 261 |
- `scripts/eval_ctt_proxy.py`: proxy support evaluation with PPTC,
|
| 262 |
-
NegativeNear, distance, diversity, and collapse metrics.
|
|
|
|
| 263 |
- `scripts/eval_ctt_generated_rollout.py`: measured rollout harness that
|
| 264 |
restores states, decodes generated tangents, executes candidates, and writes
|
| 265 |
measured candidate rows.
|
| 266 |
- `scripts/eval_ctt_rollout.py`: measured-output wrapper.
|
| 267 |
-
- `scripts/build_ctt_proxy_comparison.py`: proxy comparison/gate table
|
|
|
|
|
|
|
| 268 |
- `scripts/build_ctt_rollout_comparison.py`: measured rollout aggregation.
|
| 269 |
- `scripts/summarize_ctt_runs.py`: global CSV/Markdown summary. Markdown output
|
| 270 |
may be generated transiently, but the persistent overview is this README.
|
|
@@ -277,8 +280,10 @@ Dominance and utility:
|
|
| 277 |
Reports selected success, coverage, fallback, unsafe execution, PCCE,
|
| 278 |
selector regret, and support/selector gaps.
|
| 279 |
- `scripts/eval_learned_dominance_selector.py`: ridge learned dominance
|
| 280 |
-
selector with basic/context/tangent/source/chart-compat
|
|
|
|
| 281 |
- `scripts/eval_nonlinear_dominance_selector.py`: nonlinear selector sweep.
|
|
|
|
| 282 |
|
| 283 |
Metric and paper artifacts:
|
| 284 |
|
|
@@ -410,6 +415,7 @@ High-value run directories:
|
|
| 410 |
- `runs/ctt_gated_residual_full_seed{0,1,2}`: full gated residual CTT training.
|
| 411 |
- `runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison`: current
|
| 412 |
strongest measured support artifact.
|
|
|
|
| 413 |
- `runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison`: train
|
| 414 |
calibration rows for K=16 `env_clip`.
|
| 415 |
- `runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test`: K=16 LCB
|
|
@@ -418,6 +424,8 @@ High-value run directories:
|
|
| 418 |
LCB tau0 fallback, matches base but does not improve.
|
| 419 |
- `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`:
|
| 420 |
best current train-clean selector diagnostic.
|
|
|
|
|
|
|
| 421 |
- `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*`: fixed
|
| 422 |
nonlinear selector diagnostics.
|
| 423 |
- `runs/summary_ctt.csv`: global run summary table.
|
|
@@ -492,6 +500,29 @@ Key held-out test values:
|
|
| 492 |
|
| 493 |
Interpretation: support is strong; selector is not.
|
| 494 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 495 |
### Best current K=16 train-clean learned selector
|
| 496 |
|
| 497 |
Run:
|
|
@@ -514,6 +545,22 @@ Key values:
|
|
| 514 |
Interpretation: RGB-stat chart compatibility helps, but the selector gap is
|
| 515 |
still too large for a deployment-clean method success claim.
|
| 516 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 517 |
### K=16 score-source LCB dominance
|
| 518 |
|
| 519 |
Runs:
|
|
|
|
| 259 |
|
| 260 |
- `scripts/train_ctt.py`: trains residual or gated residual CTT.
|
| 261 |
- `scripts/eval_ctt_proxy.py`: proxy support evaluation with PPTC,
|
| 262 |
+
NegativeNear, PosCloserThanNeg, distance, diversity, and collapse metrics.
|
| 263 |
+
Use `--no-markdown-report` for README-only runs.
|
| 264 |
- `scripts/eval_ctt_generated_rollout.py`: measured rollout harness that
|
| 265 |
restores states, decodes generated tangents, executes candidates, and writes
|
| 266 |
measured candidate rows.
|
| 267 |
- `scripts/eval_ctt_rollout.py`: measured-output wrapper.
|
| 268 |
+
- `scripts/build_ctt_proxy_comparison.py`: proxy comparison/gate table with
|
| 269 |
+
by-task/by-seed JSON outputs. Use `--no-markdown-report` for README-only
|
| 270 |
+
runs.
|
| 271 |
- `scripts/build_ctt_rollout_comparison.py`: measured rollout aggregation.
|
| 272 |
- `scripts/summarize_ctt_runs.py`: global CSV/Markdown summary. Markdown output
|
| 273 |
may be generated transiently, but the persistent overview is this README.
|
|
|
|
| 280 |
Reports selected success, coverage, fallback, unsafe execution, PCCE,
|
| 281 |
selector regret, and support/selector gaps.
|
| 282 |
- `scripts/eval_learned_dominance_selector.py`: ridge learned dominance
|
| 283 |
+
selector with basic/context/tangent/source/chart-compat and score-shape
|
| 284 |
+
features. Use `--no-markdown-report` for README-only runs.
|
| 285 |
- `scripts/eval_nonlinear_dominance_selector.py`: nonlinear selector sweep.
|
| 286 |
+
Use `--no-markdown-report` for README-only runs.
|
| 287 |
|
| 288 |
Metric and paper artifacts:
|
| 289 |
|
|
|
|
| 415 |
- `runs/ctt_gated_residual_full_seed{0,1,2}`: full gated residual CTT training.
|
| 416 |
- `runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison`: current
|
| 417 |
strongest measured support artifact.
|
| 418 |
+
- `runs/ctt_val_proxy_comparison`: CTT-vs-local-atlas proxy support gate.
|
| 419 |
- `runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison`: train
|
| 420 |
calibration rows for K=16 `env_clip`.
|
| 421 |
- `runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test`: K=16 LCB
|
|
|
|
| 424 |
LCB tau0 fallback, matches base but does not improve.
|
| 425 |
- `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`:
|
| 426 |
best current train-clean selector diagnostic.
|
| 427 |
+
- `runs/ctt_base_context_obs_learned_dominance_score_*_envclip_k16_train_to_test`:
|
| 428 |
+
score-shape selector diagnostics; these did not improve the best selector.
|
| 429 |
- `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*`: fixed
|
| 430 |
nonlinear selector diagnostics.
|
| 431 |
- `runs/summary_ctt.csv`: global run summary table.
|
|
|
|
| 500 |
|
| 501 |
Interpretation: support is strong; selector is not.
|
| 502 |
|
| 503 |
+
### CTT validation proxy support gate
|
| 504 |
+
|
| 505 |
+
Run:
|
| 506 |
+
|
| 507 |
+
```text
|
| 508 |
+
runs/ctt_val_proxy_comparison
|
| 509 |
+
```
|
| 510 |
+
|
| 511 |
+
This is a proxy geometry gate, not rollout success. CTT variants pass by
|
| 512 |
+
improving mean distance to target positives while staying within the
|
| 513 |
+
NegativeNear@0.20 safety slack; they do not beat local-atlas on PPTC thresholds.
|
| 514 |
+
The gated residual variant fails the safety gate.
|
| 515 |
+
|
| 516 |
+
| Method | PPTC@0.20 | PPTC@0.40 | Neg@0.20 | Pos<Neg | MeanPos | Collapse | Gate |
|
| 517 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | --- |
|
| 518 |
+
| local-atlas | 0.4058 | 0.6812 | 0.0368 | 0.5998 | 0.7203 | 0.0661 | baseline |
|
| 519 |
+
| CTT residual full | 0.1981 | 0.6087 | 0.0296 | 0.7352 | 0.4509 | 0.0681 | pass |
|
| 520 |
+
| CTT residual base+context+obs | 0.2464 | 0.6425 | 0.0343 | 0.7717 | 0.4347 | 0.0681 | pass |
|
| 521 |
+
| CTT gated residual full | 0.2319 | 0.6135 | 0.0527 | 0.7248 | 0.4337 | 0.0681 | fail |
|
| 522 |
+
|
| 523 |
+
Interpretation: CTT improves support distance geometry, but the story is still
|
| 524 |
+
diagnostic until measured rollout and selection close the outcome gap.
|
| 525 |
+
|
| 526 |
### Best current K=16 train-clean learned selector
|
| 527 |
|
| 528 |
Run:
|
|
|
|
| 545 |
Interpretation: RGB-stat chart compatibility helps, but the selector gap is
|
| 546 |
still too large for a deployment-clean method success claim.
|
| 547 |
|
| 548 |
+
### K=16 score-shape selector diagnostic
|
| 549 |
+
|
| 550 |
+
New score-relative features were added to test whether visible row-shape
|
| 551 |
+
signals can close the selector gap. They did not beat the current best
|
| 552 |
+
chart-compat selector.
|
| 553 |
+
|
| 554 |
+
| Run family | Selected success | Coverage | Success selector gap |
|
| 555 |
+
| --- | ---: | ---: | ---: |
|
| 556 |
+
| `score_chart_compat`, utility | 0.3264 | 0.6389 | 0.2500 |
|
| 557 |
+
| `score_context_chart_compat`, utility | 0.3264 | 0.6458 | 0.2500 |
|
| 558 |
+
| `score_chart_compat`, success bonus 2 | 0.3194 | 0.5486 | 0.2500 |
|
| 559 |
+
| `score_context_chart_compat`, success bonus 2 | 0.3403 | 0.6111 | 0.2292 |
|
| 560 |
+
|
| 561 |
+
Interpretation: small deployment-visible selector features are not enough; the
|
| 562 |
+
paper should keep emphasizing positive tangent support generation and CTT.
|
| 563 |
+
|
| 564 |
### K=16 score-source LCB dominance
|
| 565 |
|
| 566 |
Runs:
|
workspace/latex/main.fdb_latexmk
CHANGED
|
@@ -1,13 +1,13 @@
|
|
| 1 |
# Fdb version 4
|
| 2 |
-
["bibtex main"] 1783136317 "main.aux" "main.bbl" "main"
|
| 3 |
"./references.bib" 1783037215 3572 027cd2cf27b344368db784320d235a8c ""
|
| 4 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/bibtex/bst/base/plain.bst" 1292289607 20613 bd3fbfa9f64872b81ac57a0dd2ed855f ""
|
| 5 |
-
"main.aux"
|
| 6 |
(generated)
|
| 7 |
"main.bbl"
|
| 8 |
"main.blg"
|
| 9 |
(rewritten before read)
|
| 10 |
-
["pdflatex"]
|
| 11 |
"../paper/sections/theory.tex" 1783037322 4023 e8c0485606fdfea0020faae044bf1189 ""
|
| 12 |
"../runs/action_bound_audit_rgb_refs/table.tex" 1783102548 379 5a7d42a0f3c58a06c4eeeaa280615fb3 ""
|
| 13 |
"../runs/ctt_base_context_obj_val_rollout_comparison/table.tex" 1783103591 2172 16a82c2b9b79cbd8d2afdfca5503a296 ""
|
|
@@ -37,10 +37,10 @@
|
|
| 37 |
"../runs/ctt_learned_dominance_val_to_test/table.tex" 1783051115 342 1951fdb40cb81709d46ad2391233a02b ""
|
| 38 |
"../runs/ctt_residual_smoke_proxy/table.tex" 1783037068 479 a3687dfb2376b4f4dbfc05bec09c73c2 ""
|
| 39 |
"../runs/ctt_test_rollout_comparison/table.tex" 1783103576 2171 9cc76279db5d4ffeb30bd3108263eda6 ""
|
| 40 |
-
"../runs/ctt_val_proxy_comparison/table.tex"
|
| 41 |
"../runs/ctt_val_rollout_comparison/table.tex" 1783103576 1874 6e2344e80436f3a5295e8d9c128d776d ""
|
| 42 |
"../runs/data_accounting/table.tex" 1783034968 472 a35c3885ec2fd2c61f296ca898e5ee0e ""
|
| 43 |
-
"../runs/paper_ctt_audit/table.tex"
|
| 44 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.enc" 1136849721 2900 1537cc8184ad1792082cd229ecc269f4 ""
|
| 45 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/map/fontname/texfonts.map" 1577235249 3524 cb3e574dea2d1052e39280babc910dc8 ""
|
| 46 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/tfm/jknappen/ec/tcrm1000.tfm" 1136768653 1536 e07581a4bb3136ece9eeb4c3ffab8233 ""
|
|
@@ -160,9 +160,9 @@
|
|
| 160 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/web2c/texmf.cnf" 1692823885 39639 818dd8e5a0dce3a72597a7e6bad45d55 ""
|
| 161 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map" 1692821535 5031293 07973d5e761f645996ac32ebf7b8d1f3 ""
|
| 162 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1692821530 1386016 abd9ba6eb53c47bc5de8d672b6ac87d3 ""
|
| 163 |
-
"main.aux"
|
| 164 |
"main.bbl" 1783136317 3120 92555a174bae4ac684748bbe3fae0b8c "bibtex main"
|
| 165 |
-
"main.out"
|
| 166 |
"main.tex" 1783136572 66789 8a257715a970bb392721a3fb72771b0d ""
|
| 167 |
"tables/car_decomposition.tex" 1783032219 662 bac24c0014c2ba573860ce199b776934 ""
|
| 168 |
"tables/main_results.tex" 1783004655 1061 19bfdcc844614aadefc921dae15c700d ""
|
|
|
|
| 1 |
# Fdb version 4
|
| 2 |
+
["bibtex main"] 1783136317 "main.aux" "main.bbl" "main" 1783137885 0
|
| 3 |
"./references.bib" 1783037215 3572 027cd2cf27b344368db784320d235a8c ""
|
| 4 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/bibtex/bst/base/plain.bst" 1292289607 20613 bd3fbfa9f64872b81ac57a0dd2ed855f ""
|
| 5 |
+
"main.aux" 1783137885 29307 d98354bd73f69b1589cc6a0a911ec166 "pdflatex"
|
| 6 |
(generated)
|
| 7 |
"main.bbl"
|
| 8 |
"main.blg"
|
| 9 |
(rewritten before read)
|
| 10 |
+
["pdflatex"] 1783137884 "main.tex" "main.pdf" "main" 1783137885 0
|
| 11 |
"../paper/sections/theory.tex" 1783037322 4023 e8c0485606fdfea0020faae044bf1189 ""
|
| 12 |
"../runs/action_bound_audit_rgb_refs/table.tex" 1783102548 379 5a7d42a0f3c58a06c4eeeaa280615fb3 ""
|
| 13 |
"../runs/ctt_base_context_obj_val_rollout_comparison/table.tex" 1783103591 2172 16a82c2b9b79cbd8d2afdfca5503a296 ""
|
|
|
|
| 37 |
"../runs/ctt_learned_dominance_val_to_test/table.tex" 1783051115 342 1951fdb40cb81709d46ad2391233a02b ""
|
| 38 |
"../runs/ctt_residual_smoke_proxy/table.tex" 1783037068 479 a3687dfb2376b4f4dbfc05bec09c73c2 ""
|
| 39 |
"../runs/ctt_test_rollout_comparison/table.tex" 1783103576 2171 9cc76279db5d4ffeb30bd3108263eda6 ""
|
| 40 |
+
"../runs/ctt_val_proxy_comparison/table.tex" 1783137797 1201 46b7a5babb84523c9075eeafcf5c151c ""
|
| 41 |
"../runs/ctt_val_rollout_comparison/table.tex" 1783103576 1874 6e2344e80436f3a5295e8d9c128d776d ""
|
| 42 |
"../runs/data_accounting/table.tex" 1783034968 472 a35c3885ec2fd2c61f296ca898e5ee0e ""
|
| 43 |
+
"../runs/paper_ctt_audit/table.tex" 1783137884 332 8ab5da6bd81c0791182283e8bd19a1c7 ""
|
| 44 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.enc" 1136849721 2900 1537cc8184ad1792082cd229ecc269f4 ""
|
| 45 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/map/fontname/texfonts.map" 1577235249 3524 cb3e574dea2d1052e39280babc910dc8 ""
|
| 46 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/tfm/jknappen/ec/tcrm1000.tfm" 1136768653 1536 e07581a4bb3136ece9eeb4c3ffab8233 ""
|
|
|
|
| 160 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/web2c/texmf.cnf" 1692823885 39639 818dd8e5a0dce3a72597a7e6bad45d55 ""
|
| 161 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map" 1692821535 5031293 07973d5e761f645996ac32ebf7b8d1f3 ""
|
| 162 |
"/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1692821530 1386016 abd9ba6eb53c47bc5de8d672b6ac87d3 ""
|
| 163 |
+
"main.aux" 1783137885 29307 d98354bd73f69b1589cc6a0a911ec166 "pdflatex"
|
| 164 |
"main.bbl" 1783136317 3120 92555a174bae4ac684748bbe3fae0b8c "bibtex main"
|
| 165 |
+
"main.out" 1783137885 2647 6c337a545f287573528c7cb52648bc95 "pdflatex"
|
| 166 |
"main.tex" 1783136572 66789 8a257715a970bb392721a3fb72771b0d ""
|
| 167 |
"tables/car_decomposition.tex" 1783032219 662 bac24c0014c2ba573860ce199b776934 ""
|
| 168 |
"tables/main_results.tex" 1783004655 1061 19bfdcc844614aadefc921dae15c700d ""
|
workspace/latex/main.log
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021 Gentoo Linux) (preloaded format=pdflatex 2023.8.23)
|
| 2 |
entering extended mode
|
| 3 |
restricted \write18 enabled.
|
| 4 |
%&-line parsing enabled.
|
|
@@ -723,7 +723,7 @@ Package rerunfilecheck Info: File `main.out' has not changed.
|
|
| 723 |
Here is how much of TeX's memory you used:
|
| 724 |
10282 strings out of 480884
|
| 725 |
158917 string characters out of 5900692
|
| 726 |
-
|
| 727 |
27157 multiletter control sequences out of 15000+600000
|
| 728 |
414764 words of font info for 71 fonts, out of 8000000 for 9000
|
| 729 |
36 hyphenation exceptions out of 8191
|
|
@@ -768,7 +768,7 @@ cm/cmtt9.pfb></cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texm
|
|
| 768 |
f-dist/fonts/type1/public/amsfonts/symbols/msbm10.pfb></cvmfs/soft.computecanad
|
| 769 |
a.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/type1/public/cm-super/sfr
|
| 770 |
m1000.pfb>
|
| 771 |
-
Output written on main.pdf (33 pages,
|
| 772 |
PDF statistics:
|
| 773 |
452 PDF objects out of 1000 (max. 8388607)
|
| 774 |
386 compressed objects within 4 object streams
|
|
|
|
| 1 |
+
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021 Gentoo Linux) (preloaded format=pdflatex 2023.8.23) 4 JUL 2026 00:04
|
| 2 |
entering extended mode
|
| 3 |
restricted \write18 enabled.
|
| 4 |
%&-line parsing enabled.
|
|
|
|
| 723 |
Here is how much of TeX's memory you used:
|
| 724 |
10282 strings out of 480884
|
| 725 |
158917 string characters out of 5900692
|
| 726 |
+
717209 words of memory out of 5000000
|
| 727 |
27157 multiletter control sequences out of 15000+600000
|
| 728 |
414764 words of font info for 71 fonts, out of 8000000 for 9000
|
| 729 |
36 hyphenation exceptions out of 8191
|
|
|
|
| 768 |
f-dist/fonts/type1/public/amsfonts/symbols/msbm10.pfb></cvmfs/soft.computecanad
|
| 769 |
a.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/type1/public/cm-super/sfr
|
| 770 |
m1000.pfb>
|
| 771 |
+
Output written on main.pdf (33 pages, 433172 bytes).
|
| 772 |
PDF statistics:
|
| 773 |
452 PDF objects out of 1000 (max. 8388607)
|
| 774 |
386 compressed objects within 4 object streams
|
workspace/latex/main.pdf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 433100
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5cfe10c2761cb08a1a5739f05bd92ba6cbd1951379db3b6a00bda4e671f78c24
|
| 3 |
size 433100
|