anhtld commited on
Commit
ec445d9
·
verified ·
1 Parent(s): ae2c3e7

auto-sync 2026-07-04T03:59:09Z workspace

Browse files
workspace/README.md CHANGED
@@ -259,12 +259,15 @@ CTT training/proxy/rollout:
259
 
260
  - `scripts/train_ctt.py`: trains residual or gated residual CTT.
261
  - `scripts/eval_ctt_proxy.py`: proxy support evaluation with PPTC,
262
- NegativeNear, distance, diversity, and collapse metrics.
 
263
  - `scripts/eval_ctt_generated_rollout.py`: measured rollout harness that
264
  restores states, decodes generated tangents, executes candidates, and writes
265
  measured candidate rows.
266
  - `scripts/eval_ctt_rollout.py`: measured-output wrapper.
267
- - `scripts/build_ctt_proxy_comparison.py`: proxy comparison/gate table.
 
 
268
  - `scripts/build_ctt_rollout_comparison.py`: measured rollout aggregation.
269
  - `scripts/summarize_ctt_runs.py`: global CSV/Markdown summary. Markdown output
270
  may be generated transiently, but the persistent overview is this README.
@@ -277,8 +280,10 @@ Dominance and utility:
277
  Reports selected success, coverage, fallback, unsafe execution, PCCE,
278
  selector regret, and support/selector gaps.
279
  - `scripts/eval_learned_dominance_selector.py`: ridge learned dominance
280
- selector with basic/context/tangent/source/chart-compat features.
 
281
  - `scripts/eval_nonlinear_dominance_selector.py`: nonlinear selector sweep.
 
282
 
283
  Metric and paper artifacts:
284
 
@@ -410,6 +415,7 @@ High-value run directories:
410
  - `runs/ctt_gated_residual_full_seed{0,1,2}`: full gated residual CTT training.
411
  - `runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison`: current
412
  strongest measured support artifact.
 
413
  - `runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison`: train
414
  calibration rows for K=16 `env_clip`.
415
  - `runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test`: K=16 LCB
@@ -418,6 +424,8 @@ High-value run directories:
418
  LCB tau0 fallback, matches base but does not improve.
419
  - `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`:
420
  best current train-clean selector diagnostic.
 
 
421
  - `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*`: fixed
422
  nonlinear selector diagnostics.
423
  - `runs/summary_ctt.csv`: global run summary table.
@@ -492,6 +500,29 @@ Key held-out test values:
492
 
493
  Interpretation: support is strong; selector is not.
494
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
495
  ### Best current K=16 train-clean learned selector
496
 
497
  Run:
@@ -514,6 +545,22 @@ Key values:
514
  Interpretation: RGB-stat chart compatibility helps, but the selector gap is
515
  still too large for a deployment-clean method success claim.
516
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
517
  ### K=16 score-source LCB dominance
518
 
519
  Runs:
 
259
 
260
  - `scripts/train_ctt.py`: trains residual or gated residual CTT.
261
  - `scripts/eval_ctt_proxy.py`: proxy support evaluation with PPTC,
262
+ NegativeNear, PosCloserThanNeg, distance, diversity, and collapse metrics.
263
+ Use `--no-markdown-report` for README-only runs.
264
  - `scripts/eval_ctt_generated_rollout.py`: measured rollout harness that
265
  restores states, decodes generated tangents, executes candidates, and writes
266
  measured candidate rows.
267
  - `scripts/eval_ctt_rollout.py`: measured-output wrapper.
268
+ - `scripts/build_ctt_proxy_comparison.py`: proxy comparison/gate table with
269
+ by-task/by-seed JSON outputs. Use `--no-markdown-report` for README-only
270
+ runs.
271
  - `scripts/build_ctt_rollout_comparison.py`: measured rollout aggregation.
272
  - `scripts/summarize_ctt_runs.py`: global CSV/Markdown summary. Markdown output
273
  may be generated transiently, but the persistent overview is this README.
 
280
  Reports selected success, coverage, fallback, unsafe execution, PCCE,
281
  selector regret, and support/selector gaps.
282
  - `scripts/eval_learned_dominance_selector.py`: ridge learned dominance
283
+ selector with basic/context/tangent/source/chart-compat and score-shape
284
+ features. Use `--no-markdown-report` for README-only runs.
285
  - `scripts/eval_nonlinear_dominance_selector.py`: nonlinear selector sweep.
286
+ Use `--no-markdown-report` for README-only runs.
287
 
288
  Metric and paper artifacts:
289
 
 
415
  - `runs/ctt_gated_residual_full_seed{0,1,2}`: full gated residual CTT training.
416
  - `runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison`: current
417
  strongest measured support artifact.
418
+ - `runs/ctt_val_proxy_comparison`: CTT-vs-local-atlas proxy support gate.
419
  - `runs/ctt_base_context_obs_train_cal_envclip_k16_rollout_comparison`: train
420
  calibration rows for K=16 `env_clip`.
421
  - `runs/ctt_base_context_obs_dominance_envclip_k16_train_to_test`: K=16 LCB
 
424
  LCB tau0 fallback, matches base but does not improve.
425
  - `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`:
426
  best current train-clean selector diagnostic.
427
+ - `runs/ctt_base_context_obs_learned_dominance_score_*_envclip_k16_train_to_test`:
428
+ score-shape selector diagnostics; these did not improve the best selector.
429
  - `runs/ctt_base_context_obs_nonlinear_dominance_chartcompat_obs_*`: fixed
430
  nonlinear selector diagnostics.
431
  - `runs/summary_ctt.csv`: global run summary table.
 
500
 
501
  Interpretation: support is strong; selector is not.
502
 
503
+ ### CTT validation proxy support gate
504
+
505
+ Run:
506
+
507
+ ```text
508
+ runs/ctt_val_proxy_comparison
509
+ ```
510
+
511
+ This is a proxy geometry gate, not rollout success. CTT variants pass by
512
+ improving mean distance to target positives while staying within the
513
+ NegativeNear@0.20 safety slack; they do not beat local-atlas on PPTC thresholds.
514
+ The gated residual variant fails the safety gate.
515
+
516
+ | Method | PPTC@0.20 | PPTC@0.40 | Neg@0.20 | Pos<Neg | MeanPos | Collapse | Gate |
517
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- |
518
+ | local-atlas | 0.4058 | 0.6812 | 0.0368 | 0.5998 | 0.7203 | 0.0661 | baseline |
519
+ | CTT residual full | 0.1981 | 0.6087 | 0.0296 | 0.7352 | 0.4509 | 0.0681 | pass |
520
+ | CTT residual base+context+obs | 0.2464 | 0.6425 | 0.0343 | 0.7717 | 0.4347 | 0.0681 | pass |
521
+ | CTT gated residual full | 0.2319 | 0.6135 | 0.0527 | 0.7248 | 0.4337 | 0.0681 | fail |
522
+
523
+ Interpretation: CTT improves support distance geometry, but the story is still
524
+ diagnostic until measured rollout and selection close the outcome gap.
525
+
526
  ### Best current K=16 train-clean learned selector
527
 
528
  Run:
 
545
  Interpretation: RGB-stat chart compatibility helps, but the selector gap is
546
  still too large for a deployment-clean method success claim.
547
 
548
+ ### K=16 score-shape selector diagnostic
549
+
550
+ New score-relative features were added to test whether visible row-shape
551
+ signals can close the selector gap. They did not beat the current best
552
+ chart-compat selector.
553
+
554
+ | Run family | Selected success | Coverage | Success selector gap |
555
+ | --- | ---: | ---: | ---: |
556
+ | `score_chart_compat`, utility | 0.3264 | 0.6389 | 0.2500 |
557
+ | `score_context_chart_compat`, utility | 0.3264 | 0.6458 | 0.2500 |
558
+ | `score_chart_compat`, success bonus 2 | 0.3194 | 0.5486 | 0.2500 |
559
+ | `score_context_chart_compat`, success bonus 2 | 0.3403 | 0.6111 | 0.2292 |
560
+
561
+ Interpretation: small deployment-visible selector features are not enough; the
562
+ paper should keep emphasizing positive tangent support generation and CTT.
563
+
564
  ### K=16 score-source LCB dominance
565
 
566
  Runs:
workspace/latex/main.fdb_latexmk CHANGED
@@ -1,13 +1,13 @@
1
  # Fdb version 4
2
- ["bibtex main"] 1783136317 "main.aux" "main.bbl" "main" 1783136580 0
3
  "./references.bib" 1783037215 3572 027cd2cf27b344368db784320d235a8c ""
4
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/bibtex/bst/base/plain.bst" 1292289607 20613 bd3fbfa9f64872b81ac57a0dd2ed855f ""
5
- "main.aux" 1783136580 29307 d98354bd73f69b1589cc6a0a911ec166 "pdflatex"
6
  (generated)
7
  "main.bbl"
8
  "main.blg"
9
  (rewritten before read)
10
- ["pdflatex"] 1783136579 "main.tex" "main.pdf" "main" 1783136580 0
11
  "../paper/sections/theory.tex" 1783037322 4023 e8c0485606fdfea0020faae044bf1189 ""
12
  "../runs/action_bound_audit_rgb_refs/table.tex" 1783102548 379 5a7d42a0f3c58a06c4eeeaa280615fb3 ""
13
  "../runs/ctt_base_context_obj_val_rollout_comparison/table.tex" 1783103591 2172 16a82c2b9b79cbd8d2afdfca5503a296 ""
@@ -37,10 +37,10 @@
37
  "../runs/ctt_learned_dominance_val_to_test/table.tex" 1783051115 342 1951fdb40cb81709d46ad2391233a02b ""
38
  "../runs/ctt_residual_smoke_proxy/table.tex" 1783037068 479 a3687dfb2376b4f4dbfc05bec09c73c2 ""
39
  "../runs/ctt_test_rollout_comparison/table.tex" 1783103576 2171 9cc76279db5d4ffeb30bd3108263eda6 ""
40
- "../runs/ctt_val_proxy_comparison/table.tex" 1783098713 956 4a14d2ddd0ebb5593886e8675922a848 ""
41
  "../runs/ctt_val_rollout_comparison/table.tex" 1783103576 1874 6e2344e80436f3a5295e8d9c128d776d ""
42
  "../runs/data_accounting/table.tex" 1783034968 472 a35c3885ec2fd2c61f296ca898e5ee0e ""
43
- "../runs/paper_ctt_audit/table.tex" 1783136579 332 8ab5da6bd81c0791182283e8bd19a1c7 ""
44
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.enc" 1136849721 2900 1537cc8184ad1792082cd229ecc269f4 ""
45
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/map/fontname/texfonts.map" 1577235249 3524 cb3e574dea2d1052e39280babc910dc8 ""
46
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/tfm/jknappen/ec/tcrm1000.tfm" 1136768653 1536 e07581a4bb3136ece9eeb4c3ffab8233 ""
@@ -160,9 +160,9 @@
160
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/web2c/texmf.cnf" 1692823885 39639 818dd8e5a0dce3a72597a7e6bad45d55 ""
161
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map" 1692821535 5031293 07973d5e761f645996ac32ebf7b8d1f3 ""
162
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1692821530 1386016 abd9ba6eb53c47bc5de8d672b6ac87d3 ""
163
- "main.aux" 1783136580 29307 d98354bd73f69b1589cc6a0a911ec166 "pdflatex"
164
  "main.bbl" 1783136317 3120 92555a174bae4ac684748bbe3fae0b8c "bibtex main"
165
- "main.out" 1783136580 2647 6c337a545f287573528c7cb52648bc95 "pdflatex"
166
  "main.tex" 1783136572 66789 8a257715a970bb392721a3fb72771b0d ""
167
  "tables/car_decomposition.tex" 1783032219 662 bac24c0014c2ba573860ce199b776934 ""
168
  "tables/main_results.tex" 1783004655 1061 19bfdcc844614aadefc921dae15c700d ""
 
1
  # Fdb version 4
2
+ ["bibtex main"] 1783136317 "main.aux" "main.bbl" "main" 1783137885 0
3
  "./references.bib" 1783037215 3572 027cd2cf27b344368db784320d235a8c ""
4
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/bibtex/bst/base/plain.bst" 1292289607 20613 bd3fbfa9f64872b81ac57a0dd2ed855f ""
5
+ "main.aux" 1783137885 29307 d98354bd73f69b1589cc6a0a911ec166 "pdflatex"
6
  (generated)
7
  "main.bbl"
8
  "main.blg"
9
  (rewritten before read)
10
+ ["pdflatex"] 1783137884 "main.tex" "main.pdf" "main" 1783137885 0
11
  "../paper/sections/theory.tex" 1783037322 4023 e8c0485606fdfea0020faae044bf1189 ""
12
  "../runs/action_bound_audit_rgb_refs/table.tex" 1783102548 379 5a7d42a0f3c58a06c4eeeaa280615fb3 ""
13
  "../runs/ctt_base_context_obj_val_rollout_comparison/table.tex" 1783103591 2172 16a82c2b9b79cbd8d2afdfca5503a296 ""
 
37
  "../runs/ctt_learned_dominance_val_to_test/table.tex" 1783051115 342 1951fdb40cb81709d46ad2391233a02b ""
38
  "../runs/ctt_residual_smoke_proxy/table.tex" 1783037068 479 a3687dfb2376b4f4dbfc05bec09c73c2 ""
39
  "../runs/ctt_test_rollout_comparison/table.tex" 1783103576 2171 9cc76279db5d4ffeb30bd3108263eda6 ""
40
+ "../runs/ctt_val_proxy_comparison/table.tex" 1783137797 1201 46b7a5babb84523c9075eeafcf5c151c ""
41
  "../runs/ctt_val_rollout_comparison/table.tex" 1783103576 1874 6e2344e80436f3a5295e8d9c128d776d ""
42
  "../runs/data_accounting/table.tex" 1783034968 472 a35c3885ec2fd2c61f296ca898e5ee0e ""
43
+ "../runs/paper_ctt_audit/table.tex" 1783137884 332 8ab5da6bd81c0791182283e8bd19a1c7 ""
44
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.enc" 1136849721 2900 1537cc8184ad1792082cd229ecc269f4 ""
45
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/map/fontname/texfonts.map" 1577235249 3524 cb3e574dea2d1052e39280babc910dc8 ""
46
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/tfm/jknappen/ec/tcrm1000.tfm" 1136768653 1536 e07581a4bb3136ece9eeb4c3ffab8233 ""
 
160
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/web2c/texmf.cnf" 1692823885 39639 818dd8e5a0dce3a72597a7e6bad45d55 ""
161
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map" 1692821535 5031293 07973d5e761f645996ac32ebf7b8d1f3 ""
162
  "/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1692821530 1386016 abd9ba6eb53c47bc5de8d672b6ac87d3 ""
163
+ "main.aux" 1783137885 29307 d98354bd73f69b1589cc6a0a911ec166 "pdflatex"
164
  "main.bbl" 1783136317 3120 92555a174bae4ac684748bbe3fae0b8c "bibtex main"
165
+ "main.out" 1783137885 2647 6c337a545f287573528c7cb52648bc95 "pdflatex"
166
  "main.tex" 1783136572 66789 8a257715a970bb392721a3fb72771b0d ""
167
  "tables/car_decomposition.tex" 1783032219 662 bac24c0014c2ba573860ce199b776934 ""
168
  "tables/main_results.tex" 1783004655 1061 19bfdcc844614aadefc921dae15c700d ""
workspace/latex/main.log CHANGED
@@ -1,4 +1,4 @@
1
- This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021 Gentoo Linux) (preloaded format=pdflatex 2023.8.23) 3 JUL 2026 23:42
2
  entering extended mode
3
  restricted \write18 enabled.
4
  %&-line parsing enabled.
@@ -723,7 +723,7 @@ Package rerunfilecheck Info: File `main.out' has not changed.
723
  Here is how much of TeX's memory you used:
724
  10282 strings out of 480884
725
  158917 string characters out of 5900692
726
- 718209 words of memory out of 5000000
727
  27157 multiletter control sequences out of 15000+600000
728
  414764 words of font info for 71 fonts, out of 8000000 for 9000
729
  36 hyphenation exceptions out of 8191
@@ -768,7 +768,7 @@ cm/cmtt9.pfb></cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/texm
768
  f-dist/fonts/type1/public/amsfonts/symbols/msbm10.pfb></cvmfs/soft.computecanad
769
  a.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/type1/public/cm-super/sfr
770
  m1000.pfb>
771
- Output written on main.pdf (33 pages, 433100 bytes).
772
  PDF statistics:
773
  452 PDF objects out of 1000 (max. 8388607)
774
  386 compressed objects within 4 object streams
 
1
+ This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021 Gentoo Linux) (preloaded format=pdflatex 2023.8.23) 4 JUL 2026 00:04
2
  entering extended mode
3
  restricted \write18 enabled.
4
  %&-line parsing enabled.
 
723
  Here is how much of TeX's memory you used:
724
  10282 strings out of 480884
725
  158917 string characters out of 5900692
726
+ 717209 words of memory out of 5000000
727
  27157 multiletter control sequences out of 15000+600000
728
  414764 words of font info for 71 fonts, out of 8000000 for 9000
729
  36 hyphenation exceptions out of 8191
 
768
  f-dist/fonts/type1/public/amsfonts/symbols/msbm10.pfb></cvmfs/soft.computecanad
769
  a.ca/gentoo/2023/x86-64-v3/usr/share/texmf-dist/fonts/type1/public/cm-super/sfr
770
  m1000.pfb>
771
+ Output written on main.pdf (33 pages, 433172 bytes).
772
  PDF statistics:
773
  452 PDF objects out of 1000 (max. 8388607)
774
  386 compressed objects within 4 object streams
workspace/latex/main.pdf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:68843586105c4feb0accff0fa9d5d449d20ff763b42597914f781db27b5c559e
3
  size 433100
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5cfe10c2761cb08a1a5739f05bd92ba6cbd1951379db3b6a00bda4e671f78c24
3
  size 433100