All Languages Pooled

highest in row   lowest in row  Sorted by Δ (max − min) descending.

Node visit rates — sorted by Δ

stateGemma (100)Qwen (100)GPT-5.5 (95)Sonnet 4.6 (95)GLM-5.2 (81)Δ
start1.01.01.01.01.00.00
improved2.01.80.00.20.01.98
churn0.10.30.10.22
stuck11.24.30.20.30.111.07
breakthrough4.82.01.31.21.33.66
stay5.71.72.62.62.53.95
regressed0.50.30.10.20.10.46
broke5.52.30.30.10.35.43
end_submit0.80.81.00.91.00.19
end_limit0.20.20.10.00.17

Edge visit rates (count / n_problems) — sorted by Δ

transitionGemma (100)Qwen (100)GPT-5.5 (95)Sonnet 4.6 (95)GLM-5.2 (81)Δ (max−min)
stuckstuck9.1×2.9×0.1×0.1×0.0×9.04×
brokebreakthrough3.0×0.5×0.2×0.0×0.2×3.02×
staystay3.3×0.8×1.6×1.7×1.5×2.43×
breakthroughbroke2.0×0.6×0.0×0.0×1.95×
staybroke2.0×0.5×0.3×0.0×0.3×1.95×
brokestuck1.6×1.1×0.1×0.0×0.1×1.55×
breakthroughstay2.4×0.9×1.0×0.9×1.0×1.52×
improvedbroke1.1×0.8×0.0×0.0×1.08×
brokeimproved0.9×0.7×0.0×0.0×0.0×0.86×
stuckimproved0.8×0.6×0.0×0.0×0.81×
stuckbreakthrough0.7×0.5×0.1×0.1×0.1×0.62×
stayend_submit0.3×0.4×0.7×0.8×0.7×0.49×
improvedstuck0.3×0.1×0.0×0.0×0.33×
stuckend_submit0.3×0.1×0.0×0.0×0.25×
startbreakthrough0.7×0.7×1.0×0.9×1.0×0.25×
breakthroughend_submit0.2×0.3×0.3×0.1×0.3×0.25×
breakthroughregressed0.2×0.2×0.1×0.1×0.0×0.23×
improvedchurn0.1×0.2×0.0×0.21×
stuckbroke0.2×0.1×0.0×0.0×0.0×0.20×
regressedbroke0.2×0.2×0.0×0.0×0.0×0.19×
improvedbreakthrough0.2×0.2×0.0×0.1×0.0×0.18×
improvedimproved0.1×0.2×0.0×0.17×
startstuck0.2×0.1×0.0×0.0×0.0×0.16×
startimproved0.1×0.2×0.0×0.1×0.0×0.15×
regressedbreakthrough0.2×0.1×0.1×0.1×0.0×0.13×
improvedregressed0.1×0.1×0.0×0.11×
churnbroke0.1×0.1×0.07×
stuckend_limit0.1×0.1×0.0×0.07×
regressedimproved0.1×0.0×0.0×0.05×
regressedstuck0.0×0.0×0.0×0.1×0.0×0.04×
stayregressed0.1×0.0×0.0×0.1×0.0×0.04×
stayend_limit0.1×0.0×0.0×0.04×
improvedend_limit0.0×0.0×0.0×0.03×
regressedchurn0.0×0.0×0.03×
improvedend_submit0.0×0.1×0.02×
churnimproved0.0×0.0×0.02×
churnchurn0.0×0.0×0.0×0.02×
churnregressed0.0×0.0×0.0×0.02×
churnbreakthrough0.0×0.0×0.01×
regressedend_submit0.0×0.0×0.01×
breakthroughend_limit0.0×0.0×0.0×0.0×0.01×
churnend_limit0.0×0.0×0.01×
regressedregressed0.0×0.0×0.0×0.01×
brokeend_limit0.0×0.0×0.00×
brokeend_submit0.0×0.00×
churnstuck0.0×0.00×
regressedend_limit0.0×0.00×
stuckregressed0.1×0.00×

Transition probabilities P(tgt | src) — sorted by Δ

transitionGemma (100)Qwen (100)GPT-5.5 (95)Sonnet 4.6 (95)GLM-5.2 (81)Δ (max−min)
improvedbreakthrough10.0%9.7%66.7%47.1%66.7%57.0pp
regressedbroke36.5%57.1%10.0%4.3%20.0%52.8pp
stuckstuck81.1%68.6%29.4%48.0%36.4%51.7pp
brokebreakthrough55.5%20.5%67.9%42.9%72.0%51.5pp
improvedbroke54.2%46.9%5.9%33.3%48.3pp
stuckbreakthrough6.2%11.9%47.1%28.0%54.5%48.3pp
breakthroughbroke40.7%28.6%1.6%1.8%39.1pp
regressedbreakthrough32.7%22.9%60.0%47.8%60.0%37.1pp
churnbreakthrough3.7%40.0%36.3pp
staybroke34.7%27.3%9.6%0.8%11.1%33.9pp
breakthroughstay49.8%45.4%74.0%77.7%75.7%32.3pp
brokeimproved15.8%32.3%3.6%28.6%4.0%28.7pp
brokestuck28.5%46.7%28.6%28.6%20.0%26.7pp
improvedstuck16.9%7.4%33.3%11.8%25.9pp
stayend_submit5.6%22.7%26.5%31.2%27.6%25.6pp
startbreakthrough73.0%72.0%96.8%85.3%96.3%24.8pp
regressedstuck7.7%2.9%10.0%21.7%20.0%18.8pp
breakthroughend_submit4.1%16.8%20.5%7.1%21.5%17.4pp
staystay57.5%48.3%62.2%64.8%59.3%16.5pp
startstuck17.0%12.0%2.1%4.2%1.2%15.8pp
startimproved10.0%16.0%1.1%10.5%2.5%14.9pp
improvedchurn4.0%13.7%17.6%13.6pp
churnend_limit7.4%20.0%12.6pp
churnregressed20.0%7.4%20.0%12.6pp
stuckend_submit2.3%1.2%11.8%4.0%10.6pp
breakthroughregressed5.2%8.2%3.9%12.5%1.9%10.6pp
churnchurn20.0%11.1%20.0%8.9pp
stuckimproved7.3%13.8%5.9%8.0%7.9pp
stuckend_limit0.8%2.1%8.0%7.2pp
stuckbroke1.9%2.3%5.9%4.0%9.1%7.2pp
regressedregressed5.7%10.0%4.3%5.7pp
churnbroke46.7%51.9%5.2pp
improvedend_limit1.0%2.3%5.9%4.9pp
regressedimproved13.5%11.4%8.7%4.8pp
improvedimproved6.5%10.3%5.9%4.4pp
regressedchurn7.7%4.3%3.4pp
churnimproved13.3%14.8%1.5pp
improvedend_submit1.5%2.9%1.4pp
regressedend_submit10.0%8.7%1.3pp
stayregressed1.2%1.7%1.6%2.4%1.5%1.2pp
improvedregressed6.0%6.9%5.9%1.0pp
breakthroughend_limit0.2%1.0%0.9%0.9%0.8pp
stayend_limit0.9%0.8%0.5%0.4pp
brokeend_limit0.2%0.4%0.2pp
brokeend_submit4.0%0.0pp
churnstuck3.7%0.0pp
regressedend_limit1.9%0.0pp
stuckregressed0.4%0.0pp

Gap label distributions — sorted by edge visit rate, labels by avg%

transitionlabelGemma (100)Qwen (100)GPT-5.5 (95)Sonnet 4.6 (95)GLM-5.2 (81)Δ (max−min)
stuckstuck (2.45×/traj)fix_bug13.4%45.5%15.4%57.1%85.7%72.3pp
revert27.8%0.0pp
debug_failure17.2%11.6%7.7%14.3%9.5pp
implement15.4%13.3%15.4%14.3%2.1pp
explore10.6%3.9%14.3%10.4pp
infra_confusion4.3%14.6%61.5%57.2pp
rewrite4.1%8.8%4.7pp
cosmetic_refactor4.7%0.9%3.8pp
execute_plan1.3%1.1%0.2pp
verify0.9%0.2%0.7pp
optimize_perf0.4%0.2%14.3%14.1pp
staystay (1.77×/traj)cosmetic_refactor33.5%30.0%3.5pp
verify16.1%10.0%6.1pp
fix_bug11.2%30.0%25.0%70.0%100.0%88.8pp
optimize_perf11.8%20.0%10.0%10.0pp
implement11.2%0.0pp
revert6.5%10.0%3.5pp
edge_case5.5%10.0%4.5pp
infra_confusion1.6%75.0%10.0%73.4pp
debug_failure1.6%0.0pp
explore0.8%0.0pp
execute_plan0.2%0.0pp
breakthroughstay (1.24×/traj)cosmetic_refactor25.1%29.4%4.3pp
implement21.1%11.8%14.3%9.3pp
fix_bug10.3%32.4%80.0%71.4%100.0%89.7pp
optimize_perf18.9%2.9%14.3%16.0pp
edge_case10.3%2.9%7.4pp
verify7.4%11.8%4.4pp
infra_confusion2.9%5.9%3.0pp
rewrite1.7%2.9%20.0%18.3pp
debug_failure1.1%0.0pp
execute_plan0.6%0.0pp
revert0.6%0.0pp
startbreakthrough (0.85×/traj)implement100.0%99.5%100.0%100.0%100.0%0.5pp
explore0.5%0.0pp
brokebreakthrough (0.79×/traj)fix_bug50.0%25.0%25.0pp
implement26.7%25.0%1.7pp
cosmetic_refactor6.7%25.0%18.3pp
rewrite6.7%25.0%18.3pp
execute_plan3.3%0.0pp
infra_confusion3.3%0.0pp
debug_failure3.3%0.0pp
staybroke (0.60×/traj)explore32.0%0.0pp
debug_failure20.0%0.0pp
implement12.0%0.0pp
fix_bug12.0%0.0pp
optimize_perf8.0%0.0pp
rewrite8.0%0.0pp
infra_confusion8.0%0.0pp
stayend_submit (0.58×/traj)verify99.1%100.0%100.0%57.1%50.0%50.0pp
infra_confusion0.9%42.9%50.0%49.1pp
brokestuck (0.56×/traj)explore21.3%15.8%5.5pp
debug_failure23.0%10.5%12.5pp
implement9.8%26.3%16.5pp
fix_bug13.1%10.5%2.6pp
rewrite8.2%26.3%18.1pp
revert14.8%0.0pp
infra_confusion1.6%10.5%8.9pp
execute_plan3.3%0.0pp
cosmetic_refactor3.3%0.0pp
optimize_perf1.6%0.0pp
breakthroughbroke (0.51×/traj)debug_failure24.1%33.3%9.2pp
fix_bug20.7%0.0pp
infra_confusion17.2%16.7%0.5pp
rewrite10.3%16.7%6.4pp
explore6.9%16.7%9.8pp
implement6.9%16.7%9.8pp
optimize_perf6.9%0.0pp
edge_case3.4%0.0pp
revert3.4%0.0pp
improvedbroke (0.39×/traj)explore37.0%21.7%15.3pp
fix_bug18.5%26.1%50.0%31.5pp
rewrite14.8%26.1%11.3pp
implement11.1%13.0%1.9pp
debug_failure3.7%13.0%50.0%46.3pp
cosmetic_refactor7.4%0.0pp
execute_plan3.7%0.0pp
infra_confusion3.7%0.0pp
brokeimproved (0.33×/traj)fix_bug14.3%42.9%28.6pp
rewrite28.6%19.0%9.6pp
implement9.5%19.0%9.5pp
execute_plan4.8%9.5%4.7pp
revert14.3%0.0pp
explore9.5%4.8%4.7pp
cosmetic_refactor9.5%100.0%90.5pp
debug_failure9.5%4.8%4.7pp
stuckimproved (0.29×/traj)fix_bug25.3%63.6%50.0%38.3pp
implement24.0%10.9%13.1pp
rewrite17.3%14.5%50.0%35.5pp
infra_confusion5.3%7.3%50.0%44.7pp
execute_plan8.0%50.0%42.0pp
cosmetic_refactor8.0%0.0pp
revert6.7%0.0pp
explore4.0%0.0pp
debug_failure1.3%1.8%0.5pp
optimize_perf1.8%0.0pp
stuckbreakthrough (0.29×/traj)fix_bug51.8%76.4%33.3%81.8%69.2%48.5pp
infra_confusion5.9%12.5%48.1%9.1%26.9%42.2pp
implement16.5%1.4%3.7%15.1pp
cosmetic_refactor7.1%2.8%3.7%3.8%4.3pp
execute_plan8.2%7.4%0.8pp
rewrite3.5%6.9%9.1%5.6pp
optimize_perf3.5%3.7%0.2pp
debug_failure3.5%0.0pp
breakthroughend_submit (0.23×/traj)verify100.0%99.6%100.0%100.0%100.0%0.4pp
fix_bug0.4%0.0pp
breakthroughregressed (0.13×/traj)fix_bug16.7%55.6%38.9pp
implement50.0%11.1%38.9pp
optimize_perf11.1%0.0pp
cosmetic_refactor16.7%0.0pp
rewrite11.1%0.0pp
infra_confusion16.7%0.0pp
debug_failure11.1%0.0pp
improvedstuck (0.10×/traj)implement17.9%29.0%11.1pp
cosmetic_refactor16.4%12.9%3.5pp
fix_bug16.4%3.2%75.0%71.8pp
debug_failure9.0%22.6%13.6pp
rewrite6.0%16.1%10.1pp
revert10.4%0.0pp
explore6.0%9.7%3.7pp
optimize_perf3.0%3.2%100.0%25.0%97.0pp
verify7.5%0.0pp
infra_confusion6.0%3.2%2.8pp
execute_plan1.5%0.0pp
improvedbreakthrough (0.10×/traj)fix_bug44.4%88.0%75.0%42.9%75.0%45.1pp
rewrite11.1%8.0%42.9%34.9pp
implement11.1%4.8%25.0%20.2pp
debug_failure11.1%12.5%4.8%7.7pp
optimize_perf11.1%4.8%6.3pp
infra_confusion11.1%12.5%1.4pp
execute_plan4.0%0.0pp
regressedbreakthrough (0.09×/traj)fix_bug66.7%66.7%0.0pp
optimize_perf16.7%0.0pp
implement16.7%0.0pp
debug_failure33.3%0.0pp
regressedbroke (0.08×/traj)rewrite50.0%33.3%16.7pp
optimize_perf33.3%0.0pp
implement25.0%0.0pp
fix_bug25.0%0.0pp
infra_confusion33.3%0.0pp
startimproved (0.08×/traj)implement88.9%95.3%100.0%100.0%100.0%11.1pp
explore7.4%4.7%2.7pp
infra_confusion3.7%0.0pp
startstuck (0.07×/traj)implement60.9%81.6%96.6%75.0%100.0%39.1pp
explore39.1%18.4%3.4%25.0%35.7pp
improvedchurn (0.07×/traj)rewrite66.7%50.0%16.7pp
fix_bug33.3%40.0%6.7pp
implement10.0%0.0pp
stuckbroke (0.07×/traj)explore26.9%12.5%14.4pp
debug_failure11.5%50.0%38.5pp
fix_bug23.1%0.0pp
rewrite15.4%25.0%9.6pp
implement15.4%12.5%2.9pp
revert3.8%0.0pp
infra_confusion3.8%0.0pp
stuckend_submit (0.07×/traj)verify100.0%0.0pp
infra_confusion100.0%0.0pp
improvedimproved (0.06×/traj)fix_bug40.0%71.4%66.7%31.4pp
implement40.0%0.0pp
rewrite10.0%28.6%18.6pp
optimize_perf10.0%33.3%23.3pp
improvedregressed (0.05×/traj)fix_bug40.0%36.4%3.6pp
rewrite20.0%27.3%50.0%30.0pp
implement40.0%25.0%15.0pp
optimize_perf27.3%0.0pp
revert9.1%0.0pp
debug_failure25.0%0.0pp
stayregressed (0.05×/traj)optimize_perf33.3%100.0%66.7pp
rewrite33.3%0.0pp
cosmetic_refactor33.3%0.0pp
churnbroke (0.04×/traj)rewrite100.0%50.0%50.0pp
explore16.7%0.0pp
implement16.7%0.0pp
debug_failure16.7%0.0pp
stuckend_limit (0.04×/traj)verify81.0%43.2%33.3%47.7pp
explore3.2%29.7%66.7%63.5pp
debug_failure12.7%16.2%3.5pp
infra_confusion8.1%0.0pp
implement1.6%0.0pp
fix_bug2.7%0.0pp
rewrite1.6%0.0pp
regressedimproved (0.03×/traj)fix_bug33.3%50.0%50.0%16.7pp
rewrite16.7%50.0%33.3pp
implement33.3%0.0pp
optimize_perf50.0%0.0pp
debug_failure16.7%0.0pp
regressedstuck (0.03×/traj)debug_failure22.2%50.0%27.8pp
optimize_perf22.2%0.0pp
cosmetic_refactor22.2%0.0pp
fix_bug11.1%50.0%38.9pp
implement11.1%0.0pp
rewrite11.1%0.0pp
stayend_limit (0.02×/traj)verify93.3%0.0pp
infra_confusion3.3%100.0%96.7pp
debug_failure3.3%0.0pp
improvedend_submit (0.02×/traj)verify100.0%100.0%0.0pp
improvedend_limit (0.01×/traj)verify61.5%38.5%9.1%52.4pp
debug_failure30.8%34.6%27.3%7.3pp
explore7.7%23.1%63.6%55.9pp
fix_bug3.8%0.0pp
churnchurn (0.01×/traj)rewrite50.0%43.8%6.2pp
fix_bug37.5%0.0pp
implement25.0%12.5%12.5pp
debug_failure25.0%6.2%18.8pp
churnregressed (0.01×/traj)implement50.0%0.0pp
fix_bug50.0%0.0pp
churnimproved (0.01×/traj)fix_bug100.0%66.7%33.3pp
implement33.3%0.0pp
breakthroughend_limit (0.01×/traj)verify100.0%100.0%75.0%25.0pp
fix_bug25.0%0.0pp
regressedchurn (0.01×/traj)implement25.0%0.0pp
edge_case25.0%0.0pp
fix_bug25.0%0.0pp
rewrite25.0%0.0pp
stuckregressed (0.01×/traj)implement57.1%0.0pp
rewrite28.6%0.0pp
fix_bug14.3%0.0pp
regressedregressed (0.01×/traj)fix_bug100.0%0.0pp
churnbreakthrough (0.01×/traj)fix_bug100.0%0.0pp
churnend_limit (0.01×/traj)explore33.3%0.0pp
debug_failure33.3%0.0pp
verify33.3%0.0pp
brokeend_limit (0.00×/traj)verify100.0%0.0pp
infra_confusion50.0%0.0pp
debug_failure50.0%0.0pp
churnstuck (0.00×/traj)implement40.0%0.0pp
fix_bug40.0%0.0pp
rewrite20.0%0.0pp
regressedend_limit (0.00×/traj)optimize_perf50.0%0.0pp
verify50.0%0.0pp

Python

highest in row   lowest in row  Sorted by Δ (max − min) descending.

Node visit rates — sorted by Δ

stateGemma (100)Qwen (97)GPT-5.5 (93)Sonnet 4.6 (91)GLM-5.2 (75)Δ
start1.01.01.01.01.00.00
improved0.40.30.00.10.00.38
churn0.00.20.18
stuck1.00.40.00.10.96
breakthrough1.00.91.00.91.00.10
stay2.80.10.00.10.02.76
regressed0.10.10.00.14
broke0.20.10.00.18
end_submit0.80.81.00.91.00.19
end_limit0.20.20.10.00.18

Edge visit rates (count / n_problems) — sorted by Δ

transitionGemma (100)Qwen (97)GPT-5.5 (93)Sonnet 4.6 (91)GLM-5.2 (75)Δ (max−min)
staystay2.2×0.0×0.0×0.0×0.0×2.24×
breakthroughend_submit0.4×0.8×1.0×0.9×1.0×0.58×
breakthroughstay0.5×0.1×0.0×0.0×0.0×0.53×
stuckstuck0.5×0.2×0.0×0.48×
stayend_submit0.3×0.1×0.0×0.0×0.0×0.32×
startbreakthrough0.7×0.7×1.0×0.9×1.0×0.26×
stuckimproved0.2×0.1×0.0×0.0×0.16×
startstuck0.2×0.1×0.0×0.0×0.15×
improvedstuck0.1×0.1×0.0×0.0×0.14×
startimproved0.1×0.1×0.0×0.1×0.0×0.13×
stuckbreakthrough0.1×0.1×0.0×0.0×0.11×
churnchurn0.0×0.1×0.09×
stayend_limit0.1×0.0×0.09×
brokestuck0.1×0.0×0.07×
improvedregressed0.1×0.1×0.0×0.06×
regressedstuck0.1×0.0×0.05×
improvedbreakthrough0.1×0.1×0.0×0.1×0.04×
stuckbroke0.1×0.0×0.04×
improvedchurn0.0×0.0×0.03×
improvedimproved0.0×0.0×0.0×0.03×
regressedimproved0.0×0.0×0.0×0.03×
improvedend_limit0.0×0.1×0.0×0.03×
regressedbreakthrough0.0×0.0×0.02×
stuckend_limit0.0×0.1×0.0×0.02×
regressedchurn0.0×0.0×0.01×
stayregressed0.0×0.0×0.01×
stuckregressed0.0×0.0×0.01×
improvedbroke0.0×0.0×0.0×0.01×
brokeimproved0.0×0.0×0.01×
improvedend_submit0.0×0.0×0.00×
breakthroughregressed0.0×0.0×0.00×
breakthroughbroke0.0×0.0×0.00×
churnbreakthrough0.0×0.0×0.00×
breakthroughend_limit0.0×0.00×
brokebreakthrough0.1×0.00×
brokeend_limit0.0×0.00×
churnbroke0.0×0.00×
churnend_limit0.0×0.00×
churnimproved0.0×0.00×
churnstuck0.1×0.00×
regressedbroke0.0×0.00×
regressedend_limit0.0×0.00×
regressedregressed0.0×0.00×
staybroke0.1×0.00×
stuckchurn0.0×0.00×
stuckend_submit0.1×0.00×

Transition probabilities P(tgt | src) — sorted by Δ

transitionGemma (100)Qwen (97)GPT-5.5 (93)Sonnet 4.6 (91)GLM-5.2 (75)Δ (max−min)
improvedend_submit2.6%100.0%97.4pp
regressedimproved26.7%27.3%100.0%73.3pp
breakthroughend_submit40.8%85.1%97.8%96.4%97.3%57.0pp
stuckbreakthrough12.1%14.3%66.7%11.1%55.6pp
stayend_submit11.8%53.8%66.7%42.9%50.0%54.9pp
breakthroughstay55.1%10.3%2.2%3.6%1.4%53.7pp
staystay80.6%30.8%33.3%57.1%50.0%49.8pp
improvedstuck38.5%22.6%50.0%8.3%41.7pp
brokeimproved21.1%60.0%38.9pp
improvedbreakthrough12.8%16.1%50.0%41.7%37.2pp
regressedstuck40.0%9.1%30.9pp
stuckend_limit4.0%11.9%33.3%29.3pp
startbreakthrough73.0%74.2%96.8%85.7%98.7%25.7pp
churnchurn25.0%47.6%22.6pp
stuckimproved17.2%21.4%33.3%11.1%22.2pp
regressedbreakthrough6.7%27.3%20.6pp
churnbreakthrough25.0%4.8%20.2pp
startstuck17.0%11.3%2.2%4.4%14.8pp
improvedend_limit10.3%19.4%25.0%14.7pp
stuckstuck52.5%38.1%44.4%14.4pp
regressedchurn13.3%27.3%14.0pp
startimproved10.0%14.4%1.1%9.9%1.3%13.3pp
improvedregressed17.9%19.4%8.3%11.1pp
improvedchurn2.6%12.9%10.3pp
brokestuck47.4%40.0%7.4pp
improvedimproved10.3%3.2%8.3%7.1pp
stayregressed0.7%7.7%7.0pp
stayend_limit3.6%7.7%4.1pp
improvedbroke5.1%6.5%8.3%3.2pp
stuckbroke5.1%2.4%2.7pp
stuckregressed2.0%2.4%0.4pp
breakthroughregressed3.1%3.4%0.3pp
breakthroughbroke1.0%1.1%0.1pp
breakthroughend_limit1.4%0.0pp
brokebreakthrough31.6%0.0pp
brokeend_limit100.0%0.0pp
churnbroke50.0%0.0pp
churnend_limit19.0%0.0pp
churnimproved4.8%0.0pp
churnstuck23.8%0.0pp
regressedbroke9.1%0.0pp
regressedend_limit6.7%0.0pp
regressedregressed6.7%0.0pp
staybroke3.2%0.0pp
stuckchurn9.5%0.0pp
stuckend_submit7.1%0.0pp

Gap label distributions — sorted by edge visit rate, labels by avg%

transitionlabelGemma (100)Qwen (97)GPT-5.5 (93)Sonnet 4.6 (91)GLM-5.2 (75)Δ (max−min)
startbreakthrough (0.86×/traj)implement100.0%100.0%100.0%100.0%100.0%0.0pp
breakthroughend_submit (0.80×/traj)verify100.0%100.0%100.0%100.0%100.0%0.0pp
staystay (0.47×/traj)cosmetic_refactor35.1%0.0pp
optimize_perf17.3%50.0%25.0%32.7pp
implement15.6%0.0pp
fix_bug8.9%50.0%100.0%25.0%100.0%91.1pp
verify8.9%0.0pp
revert6.2%25.0%18.8pp
edge_case2.7%0.0pp
infra_confusion1.3%25.0%23.7pp
explore1.8%0.0pp
debug_failure1.8%0.0pp
execute_plan0.4%0.0pp
stuckstuck (0.15×/traj)explore25.0%18.8%25.0%6.2pp
fix_bug7.7%37.5%25.0%29.8pp
debug_failure9.6%25.0%15.4pp
revert15.4%0.0pp
implement11.5%6.2%25.0%18.8pp
rewrite13.5%6.2%7.3pp
cosmetic_refactor9.6%0.0pp
optimize_perf1.9%6.2%25.0%23.1pp
verify5.8%0.0pp
breakthroughstay (0.14×/traj)optimize_perf31.5%11.1%33.3%22.2pp
implement22.2%22.2%33.3%11.1pp
cosmetic_refactor24.1%11.1%13.0pp
fix_bug11.1%22.2%50.0%33.3%100.0%88.9pp
rewrite3.7%11.1%50.0%46.3pp
verify1.9%22.2%20.3pp
edge_case3.7%0.0pp
execute_plan1.9%0.0pp
stayend_submit (0.09×/traj)verify96.9%100.0%100.0%66.7%33.3pp
infra_confusion3.1%33.3%100.0%96.9pp
startimproved (0.07×/traj)implement90.0%100.0%100.0%100.0%100.0%10.0pp
explore10.0%0.0pp
startstuck (0.07×/traj)explore76.5%18.2%50.0%58.3pp
implement23.5%81.8%100.0%50.0%76.5pp
stuckimproved (0.06×/traj)fix_bug17.6%66.7%100.0%82.4pp
execute_plan29.4%100.0%70.6pp
implement29.4%11.1%18.3pp
rewrite17.6%22.2%4.6pp
explore5.9%0.0pp
improvedstuck (0.05×/traj)cosmetic_refactor33.3%28.6%4.7pp
implement26.7%14.3%12.4pp
optimize_perf6.7%100.0%100.0%93.3pp
explore6.7%14.3%7.6pp
fix_bug13.3%0.0pp
rewrite28.6%0.0pp
debug_failure6.7%14.3%7.6pp
revert6.7%0.0pp
stuckbreakthrough (0.04×/traj)fix_bug33.3%66.7%33.4pp
execute_plan25.0%50.0%25.0pp
optimize_perf25.0%50.0%25.0pp
implement16.7%0.0pp
rewrite33.3%0.0pp
infra_confusion100.0%0.0pp
improvedbreakthrough (0.03×/traj)fix_bug40.0%100.0%40.0%60.0pp
rewrite20.0%40.0%20.0pp
optimize_perf20.0%20.0%0.0pp
implement20.0%0.0pp
infra_confusion100.0%0.0pp
improvedregressed (0.03×/traj)fix_bug57.1%16.7%40.4pp
rewrite28.6%33.3%4.7pp
optimize_perf50.0%0.0pp
implement14.3%100.0%85.7pp
improvedend_limit (0.03×/traj)debug_failure25.0%50.0%33.3%25.0pp
explore33.3%66.7%33.4pp
verify75.0%16.7%58.3pp
stuckend_limit (0.02×/traj)verify100.0%60.0%33.3%66.7pp
explore20.0%66.7%46.7pp
debug_failure20.0%0.0pp
churnchurn (0.02×/traj)rewrite60.0%0.0pp
fix_bug20.0%0.0pp
debug_failure100.0%10.0%90.0pp
implement10.0%0.0pp
brokestuck (0.02×/traj)explore55.6%0.0pp
debug_failure22.2%100.0%77.8pp
optimize_perf11.1%0.0pp
execute_plan11.1%0.0pp
stayend_limit (0.02×/traj)verify90.9%0.0pp
debug_failure9.1%100.0%90.9pp
staybroke (0.02×/traj)explore77.8%0.0pp
rewrite11.1%0.0pp
infra_confusion11.1%0.0pp
regressedimproved (0.02×/traj)fix_bug25.0%66.7%41.7pp
rewrite25.0%33.3%8.3pp
implement50.0%0.0pp
optimize_perf100.0%0.0pp
brokeimproved (0.01×/traj)fix_bug25.0%66.7%41.7pp
rewrite50.0%0.0pp
implement25.0%0.0pp
execute_plan33.3%0.0pp
regressedstuck (0.01×/traj)fix_bug16.7%100.0%83.3pp
optimize_perf16.7%0.0pp
implement16.7%0.0pp
cosmetic_refactor16.7%0.0pp
rewrite16.7%0.0pp
debug_failure16.7%0.0pp
stuckend_submit (0.01×/traj)verify100.0%0.0pp
improvedimproved (0.01×/traj)fix_bug75.0%100.0%25.0pp
optimize_perf25.0%100.0%75.0pp
breakthroughregressed (0.01×/traj)implement100.0%0.0pp
optimize_perf33.3%0.0pp
fix_bug33.3%0.0pp
rewrite33.3%0.0pp
stuckbroke (0.01×/traj)explore60.0%0.0pp
rewrite20.0%0.0pp
implement100.0%0.0pp
debug_failure20.0%0.0pp
brokebreakthrough (0.01×/traj)rewrite33.3%0.0pp
execute_plan16.7%0.0pp
implement16.7%0.0pp
fix_bug16.7%0.0pp
cosmetic_refactor16.7%0.0pp
improvedbroke (0.01×/traj)explore100.0%50.0%50.0pp
implement50.0%0.0pp
debug_failure100.0%0.0pp
churnstuck (0.01×/traj)implement40.0%0.0pp
fix_bug40.0%0.0pp
rewrite20.0%0.0pp
improvedchurn (0.01×/traj)rewrite100.0%0.0pp
fix_bug100.0%0.0pp
regressedchurn (0.01×/traj)fix_bug50.0%33.3%16.7pp
optimize_perf33.3%0.0pp
edge_case50.0%0.0pp
rewrite33.3%0.0pp
churnend_limit (0.01×/traj)explore50.0%0.0pp
verify50.0%0.0pp
stuckchurn (0.01×/traj)fix_bug50.0%0.0pp
rewrite25.0%0.0pp
implement25.0%0.0pp
regressedbreakthrough (0.01×/traj)fix_bug100.0%33.3%66.7pp
optimize_perf33.3%0.0pp
implement33.3%0.0pp
stuckregressed (0.01×/traj)implement100.0%0.0pp
fix_bug100.0%0.0pp
stayregressed (0.01×/traj)optimize_perf50.0%100.0%50.0pp
cosmetic_refactor50.0%0.0pp
improvedend_submit (0.00×/traj)verify100.0%100.0%0.0pp
churnbreakthrough (0.00×/traj)revert100.0%0.0pp
fix_bug100.0%0.0pp
breakthroughbroke (0.00×/traj)explore100.0%0.0pp
implement100.0%0.0pp
churnbroke (0.00×/traj)rewrite100.0%0.0pp
breakthroughend_limit (0.00×/traj)verify100.0%0.0pp
brokeend_limit (0.00×/traj)explore100.0%0.0pp
churnimproved (0.00×/traj)fix_bug100.0%0.0pp
regressedbroke (0.00×/traj)optimize_perf100.0%0.0pp
regressedregressed (0.00×/traj)rewrite100.0%0.0pp
regressedend_limit (0.00×/traj)verify100.0%0.0pp

Java

highest in row   lowest in row  Sorted by Δ (max − min) descending.

Node visit rates — sorted by Δ

stateGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ
start1.01.01.01.01.00.00
improved0.30.50.00.10.00.42
churn0.00.10.11
stuck0.90.50.00.90
breakthrough0.90.81.01.01.00.16
stay0.80.10.00.10.00.78
regressed0.10.10.00.08
broke0.10.10.02
end_submit0.80.81.01.01.00.22
end_limit0.20.20.00.00.21

Edge visit rates (count / n_problems) — sorted by Δ

transitionGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ (max−min)
breakthroughend_submit0.5×0.7×1.0×0.9×1.0×0.53×
stuckstuck0.5×0.2×0.0×0.51×
staystay0.4×0.0×0.0×0.0×0.42×
breakthroughstay0.4×0.0×0.0×0.0×0.0×0.35×
stayend_submit0.3×0.0×0.0×0.0×0.0×0.31×
startbreakthrough0.7×0.7×1.0×0.9×1.0×0.31×
startstuck0.2×0.2×0.0×0.15×
stuckimproved0.1×0.1×0.0×0.13×
startimproved0.1×0.2×0.0×0.1×0.0×0.13×
improvedend_limit0.0×0.1×0.0×0.0×0.07×
stuckbreakthrough0.1×0.1×0.0×0.07×
improvedbreakthrough0.0×0.1×0.0×0.1×0.0×0.06×
brokeimproved0.0×0.1×0.05×
improvedbroke0.0×0.1×0.05×
improvedstuck0.2×0.1×0.05×
breakthroughregressed0.0×0.0×0.03×
churnbroke0.0×0.0×0.03×
churnchurn0.0×0.0×0.03×
stuckregressed0.0×0.0×0.03×
brokestuck0.1×0.0×0.03×
improvedchurn0.0×0.0×0.02×
improvedregressed0.0×0.0×0.0×0.02×
stuckchurn0.0×0.0×0.02×
breakthroughbroke0.0×0.0×0.02×
stayend_limit0.0×0.0×0.02×
stuckbroke0.0×0.0×0.02×
improvedimproved0.0×0.0×0.0×0.02×
brokeend_limit0.0×0.0×0.01×
churnend_limit0.0×0.0×0.01×
regressedbreakthrough0.0×0.0×0.01×
regressedimproved0.0×0.0×0.01×
stuckend_limit0.1×0.1×0.00×
improvedend_submit0.0×0.0×0.0×0.0×0.0×0.00×
regressedend_limit0.0×0.0×0.00×
regressedstuck0.0×0.0×0.00×
breakthroughend_limit0.0×0.00×
brokebreakthrough0.0×0.00×
churnbreakthrough0.0×0.00×
churnimproved0.0×0.00×
regressedbroke0.0×0.00×
regressedchurn0.0×0.00×
regressedregressed0.0×0.00×
staybroke0.0×0.00×
stuckend_submit0.0×0.00×

Transition probabilities P(tgt | src) — sorted by Δ

transitionGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ (max−min)
regressedend_limit16.7%100.0%83.3pp
improvedbreakthrough6.2%15.9%75.0%58.3%33.3%68.8pp
stayend_submit40.5%60.0%50.0%40.0%100.0%60.0pp
breakthroughend_submit52.3%88.8%98.9%97.7%98.7%46.6pp
breakthroughstay41.9%5.0%1.1%2.3%1.3%40.8pp
staystay54.4%20.0%50.0%60.0%40.0pp
brokeimproved30.8%64.3%33.5pp
improvedend_submit3.1%2.3%25.0%8.3%33.3%31.0pp
startbreakthrough73.0%65.6%95.7%86.7%96.2%30.6pp
improvedstuck53.1%27.3%25.8pp
stuckstuck56.4%31.2%50.0%25.2pp
brokestuck46.2%21.4%24.8pp
improvedend_limit12.5%18.2%16.7%33.3%20.8pp
churnend_limit33.3%15.4%17.9pp
stuckbreakthrough8.5%12.5%25.0%16.5pp
stayend_limit3.8%20.0%16.2pp
startstuck17.0%17.7%2.2%15.5pp
stuckimproved14.9%29.2%25.0%14.3pp
startimproved10.0%16.7%4.3%11.1%3.8%12.9pp
improvedbroke9.4%18.2%8.8pp
stuckend_limit8.5%16.7%8.2pp
brokeend_limit7.7%14.3%6.6pp
regressedstuck16.7%11.1%5.6pp
regressedbreakthrough16.7%22.2%5.5pp
regressedimproved16.7%22.2%5.5pp
improvedregressed3.1%6.8%8.3%5.2pp
stuckchurn1.1%6.2%5.1pp
improvedimproved9.4%4.5%8.3%4.9pp
breakthroughregressed1.2%5.0%3.8pp
improvedchurn3.1%6.8%3.7pp
churnbroke33.3%30.8%2.5pp
churnchurn33.3%30.8%2.5pp
breakthroughbroke3.5%1.2%2.3pp
stuckregressed4.3%2.1%2.2pp
stuckbroke3.2%2.1%1.1pp
breakthroughend_limit1.2%0.0pp
brokebreakthrough15.4%0.0pp
churnbreakthrough15.4%0.0pp
churnimproved7.7%0.0pp
regressedbroke33.3%0.0pp
regressedchurn33.3%0.0pp
regressedregressed11.1%0.0pp
staybroke1.3%0.0pp
stuckend_submit3.2%0.0pp

Gap label distributions — sorted by edge visit rate, labels by avg%

transitionlabelGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ (max−min)
startbreakthrough (0.83×/traj)implement100.0%98.4%100.0%100.0%100.0%1.6pp
explore1.6%0.0pp
breakthroughend_submit (0.81×/traj)verify100.0%100.0%100.0%100.0%100.0%0.0pp
stuckstuck (0.14×/traj)explore32.1%0.0pp
implement20.8%26.7%5.9pp
fix_bug11.3%26.7%100.0%88.7pp
debug_failure13.2%33.3%20.1pp
rewrite17.0%13.3%3.7pp
optimize_perf3.8%0.0pp
infra_confusion1.9%0.0pp
staystay (0.10×/traj)cosmetic_refactor30.2%0.0pp
verify23.3%0.0pp
fix_bug11.6%100.0%100.0%88.4pp
optimize_perf14.0%0.0pp
edge_case11.6%0.0pp
revert4.7%0.0pp
implement4.7%0.0pp
infra_confusion100.0%0.0pp
startimproved (0.09×/traj)implement80.0%87.5%100.0%100.0%100.0%20.0pp
explore10.0%12.5%2.5pp
infra_confusion10.0%0.0pp
breakthroughstay (0.09×/traj)cosmetic_refactor38.9%0.0pp
fix_bug11.1%75.0%100.0%100.0%100.0%88.9pp
optimize_perf19.4%0.0pp
implement16.7%0.0pp
verify8.3%25.0%16.7pp
edge_case5.6%0.0pp
stayend_submit (0.08×/traj)verify100.0%100.0%100.0%100.0%100.0%0.0pp
startstuck (0.07×/traj)implement47.1%76.5%50.0%29.4pp
explore52.9%23.5%50.0%29.4pp
stuckimproved (0.06×/traj)fix_bug42.9%50.0%7.1pp
rewrite28.6%35.7%100.0%71.4pp
implement28.6%14.3%14.3pp
improvedstuck (0.06×/traj)implement23.5%41.7%18.2pp
debug_failure17.6%16.7%0.9pp
infra_confusion17.6%8.3%9.3pp
explore5.9%8.3%2.4pp
optimize_perf5.9%8.3%2.4pp
fix_bug11.8%0.0pp
cosmetic_refactor5.9%8.3%2.4pp
rewrite5.9%8.3%2.4pp
execute_plan5.9%0.0pp
improvedbreakthrough (0.04×/traj)fix_bug100.0%85.7%66.7%57.1%42.9pp
rewrite14.3%42.9%28.6pp
implement100.0%0.0pp
debug_failure33.3%0.0pp
stuckend_limit (0.03×/traj)verify100.0%37.5%62.5pp
explore25.0%0.0pp
debug_failure25.0%0.0pp
fix_bug12.5%0.0pp
improvedend_limit (0.03×/traj)verify40.0%50.0%50.0%10.0pp
explore20.0%12.5%50.0%100.0%87.5pp
debug_failure40.0%25.0%15.0pp
fix_bug12.5%0.0pp
stuckbreakthrough (0.03×/traj)fix_bug50.0%83.3%100.0%50.0pp
implement37.5%0.0pp
rewrite16.7%0.0pp
execute_plan12.5%0.0pp
brokeimproved (0.03×/traj)rewrite50.0%22.2%27.8pp
implement33.3%0.0pp
fix_bug33.3%0.0pp
explore50.0%0.0pp
execute_plan11.1%0.0pp
improvedbroke (0.02×/traj)explore100.0%25.0%75.0pp
implement25.0%0.0pp
rewrite25.0%0.0pp
fix_bug12.5%0.0pp
debug_failure12.5%0.0pp
brokestuck (0.02×/traj)rewrite33.3%33.3%0.0pp
implement16.7%66.7%50.0pp
debug_failure33.3%0.0pp
explore16.7%0.0pp
improvedimproved (0.01×/traj)implement100.0%0.0pp
fix_bug100.0%100.0%0.0pp
improvedend_submit (0.01×/traj)verify100.0%100.0%100.0%100.0%0.0pp
improvedregressed (0.01×/traj)revert33.3%0.0pp
implement100.0%0.0pp
fix_bug33.3%0.0pp
rewrite33.3%0.0pp
debug_failure100.0%0.0pp
churnbroke (0.01×/traj)rewrite100.0%50.0%50.0pp
explore25.0%0.0pp
debug_failure25.0%0.0pp
breakthroughregressed (0.01×/traj)fix_bug50.0%0.0pp
implement25.0%0.0pp
cosmetic_refactor100.0%0.0pp
debug_failure25.0%0.0pp
churnchurn (0.01×/traj)rewrite100.0%25.0%75.0pp
fix_bug50.0%0.0pp
implement25.0%0.0pp
stuckregressed (0.01×/traj)rewrite50.0%0.0pp
fix_bug25.0%100.0%75.0pp
implement25.0%0.0pp
improvedchurn (0.01×/traj)rewrite100.0%33.3%66.7pp
fix_bug66.7%0.0pp
stuckchurn (0.01×/traj)debug_failure100.0%66.7%33.3pp
rewrite33.3%0.0pp
breakthroughbroke (0.01×/traj)debug_failure33.3%100.0%66.7pp
rewrite33.3%0.0pp
implement33.3%0.0pp
stayend_limit (0.01×/traj)verify100.0%100.0%0.0pp
stuckbroke (0.01×/traj)implement66.7%0.0pp
rewrite33.3%0.0pp
debug_failure100.0%0.0pp
regressedchurn (0.01×/traj)rewrite66.7%0.0pp
fix_bug33.3%0.0pp
churnend_limit (0.01×/traj)debug_failure100.0%0.0pp
verify100.0%0.0pp
regressedimproved (0.01×/traj)rewrite100.0%0.0pp
fix_bug100.0%0.0pp
regressedbreakthrough (0.01×/traj)fix_bug100.0%100.0%0.0pp
brokeend_limit (0.01×/traj)infra_confusion50.0%0.0pp
debug_failure50.0%0.0pp
verify100.0%0.0pp
stuckend_submit (0.01×/traj)verify100.0%0.0pp
regressedend_limit (0.00×/traj)optimize_perf100.0%0.0pp
debug_failure100.0%0.0pp
churnbreakthrough (0.00×/traj)fix_bug100.0%0.0pp
regressedstuck (0.00×/traj)optimize_perf100.0%0.0pp
debug_failure100.0%0.0pp
regressedbroke (0.00×/traj)rewrite100.0%0.0pp
brokebreakthrough (0.00×/traj)implement50.0%0.0pp
fix_bug50.0%0.0pp
churnimproved (0.00×/traj)implement100.0%0.0pp
regressedregressed (0.00×/traj)fix_bug100.0%0.0pp
staybroke (0.00×/traj)debug_failure100.0%0.0pp
breakthroughend_limit (0.00×/traj)verify100.0%0.0pp

Rust

highest in row   lowest in row  Sorted by Δ (max − min) descending.

Node visit rates — sorted by Δ

stateGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ
start1.01.01.01.01.00.00
improved0.30.30.00.10.00.29
churn0.10.00.03
stuck1.10.60.10.11.09
breakthrough0.90.91.01.01.00.12
stay2.40.10.00.02.34
regressed0.10.00.03
broke0.20.10.12
end_submit0.80.81.01.01.00.22
end_limit0.20.20.00.00.00.22

Edge visit rates (count / n_problems) — sorted by Δ

transitionGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ (max−min)
staystay1.8×0.0×0.0×0.0×1.79×
stuckstuck0.7×0.3×0.0×0.67×
breakthroughend_submit0.3×0.7×1.0×0.9×0.9×0.64×
breakthroughstay0.6×0.1×0.0×0.0×0.55×
stayend_submit0.4×0.1×0.0×0.0×0.40×
startbreakthrough0.7×0.6×1.0×0.9×0.9×0.37×
startstuck0.2×0.3×0.0×0.1×0.25×
improvedstuck0.1×0.0×0.0×0.14×
stuckbreakthrough0.1×0.2×0.0×0.0×0.13×
startimproved0.1×0.1×0.0×0.1×0.0×0.10×
stayend_limit0.1×0.0×0.09×
stuckimproved0.2×0.1×0.08×
improvedend_limit0.0×0.1×0.0×0.0×0.07×
brokebreakthrough0.1×0.0×0.07×
stuckend_limit0.1×0.1×0.0×0.07×
improvedbreakthrough0.0×0.1×0.0×0.0×0.0×0.06×
brokestuck0.1×0.0×0.05×
improvedbroke0.1×0.0×0.05×
breakthroughbroke0.0×0.0×0.03×
improvedimproved0.0×0.0×0.0×0.0×0.02×
stuckbroke0.0×0.0×0.02×
improvedchurn0.0×0.0×0.01×
churnchurn0.0×0.0×0.01×
improvedregressed0.0×0.0×0.01×
breakthroughend_limit0.0×0.0×0.01×
improvedend_submit0.0×0.0×0.0×0.00×
brokeimproved0.0×0.0×0.00×
stuckregressed0.0×0.0×0.00×
churnbroke0.0×0.00×
churnend_limit0.0×0.00×
churnimproved0.0×0.00×
churnregressed0.0×0.00×
regressedbreakthrough0.0×0.00×
regressedbroke0.0×0.00×
regressedchurn0.0×0.00×
regressedend_limit0.0×0.00×
regressedimproved0.0×0.00×
regressedstuck0.0×0.00×
staybroke0.1×0.00×
stuckchurn0.0×0.00×
stuckend_submit0.0×0.00×

Transition probabilities P(tgt | src) — sorted by Δ

transitionGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ (max−min)
stuckbreakthrough9.6%25.8%80.0%75.0%70.4pp
breakthroughend_submit34.7%86.6%98.9%98.8%97.4%64.2pp
breakthroughstay58.9%12.2%1.1%1.2%57.8pp
staystay76.3%23.1%50.0%66.7%53.2pp
stayend_submit17.4%69.2%50.0%33.3%51.8pp
improvedend_limit6.1%28.6%25.0%50.0%43.9pp
stuckstuck59.6%43.5%20.0%39.6pp
startbreakthrough73.0%59.4%96.8%90.0%92.3%37.4pp
improvedstuck45.5%14.3%16.7%31.2pp
improvedbreakthrough3.0%25.0%25.0%16.7%33.3%30.3pp
improvedimproved9.1%7.1%25.0%33.3%26.2pp
startstuck21.0%28.1%3.3%5.1%24.8pp
brokeimproved18.2%40.0%21.8pp
brokebreakthrough40.9%20.0%20.9pp
stuckend_limit7.0%9.7%25.0%18.0pp
improvedend_submit25.0%16.7%33.3%16.6pp
improvedbroke27.3%14.3%13.0pp
startimproved6.0%12.5%3.2%6.7%2.6%9.9pp
improvedchurn3.0%7.1%4.1pp
stayend_limit4.2%7.7%3.5pp
breakthroughbroke4.2%1.2%3.0pp
improvedregressed6.1%3.6%2.5pp
brokestuck40.9%40.0%0.9pp
stuckregressed0.9%1.6%0.7pp
breakthroughend_limit2.1%2.6%0.5pp
stuckimproved15.8%16.1%0.3pp
stuckbroke3.5%3.2%0.3pp
churnbroke66.7%0.0pp
churnchurn33.3%33.3%0.0pp
churnend_limit16.7%0.0pp
churnimproved16.7%0.0pp
churnregressed33.3%0.0pp
regressedbreakthrough20.0%0.0pp
regressedbroke50.0%0.0pp
regressedchurn40.0%0.0pp
regressedend_limit50.0%0.0pp
regressedimproved20.0%0.0pp
regressedstuck20.0%0.0pp
staybroke2.1%0.0pp
stuckchurn0.9%0.0pp
stuckend_submit2.6%0.0pp

Gap label distributions — sorted by edge visit rate, labels by avg%

transitionlabelGemma (100)Qwen (96)GPT-5.5 (94)Sonnet 4.6 (90)GLM-5.2 (78)Δ (max−min)
startbreakthrough (0.82×/traj)implement100.0%100.0%100.0%100.0%100.0%0.0pp
breakthroughend_submit (0.79×/traj)verify100.0%100.0%100.0%100.0%100.0%0.0pp
staystay (0.37×/traj)cosmetic_refactor37.2%66.7%29.5pp
verify19.4%0.0pp
fix_bug12.8%100.0%87.2pp
implement7.8%0.0pp
optimize_perf7.2%0.0pp
revert6.7%0.0pp
edge_case5.6%33.3%27.7pp
infra_confusion2.2%100.0%97.8pp
debug_failure1.1%0.0pp
stuckstuck (0.19×/traj)explore44.1%0.0pp
implement20.6%22.2%1.6pp
rewrite14.7%25.9%11.2pp
fix_bug5.9%37.0%100.0%94.1pp
verify5.9%0.0pp
debug_failure1.5%11.1%9.6pp
cosmetic_refactor4.4%0.0pp
execute_plan1.5%0.0pp
optimize_perf1.5%0.0pp
infra_confusion3.7%0.0pp
breakthroughstay (0.14×/traj)implement32.1%20.0%12.1pp
cosmetic_refactor25.0%40.0%15.0pp
fix_bug8.9%30.0%100.0%100.0%91.1pp
edge_case16.1%10.0%6.1pp
optimize_perf10.7%0.0pp
infra_confusion1.8%0.0pp
revert1.8%0.0pp
verify1.8%0.0pp
debug_failure1.8%0.0pp
startstuck (0.12×/traj)implement42.9%85.2%100.0%100.0%57.1pp
explore57.1%14.8%42.3pp
stayend_submit (0.11×/traj)verify100.0%100.0%100.0%0.0pp
infra_confusion100.0%0.0pp
stuckbreakthrough (0.07×/traj)fix_bug27.3%87.5%75.0%100.0%72.7pp
implement63.6%0.0pp
infra_confusion12.5%0.0pp
rewrite9.1%25.0%15.9pp
startimproved (0.06×/traj)implement100.0%100.0%100.0%100.0%100.0%0.0pp
stuckimproved (0.06×/traj)implement27.8%20.0%7.8pp
rewrite33.3%10.0%23.3pp
fix_bug5.6%50.0%44.4pp
revert11.1%0.0pp
explore11.1%0.0pp
debug_failure5.6%10.0%4.4pp
execute_plan5.6%0.0pp
optimize_perf10.0%0.0pp
improvedstuck (0.04×/traj)implement26.7%25.0%1.7pp
cosmetic_refactor26.7%25.0%1.7pp
rewrite13.3%50.0%36.7pp
explore13.3%0.0pp
fix_bug6.7%100.0%93.3pp
verify6.7%0.0pp
debug_failure6.7%0.0pp
stuckend_limit (0.03×/traj)verify100.0%66.7%33.3pp
explore33.3%100.0%66.7pp
improvedend_limit (0.03×/traj)debug_failure50.0%66.7%16.7pp
verify100.0%25.0%100.0%75.0pp
explore25.0%33.3%8.3pp
brokestuck (0.03×/traj)explore33.3%25.0%8.3pp
implement33.3%25.0%8.3pp
rewrite22.2%25.0%2.8pp
fix_bug25.0%0.0pp
debug_failure11.1%0.0pp
improvedbroke (0.03×/traj)explore44.4%25.0%19.4pp
rewrite33.3%50.0%16.7pp
execute_plan11.1%0.0pp
implement11.1%0.0pp
fix_bug25.0%0.0pp
improvedbreakthrough (0.02×/traj)fix_bug71.4%100.0%100.0%28.6pp
rewrite14.3%100.0%85.7pp
debug_failure100.0%0.0pp
execute_plan14.3%0.0pp
brokebreakthrough (0.02×/traj)fix_bug66.7%50.0%16.7pp
implement33.3%0.0pp
rewrite50.0%0.0pp
stayend_limit (0.02×/traj)verify100.0%0.0pp
debug_failure100.0%0.0pp
brokeimproved (0.02×/traj)rewrite50.0%50.0%0.0pp
execute_plan25.0%0.0pp
revert25.0%0.0pp
explore25.0%0.0pp
fix_bug25.0%0.0pp
improvedimproved (0.01×/traj)rewrite33.3%100.0%100.0%66.7pp
fix_bug33.3%100.0%66.7pp
implement33.3%0.0pp
stuckbroke (0.01×/traj)explore75.0%0.0pp
rewrite25.0%50.0%25.0pp
debug_failure50.0%0.0pp
breakthroughbroke (0.01×/traj)rewrite25.0%100.0%75.0pp
explore25.0%0.0pp
implement25.0%0.0pp
debug_failure25.0%0.0pp
staybroke (0.01×/traj)optimize_perf40.0%0.0pp
explore20.0%0.0pp
implement20.0%0.0pp
fix_bug20.0%0.0pp
breakthroughend_limit (0.01×/traj)verify100.0%66.7%33.3pp
fix_bug33.3%0.0pp
improvedend_submit (0.01×/traj)verify100.0%100.0%100.0%0.0pp
improvedchurn (0.01×/traj)rewrite100.0%0.0pp
implement50.0%0.0pp
fix_bug50.0%0.0pp
improvedregressed (0.01×/traj)implement100.0%0.0pp
fix_bug100.0%0.0pp
churnchurn (0.01×/traj)rewrite50.0%0.0pp
implement50.0%0.0pp
fix_bug100.0%0.0pp
stuckend_submit (0.01×/traj)verify100.0%0.0pp
churnbroke (0.00×/traj)rewrite50.0%0.0pp
implement50.0%0.0pp
stuckregressed (0.00×/traj)explore100.0%0.0pp
implement100.0%0.0pp
regressedchurn (0.00×/traj)rewrite50.0%0.0pp
implement50.0%0.0pp
churnregressed (0.00×/traj)implement50.0%0.0pp
fix_bug50.0%0.0pp
regressedbroke (0.00×/traj)rewrite100.0%0.0pp
regressedend_limit (0.00×/traj)explore100.0%0.0pp
regressedstuck (0.00×/traj)debug_failure100.0%0.0pp
churnimproved (0.00×/traj)fix_bug100.0%0.0pp
churnend_limit (0.00×/traj)verify100.0%0.0pp
regressedimproved (0.00×/traj)debug_failure100.0%0.0pp
regressedbreakthrough (0.00×/traj)debug_failure100.0%0.0pp
stuckchurn (0.00×/traj)explore100.0%0.0pp

Ocaml

highest in row   lowest in row  Sorted by Δ (max − min) descending.

Node visit rates — sorted by Δ

stateGemma (100)Qwen (93)GPT-5.5 (92)Sonnet 4.6 (91)GLM-5.2 (77)Δ
start1.01.01.01.01.00.00
improved0.40.30.10.20.00.33
churn0.00.00
stuck13.45.30.40.10.413.31
breakthrough0.90.81.00.91.00.18
stay0.90.10.00.00.00.88
regressed0.00.00.00.01
broke0.60.20.00.59
end_submit0.50.71.01.01.00.51
end_limit0.50.30.00.00.00.51

Edge visit rates (count / n_problems) — sorted by Δ

transitionGemma (100)Qwen (93)GPT-5.5 (92)Sonnet 4.6 (91)GLM-5.2 (77)Δ (max−min)
stuckstuck12.0×4.4×0.1×0.1×11.91×
startstuck0.8×0.8×0.3×0.0×0.3×0.75×
breakthroughend_submit0.3×0.6×1.0×0.9×1.0×0.63×
startbreakthrough0.2×0.2×0.7×0.8×0.7×0.59×
staystay0.6×0.0×0.0×0.0×0.59×
stuckbreakthrough0.5×0.5×0.3×0.1×0.3×0.49×
stuckend_limit0.4×0.2×0.0×0.41×
breakthroughstay0.3×0.1×0.0×0.0×0.0×0.28×
brokestuck0.4×0.1×0.26×
stuckimproved0.3×0.2×0.0×0.25×
improvedstuck0.2×0.1×0.0×0.18×
breakthroughbroke0.2×0.0×0.18×
startimproved0.0×0.0×0.0×0.2×0.0×0.15×
stayend_submit0.1×0.1×0.0×0.0×0.13×
improvedbroke0.1×0.1×0.0×0.12×
brokebreakthrough0.1×0.0×0.11×
stuckbroke0.1×0.0×0.10×
brokeimproved0.1×0.1×0.0×0.08×
improvedbreakthrough0.0×0.1×0.0×0.1×0.0×0.08×
improvedend_limit0.0×0.0×0.0×0.0×0.03×
stayend_limit0.0×0.0×0.03×
improvedregressed0.0×0.0×0.01×
improvedimproved0.0×0.0×0.01×
regressedbroke0.0×0.0×0.01×
breakthroughend_limit0.0×0.0×0.00×
breakthroughregressed0.0×0.0×0.00×
improvedend_submit0.0×0.0×0.0×0.00×
regressedimproved0.0×0.0×0.00×
brokeend_limit0.0×0.00×
churnchurn0.0×0.00×
churnimproved0.0×0.00×
improvedchurn0.0×0.00×
regressedbreakthrough0.0×0.00×
regressedend_limit0.0×0.00×
regressedstuck0.0×0.00×
staybroke0.1×0.00×
stayregressed0.0×0.00×
stuckend_submit0.0×0.00×

Transition probabilities P(tgt | src) — sorted by Δ

transitionGemma (100)Qwen (93)GPT-5.5 (92)Sonnet 4.6 (91)GLM-5.2 (77)Δ (max−min)
improvedbreakthrough2.8%18.8%60.0%44.4%100.0%97.2pp
stuckbreakthrough4.0%8.9%62.5%100.0%74.2%96.0pp
stayend_limit4.5%100.0%95.5pp
brokeimproved15.0%29.4%100.0%85.0pp
startstuck78.0%75.3%29.3%3.3%31.2%74.7pp
stayend_submit15.7%84.6%50.0%50.0%68.9pp
stuckstuck89.8%82.3%32.5%22.6%67.2pp
breakthroughend_submit38.2%74.7%98.9%98.8%98.7%60.7pp
startbreakthrough21.0%23.7%66.3%80.2%66.2%59.2pp
staystay67.4%15.4%50.0%50.0%52.0pp
improvedstuck55.6%25.0%11.1%44.5pp
regressedbroke66.7%33.3%33.4pp
breakthroughstay32.6%14.7%1.1%1.2%1.3%31.5pp
improvedbroke36.1%28.1%5.6%30.5pp
breakthroughbroke23.6%4.0%19.6pp
improvedend_submit3.1%20.0%5.6%16.9pp
regressedimproved33.3%50.0%16.7pp
startimproved1.0%1.1%4.3%16.5%2.6%15.5pp
improvedend_limit5.6%12.5%20.0%16.7%14.4pp
brokebreakthrough21.7%11.8%9.9pp
improvedregressed3.1%11.1%8.0pp
brokestuck61.7%58.8%2.9pp
stuckimproved1.9%4.4%2.5%2.5pp
improvedimproved6.2%5.6%0.6pp
breakthroughend_limit3.4%4.0%0.6pp
breakthroughregressed2.2%2.7%0.5pp
stuckend_limit3.1%3.6%3.2%0.5pp
stuckbroke1.0%0.8%0.2pp
brokeend_limit1.7%0.0pp
churnchurn50.0%0.0pp
churnimproved50.0%0.0pp
improvedchurn3.1%0.0pp
regressedbreakthrough33.3%0.0pp
regressedend_limit50.0%0.0pp
regressedstuck33.3%0.0pp
staybroke11.2%0.0pp
stayregressed1.1%0.0pp
stuckend_submit2.5%0.0pp

Gap label distributions — sorted by edge visit rate, labels by avg%

transitionlabelGemma (100)Qwen (93)GPT-5.5 (92)Sonnet 4.6 (91)GLM-5.2 (77)Δ (max−min)
stuckstuck (3.32×/traj)revert31.2%0.0pp
fix_bug14.2%47.1%15.4%85.7%71.5pp
debug_failure18.6%10.3%7.7%14.3%10.9pp
implement15.0%12.5%15.4%2.9pp
infra_confusion4.8%16.4%61.5%56.7pp
explore7.1%3.7%3.4pp
cosmetic_refactor4.8%1.0%3.8pp
rewrite2.5%7.6%5.1pp
execute_plan1.4%1.2%0.2pp
verify0.4%0.2%0.2pp
optimize_perf0.1%0.0pp
breakthroughend_submit (0.76×/traj)verify100.0%98.2%100.0%100.0%100.0%1.8pp
fix_bug1.8%0.0pp
startbreakthrough (0.51×/traj)implement100.0%100.0%100.0%100.0%100.0%0.0pp
startstuck (0.43×/traj)implement76.9%81.4%96.3%100.0%100.0%23.1pp
explore23.1%18.6%3.7%19.4pp
stuckbreakthrough (0.33×/traj)fix_bug61.1%72.7%36.0%100.0%65.2%64.0pp
infra_confusion9.3%15.9%52.0%30.4%42.7pp
cosmetic_refactor11.1%4.5%4.0%4.3%7.1pp
execute_plan5.6%4.0%1.6pp
implement3.7%2.3%4.0%1.7pp
rewrite3.7%4.5%0.8pp
debug_failure5.6%0.0pp
staystay (0.13×/traj)verify28.3%50.0%21.7pp
cosmetic_refactor18.3%50.0%31.7pp
fix_bug15.0%100.0%85.0pp
edge_case11.7%0.0pp
implement10.0%0.0pp
revert8.3%0.0pp
infra_confusion1.7%100.0%98.3pp
optimize_perf3.3%0.0pp
debug_failure3.3%0.0pp
stuckend_limit (0.13×/traj)verify71.4%33.3%100.0%66.7pp
debug_failure19.0%16.7%2.3pp
explore4.8%33.3%28.5pp
infra_confusion16.7%0.0pp
implement2.4%0.0pp
rewrite2.4%0.0pp
stuckimproved (0.10×/traj)fix_bug34.6%77.3%42.7pp
infra_confusion15.4%18.2%100.0%84.6pp
cosmetic_refactor23.1%0.0pp
implement15.4%4.5%10.9pp
revert11.5%0.0pp
brokestuck (0.10×/traj)revert24.3%0.0pp
fix_bug21.6%10.0%11.6pp
debug_failure24.3%0.0pp
explore10.8%20.0%9.2pp
implement5.4%20.0%14.6pp
rewrite2.7%30.0%27.3pp
infra_confusion2.7%20.0%17.3pp
cosmetic_refactor5.4%0.0pp
execute_plan2.7%0.0pp
breakthroughstay (0.09×/traj)fix_bug10.3%27.3%100.0%100.0%100.0%89.7pp
verify27.6%9.1%18.5pp
cosmetic_refactor10.3%45.5%35.2pp
infra_confusion13.8%18.2%4.4pp
edge_case17.2%0.0pp
optimize_perf10.3%0.0pp
implement3.4%0.0pp
rewrite3.4%0.0pp
debug_failure3.4%0.0pp
improvedstuck (0.06×/traj)fix_bug30.0%12.5%100.0%87.5pp
revert30.0%0.0pp
debug_failure5.0%50.0%45.0pp
verify20.0%0.0pp
implement25.0%0.0pp
explore12.5%0.0pp
cosmetic_refactor5.0%0.0pp
rewrite5.0%0.0pp
infra_confusion5.0%0.0pp
stayend_submit (0.06×/traj)verify100.0%100.0%100.0%0.0pp
infra_confusion100.0%0.0pp
startimproved (0.05×/traj)implement100.0%100.0%100.0%100.0%100.0%0.0pp
breakthroughbroke (0.05×/traj)fix_bug28.6%0.0pp
infra_confusion23.8%33.3%9.5pp
debug_failure23.8%33.3%9.5pp
optimize_perf9.5%0.0pp
edge_case4.8%0.0pp
revert4.8%0.0pp
explore33.3%0.0pp
rewrite4.8%0.0pp
improvedbroke (0.05×/traj)fix_bug38.5%44.4%100.0%61.5pp
rewrite7.7%22.2%14.5pp
debug_failure7.7%22.2%14.5pp
explore7.7%11.1%3.4pp
implement15.4%0.0pp
cosmetic_refactor15.4%0.0pp
infra_confusion7.7%0.0pp
improvedbreakthrough (0.04×/traj)fix_bug100.0%100.0%37.5%100.0%62.5pp
rewrite37.5%0.0pp
implement12.5%0.0pp
infra_confusion100.0%0.0pp
debug_failure12.5%0.0pp
stuckbroke (0.04×/traj)fix_bug42.9%0.0pp
debug_failure14.3%50.0%35.7pp
explore7.1%25.0%17.9pp
implement14.3%0.0pp
rewrite7.1%25.0%17.9pp
revert7.1%0.0pp
infra_confusion7.1%0.0pp
brokeimproved (0.03×/traj)fix_bug22.2%60.0%37.8pp
cosmetic_refactor22.2%100.0%77.8pp
debug_failure22.2%20.0%2.2pp
revert22.2%0.0pp
implement11.1%20.0%8.9pp
brokebreakthrough (0.03×/traj)fix_bug53.8%0.0pp
implement23.1%50.0%26.9pp
cosmetic_refactor7.7%50.0%42.3pp
infra_confusion7.7%0.0pp
debug_failure7.7%0.0pp
improvedend_limit (0.02×/traj)explore25.0%100.0%100.0%75.0pp
verify50.0%75.0%25.0pp
debug_failure50.0%0.0pp
staybroke (0.02×/traj)debug_failure40.0%0.0pp
implement20.0%0.0pp
fix_bug20.0%0.0pp
rewrite10.0%0.0pp
infra_confusion10.0%0.0pp
breakthroughend_limit (0.01×/traj)verify100.0%100.0%0.0pp
stayend_limit (0.01×/traj)verify80.0%0.0pp
infra_confusion20.0%100.0%80.0pp
breakthroughregressed (0.01×/traj)fix_bug50.0%100.0%50.0pp
infra_confusion50.0%0.0pp
improvedregressed (0.01×/traj)rewrite100.0%0.0pp
fix_bug100.0%0.0pp
improvedend_submit (0.01×/traj)verify100.0%100.0%100.0%0.0pp
improvedimproved (0.01×/traj)fix_bug100.0%100.0%0.0pp
regressedbroke (0.01×/traj)infra_confusion100.0%0.0pp
implement50.0%0.0pp
fix_bug50.0%0.0pp
regressedimproved (0.00×/traj)fix_bug100.0%100.0%0.0pp
regressedend_limit (0.00×/traj)fix_bug100.0%0.0pp
stuckend_submit (0.00×/traj)infra_confusion100.0%0.0pp
churnimproved (0.00×/traj)fix_bug100.0%0.0pp
improvedchurn (0.00×/traj)fix_bug100.0%0.0pp
regressedbreakthrough (0.00×/traj)fix_bug100.0%0.0pp
churnchurn (0.00×/traj)fix_bug100.0%0.0pp
regressedstuck (0.00×/traj)cosmetic_refactor100.0%0.0pp
brokeend_limit (0.00×/traj)verify100.0%0.0pp
stayregressed (0.00×/traj)rewrite100.0%0.0pp