Refresh scores and tools

research/compare_template.html

@@ -37,9 +37,9 @@ td.tag{color:var(--accent);font-size:11px}
 <header><h1>MOSS SFT DeBand: candidate fixes side by side</h1>
 <div class="sub">Generated __WHEN__. Same prompt and seed in every column. Add, remove or change columns freely (your layout is kept in the page link). Drag to pan, wheel or buttons to zoom; all columns move together. Click a leaderboard row to put it in the selected column (outlined); shift-click adds it as a new column. <a href="edit_anatomy.html">What each fix changes inside the model →</a></div></header>
 <details class="explain"><summary>What the arms are (exact revert vs Gram solve, what ρ means)</summary>
-<p><b>Exact revert 21–41</b>: copy MOSS-Image Base's weights into SFT for transformer layers 21–41 (every attention and MLP matrix). It removes the stripes about as well as Base, but it also undoes SFT's fine-tuning in half the network, so faces and style drift a lot (ArcFace ≈ 0.65).</p>
+<p><b>Exact revert 21–41</b>: copy MOSS-Image Base's weights into SFT for transformer layers 21–41 (every attention and MLP matrix). It removes most of the stripes (median −12.7 dB; Base −16.6), but it also undoes SFT's fine-tuning in half the network, so faces and style drift a lot (ArcFace ≈ 0.59).</p>
 <p><b>Gram solve</b>: instead of copying Base's weights, each layer gets a weight change D chosen so that its <i>output</i> matches Base only for the kinds of input the layer actually sees while SFT draws a solid colour. Those inputs were recorded from 36 solid-colour SFT renders (the "flat Gram" G). For inputs that look unlike those, the layer keeps SFT's behaviour. Formula: D = (W<sub>Base</sub> − W<sub>SFT</sub>) · G (G + εI)<sup>−1</sup>.</p>
-<p><b>ρ</b> sets ε (ε = ρ × the average eigenvalue of G). It controls how narrow the edit is. ρ → 0 makes D the full Base−SFT difference, the same as the exact revert. Larger ρ keeps only the strongest solid-colour input directions, so the edit gets smaller and the result looks more like SFT: ρ=0.1 keeps most of the revert (cleanest solids, faces ≈ 0.76), ρ=1 keeps less (faces ≈ 0.81, solids only −5 dB), and ρ=10 barely changes anything.</p>
+<p><b>ρ</b> sets ε (ε = ρ × the average eigenvalue of G). It controls how narrow the edit is. ρ → 0 makes D the full Base−SFT difference, the same as the exact revert. Larger ρ keeps only the strongest solid-colour input directions, so the edit gets smaller and the result looks more like SFT: ρ=0.1 keeps most of the revert (solids −6.6 dB median, faces ≈ 0.72), ρ=1 keeps less (faces ≈ 0.83, solids only −3 dB), and ρ=10 barely changes anything.</p>
 <p><b>Kijai LoRA @ −1 / −0.5</b>: subtracts Kijai's LoRA (or half of it). It is a rank-512 compressed approximation of the SFT↔Base difference covering layers 3, 4 and 21–41 plus the norms, so even at −1 it is only part of a full revert. It is not the same as our plain revert.</p>
 <p><b>Plain revert ×0.55</b>: the control, our own exact blend W = SFT + 0.55·(Base − SFT) on layers 21–41, using the full difference with no compression. It moves every weight 55% of the way to Base, an edit about as big as ρ=0.1's. It only reaches −5.6 dB at about the same face drift, so the solve really is choosing <i>where</i> to edit, not just editing less.</p>
 <p><b>Centred-only / DC-only solves</b>: the activations are split into two parts. One is each token row's average along the width: the overall colour and brightness, plus the timestep modulation shift ("DC"). The other is the horizontal variation around that average, which is where the stripes live. <i>Centred-only</i> makes the layers match Base on the horizontal variation and keeps SFT on the DC part. <i>DC-only</i> does the reverse. Centred-only does nothing for solids, DC-only gets about −5 dB with faces ≈ 0.86, and the full solve needs both parts together.</p>

research/scores/scores.jsonl

@@ -1954,3 +1954,69 @@
 {"arm": "strong_r64", "id": "t_cafe_chalkboard", "seed": 11, "category": "typography", "band": 1.0583546161651611, "band_ref": 1.708121418952942, "ssim": 0.8306995005506957, "dino_dist": 0.0105665922164917}
 {"arm": "strong_r64", "id": "x_golden_retriever", "seed": 11, "category": "animals", "band": 1.5115619897842407, "band_ref": 1.3215339183807373, "ssim": 0.7650482569110143, "dino_dist": 0.0559154748916626}
 {"arm": "strong_r64", "id": "x_red_fox_snow", "seed": 11, "category": "animals", "band": 1.113404631614685, "band_ref": 1.580950140953064, "ssim": 0.8473237794454628, "dino_dist": 0.004145622253417969}
+{"arm": "solve_mu0_L21-41", "id": "a_anime_girl_school", "seed": 11, "category": "anime_people", "band": 0.8685954213142395, "band_ref": 1.0670256614685059, "ssim": 0.500511601216772, "dino_dist": 0.1324683427810669}
+{"arm": "solve_mu0_L21-41", "id": "bench_fjord_sky", "seed": 11, "category": "bench", "band": 1.3401745557785034, "band_ref": 2.209906578063965, "ssim": 0.7934762880927481, "dino_dist": 0.22367453575134277}
+{"arm": "solve_mu0_L21-41", "id": "bench_fog_forest", "seed": 11, "category": "bench", "band": 1.0942302942276, "band_ref": 2.3328261375427246, "ssim": 0.5972304802299249, "dino_dist": 0.07290434837341309}
+{"arm": "solve_mu0_L21-41", "id": "o_kitchen_scene", "seed": 11, "category": "scenes", "band": 1.2363791465759277, "band_ref": 1.3171018362045288, "ssim": 0.438992837401517, "dino_dist": 0.12241065502166748}
+{"arm": "solve_mu0_L21-41", "id": "p_child_boy_black", "seed": 11, "category": "people", "band": 0.9539889693260193, "band_ref": 1.523751974105835, "ssim": 0.7217157977273176, "dino_dist": 0.3213276267051697, "arcface": 0.7385048866271973}
+{"arm": "solve_mu0_L21-41", "id": "p_child_boy_black", "seed": 14, "category": "people", "band": 0.8218343257904053, "band_ref": 1.2047946453094482, "ssim": 0.6601185386616516, "dino_dist": 0.29305505752563477, "arcface": 0.6049118041992188}
+{"arm": "solve_mu0_L21-41", "id": "p_elder_man_west_african", "seed": 11, "category": "people", "band": 1.6189360618591309, "band_ref": 1.0768357515335083, "ssim": 0.6778666872337279, "dino_dist": 0.07305985689163208, "arcface": 0.6493797898292542}
+{"arm": "solve_mu0_L21-41", "id": "p_elder_man_west_african", "seed": 14, "category": "people", "band": 1.44669771194458, "band_ref": 1.5866210460662842, "ssim": 0.567217496222291, "dino_dist": 0.14519309997558594, "arcface": 0.5208163261413574}
+{"arm": "solve_mu0_L21-41", "id": "p_young_woman_east_asian", "seed": 11, "category": "people", "band": 0.949829638004303, "band_ref": 2.2854697704315186, "ssim": 0.6311499131599004, "dino_dist": 0.09046530723571777, "arcface": 0.5520361661911011}
+{"arm": "solve_mu0_L21-41", "id": "p_young_woman_east_asian", "seed": 14, "category": "people", "band": 1.1319016218185425, "band_ref": 2.0958423614501953, "ssim": 0.7118468914400021, "dino_dist": 0.11475682258605957, "arcface": 0.4743722081184387}
+{"arm": "solve_mu0_L21-41", "id": "solid_beige", "seed": 11, "category": "solid", "band": 0.8819299936294556, "band_ref": 1.2300207614898682, "ssim": 0.9948973456538682, "hf_db": -17.043692010724293}
+{"arm": "solve_mu0_L21-41", "id": "solid_beige", "seed": 14, "category": "solid", "band": 1.1488767862319946, "band_ref": 1.9000169038772583, "ssim": 0.9927625781705224, "hf_db": -4.34377720479636}
+{"arm": "solve_mu0_L21-41", "id": "solid_beige", "seed": 15, "category": "solid", "band": 0.9340606331825256, "band_ref": 2.633002519607544, "ssim": 0.9903973413507304, "hf_db": -5.646240768161789}
+{"arm": "solve_mu0_L21-41", "id": "solid_blue", "seed": 11, "category": "solid", "band": 0.9114198088645935, "band_ref": 1.6862902641296387, "ssim": 0.9902217574936162, "hf_db": -11.7987820440384}
+{"arm": "solve_mu0_L21-41", "id": "solid_blue", "seed": 14, "category": "solid", "band": 0.8234798312187195, "band_ref": 2.0521998405456543, "ssim": 0.9910494774677044, "hf_db": -29.458743862496913}
+{"arm": "solve_mu0_L21-41", "id": "solid_blue", "seed": 15, "category": "solid", "band": 0.9404176473617554, "band_ref": 2.5136542320251465, "ssim": 0.9819599136923552, "hf_db": -5.6711212824163955}
+{"arm": "solve_mu0_L21-41", "id": "solid_grey", "seed": 11, "category": "solid", "band": 0.9239828586578369, "band_ref": 2.2142326831817627, "ssim": 0.9858921781731581, "hf_db": -31.210417825800548}
+{"arm": "solve_mu0_L21-41", "id": "solid_grey", "seed": 14, "category": "solid", "band": 1.1412206888198853, "band_ref": 2.3253705501556396, "ssim": 0.9934223072056051, "hf_db": -12.70237457533714}
+{"arm": "solve_mu0_L21-41", "id": "solid_grey", "seed": 15, "category": "solid", "band": 1.1903011798858643, "band_ref": 3.0369372367858887, "ssim": 0.9917335805614353, "hf_db": -18.256601289873696}
+{"arm": "solve_mu0_L21-41", "id": "t_cafe_chalkboard", "seed": 11, "category": "typography", "band": 1.0291728973388672, "band_ref": 1.708121418952942, "ssim": 0.46028704663224146, "dino_dist": 0.05389404296875}
+{"arm": "solve_mu0_L21-41", "id": "x_golden_retriever", "seed": 11, "category": "animals", "band": 1.3605784177780151, "band_ref": 1.3215339183807373, "ssim": 0.46255877327388245, "dino_dist": 0.16304337978363037}
+{"arm": "solve_mu0_L21-41", "id": "x_red_fox_snow", "seed": 11, "category": "animals", "band": 1.2282706499099731, "band_ref": 1.580950140953064, "ssim": 0.7131553696354381, "dino_dist": 0.028489291667938232}
+{"arm": "solve_mu0_rho1_L21-41", "id": "a_anime_girl_school", "seed": 11, "category": "anime_people", "band": 0.9980393648147583, "band_ref": 1.0670256614685059, "ssim": 0.5220761539567909, "dino_dist": 0.08072888851165771}
+{"arm": "solve_mu0_rho1_L21-41", "id": "bench_fjord_sky", "seed": 11, "category": "bench", "band": 1.0229761600494385, "band_ref": 2.209906578063965, "ssim": 0.8220320484115557, "dino_dist": 0.16431856155395508}
+{"arm": "solve_mu0_rho1_L21-41", "id": "bench_fog_forest", "seed": 11, "category": "bench", "band": 1.2614099979400635, "band_ref": 2.3328261375427246, "ssim": 0.6619406150490851, "dino_dist": 0.06419825553894043}
+{"arm": "solve_mu0_rho1_L21-41", "id": "o_kitchen_scene", "seed": 11, "category": "scenes", "band": 1.1279630661010742, "band_ref": 1.3171018362045288, "ssim": 0.5467492265373028, "dino_dist": 0.06120115518569946}
+{"arm": "solve_mu0_rho1_L21-41", "id": "p_child_boy_black", "seed": 11, "category": "people", "band": 1.0795423984527588, "band_ref": 1.523751974105835, "ssim": 0.7913477336898183, "dino_dist": 0.1672145128250122, "arcface": 0.8415842652320862}
+{"arm": "solve_mu0_rho1_L21-41", "id": "p_child_boy_black", "seed": 14, "category": "people", "band": 1.1065877676010132, "band_ref": 1.2047946453094482, "ssim": 0.7225895533356717, "dino_dist": 0.17360162734985352, "arcface": 0.8585579991340637}
+{"arm": "solve_mu0_rho1_L21-41", "id": "p_elder_man_west_african", "seed": 11, "category": "people", "band": 0.8740415573120117, "band_ref": 1.0768357515335083, "ssim": 0.8190127452351232, "dino_dist": 0.027416110038757324, "arcface": 0.9029984474182129}
+{"arm": "solve_mu0_rho1_L21-41", "id": "p_elder_man_west_african", "seed": 14, "category": "people", "band": 1.0842944383621216, "band_ref": 1.5866210460662842, "ssim": 0.7504835848230765, "dino_dist": 0.012847542762756348, "arcface": 0.8069422245025635}
+{"arm": "solve_mu0_rho1_L21-41", "id": "p_young_woman_east_asian", "seed": 11, "category": "people", "band": 0.9523270130157471, "band_ref": 2.2854697704315186, "ssim": 0.676073542523663, "dino_dist": 0.041439056396484375, "arcface": 0.6690477132797241}
+{"arm": "solve_mu0_rho1_L21-41", "id": "p_young_woman_east_asian", "seed": 14, "category": "people", "band": 0.9498544335365295, "band_ref": 2.0958423614501953, "ssim": 0.8888951037199296, "dino_dist": 0.025026977062225342, "arcface": 0.9164827466011047}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_beige", "seed": 11, "category": "solid", "band": 1.3916088342666626, "band_ref": 1.2300207614898682, "ssim": 0.9934222308731528, "hf_db": -5.557835116739032}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_beige", "seed": 14, "category": "solid", "band": 1.1135655641555786, "band_ref": 1.9000169038772583, "ssim": 0.9908888829491472, "hf_db": -1.3986008684018703}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_beige", "seed": 15, "category": "solid", "band": 1.032639741897583, "band_ref": 2.633002519607544, "ssim": 0.9912597104920091, "hf_db": -3.2099332536187752}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_blue", "seed": 11, "category": "solid", "band": 0.9736127853393555, "band_ref": 1.6862902641296387, "ssim": 0.9840464531841814, "hf_db": -3.0809888465333395}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_blue", "seed": 14, "category": "solid", "band": 1.055769443511963, "band_ref": 2.0521998405456543, "ssim": 0.981023888196563, "hf_db": 1.7189824935017748}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_blue", "seed": 15, "category": "solid", "band": 1.0441529750823975, "band_ref": 2.5136542320251465, "ssim": 0.9895181769717022, "hf_db": -2.049827758247686}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_grey", "seed": 11, "category": "solid", "band": 0.9175907969474792, "band_ref": 2.2142326831817627, "ssim": 0.9861989019924247, "hf_db": -5.247637313777968}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_grey", "seed": 14, "category": "solid", "band": 2.3537333011627197, "band_ref": 2.3253705501556396, "ssim": 0.9896510081246244, "hf_db": -1.976585795110565}
+{"arm": "solve_mu0_rho1_L21-41", "id": "solid_grey", "seed": 15, "category": "solid", "band": 2.167928457260132, "band_ref": 3.0369372367858887, "ssim": 0.9914912368277253, "hf_db": -3.520846957488663}
+{"arm": "solve_mu0_rho1_L21-41", "id": "t_cafe_chalkboard", "seed": 11, "category": "typography", "band": 0.9793878197669983, "band_ref": 1.708121418952942, "ssim": 0.5616217843543098, "dino_dist": 0.030701637268066406}
+{"arm": "solve_mu0_rho1_L21-41", "id": "x_golden_retriever", "seed": 11, "category": "animals", "band": 0.9994532465934753, "band_ref": 1.3215339183807373, "ssim": 0.6888449807236041, "dino_dist": 0.028022587299346924}
+{"arm": "solve_mu0_rho1_L21-41", "id": "x_red_fox_snow", "seed": 11, "category": "animals", "band": 1.159222960472107, "band_ref": 1.580950140953064, "ssim": 0.7739011885531829, "dino_dist": 0.013748884201049805}
+{"arm": "kijai_m05", "id": "a_anime_girl_school", "seed": 11, "category": "anime_people", "band": 1.249603509902954, "band_ref": 1.0670256614685059, "ssim": 0.7137945200517185, "dino_dist": 0.03721320629119873}
+{"arm": "kijai_m05", "id": "bench_fjord_sky", "seed": 11, "category": "bench", "band": 1.7375454902648926, "band_ref": 2.209906578063965, "ssim": 0.8817018487868387, "dino_dist": 0.10590159893035889}
+{"arm": "kijai_m05", "id": "bench_fog_forest", "seed": 11, "category": "bench", "band": 1.7004669904708862, "band_ref": 2.3328261375427246, "ssim": 0.8098231078988191, "dino_dist": 0.03623998165130615}
+{"arm": "kijai_m05", "id": "o_kitchen_scene", "seed": 11, "category": "scenes", "band": 1.2818944454193115, "band_ref": 1.3171018362045288, "ssim": 0.5722944348837573, "dino_dist": 0.05294311046600342}
+{"arm": "kijai_m05", "id": "p_child_boy_black", "seed": 11, "category": "people", "band": 0.9658836722373962, "band_ref": 1.523751974105835, "ssim": 0.7949705043429831, "dino_dist": 0.19828087091445923, "arcface": 0.9162356853485107}
+{"arm": "kijai_m05", "id": "p_child_boy_black", "seed": 14, "category": "people", "band": 1.1852126121520996, "band_ref": 1.2047946453094482, "ssim": 0.7689890179829818, "dino_dist": 0.15497159957885742, "arcface": 0.881453275680542}
+{"arm": "kijai_m05", "id": "p_elder_man_west_african", "seed": 11, "category": "people", "band": 1.5359485149383545, "band_ref": 1.0768357515335083, "ssim": 0.8740490483066026, "dino_dist": 0.017622828483581543, "arcface": 0.9243819713592529}
+{"arm": "kijai_m05", "id": "p_elder_man_west_african", "seed": 14, "category": "people", "band": 1.1077895164489746, "band_ref": 1.5866210460662842, "ssim": 0.892502663894549, "dino_dist": 0.007449805736541748, "arcface": 0.9214787483215332}
+{"arm": "kijai_m05", "id": "p_young_woman_east_asian", "seed": 11, "category": "people", "band": 1.0034066438674927, "band_ref": 2.2854697704315186, "ssim": 0.7365996431252412, "dino_dist": 0.02831214666366577, "arcface": 0.7009682655334473}
+{"arm": "kijai_m05", "id": "p_young_woman_east_asian", "seed": 14, "category": "people", "band": 1.2627205848693848, "band_ref": 2.0958423614501953, "ssim": 0.9247996189657929, "dino_dist": 0.008061468601226807, "arcface": 0.8841421604156494}
+{"arm": "kijai_m05", "id": "solid_beige", "seed": 11, "category": "solid", "band": 1.518032193183899, "band_ref": 1.2300207614898682, "ssim": 0.9939607080409224, "hf_db": 1.1040827235757933}
+{"arm": "kijai_m05", "id": "solid_beige", "seed": 14, "category": "solid", "band": 1.6469022035598755, "band_ref": 1.9000169038772583, "ssim": 0.9963568058439846, "hf_db": -2.020197796331015}
+{"arm": "kijai_m05", "id": "solid_beige", "seed": 15, "category": "solid", "band": 1.8130478858947754, "band_ref": 2.633002519607544, "ssim": 0.993110294203572, "hf_db": -3.457676329453339}
+{"arm": "kijai_m05", "id": "solid_blue", "seed": 11, "category": "solid", "band": 1.737544298171997, "band_ref": 1.6862902641296387, "ssim": 0.9928068435154213, "hf_db": -3.4062147388765798}
+{"arm": "kijai_m05", "id": "solid_blue", "seed": 14, "category": "solid", "band": 2.7774157524108887, "band_ref": 2.0521998405456543, "ssim": 0.9928553570209946, "hf_db": -4.044837189928031}
+{"arm": "kijai_m05", "id": "solid_blue", "seed": 15, "category": "solid", "band": 2.6027944087982178, "band_ref": 2.5136542320251465, "ssim": 0.9936115934449198, "hf_db": -1.8465887342009641}
+{"arm": "kijai_m05", "id": "solid_grey", "seed": 11, "category": "solid", "band": 1.12076997756958, "band_ref": 2.2142326831817627, "ssim": 0.9908615829217533, "hf_db": -1.053543234674581}
+{"arm": "kijai_m05", "id": "solid_grey", "seed": 14, "category": "solid", "band": 2.676114082336426, "band_ref": 2.3253705501556396, "ssim": 0.9883272166423337, "hf_db": -0.8148266425803135}
+{"arm": "kijai_m05", "id": "solid_grey", "seed": 15, "category": "solid", "band": 2.0163087844848633, "band_ref": 3.0369372367858887, "ssim": 0.9918482265038867, "hf_db": -1.1414289860973295}
+{"arm": "kijai_m05", "id": "t_cafe_chalkboard", "seed": 11, "category": "typography", "band": 1.0151792764663696, "band_ref": 1.708121418952942, "ssim": 0.7519084949434154, "dino_dist": 0.01659846305847168}
+{"arm": "kijai_m05", "id": "x_golden_retriever", "seed": 11, "category": "animals", "band": 1.1687331199645996, "band_ref": 1.3215339183807373, "ssim": 0.6828106468346872, "dino_dist": 0.046830058097839355}
+{"arm": "kijai_m05", "id": "x_red_fox_snow", "seed": 11, "category": "animals", "band": 1.2775485515594482, "band_ref": 1.580950140953064, "ssim": 0.8493190480307304, "dino_dist": 0.017091989517211914}