# A learned layout generator trained on Crello designers (Dev 3, 2026-10-27) **Question.** The selector is good (judge-distilled critic, tie-breaks: `docs/crello-gains-2026-10-26.md`) but can only choose among what the section 2 generators propose, and on 3,000 designer-made size variants the uniform stretch is closer to the designer than our pick (0.190 vs 0.237 mean distance). Can a small model that sees the source layout and the target size propose better candidates, and do the fitter, hard checks, rule score and critic make use of them? **Answer in one paragraph.** A 33-input, 2 x 64 tanh MLP (31 KB, numpy) predicts a box per content unit. Alone it beats the stretch clearly on held-out design families (unit distance 0.1886 → 0.1525, element distance 0.1905 → 0.1545; better in 67% of pairs). As a candidate family (`learned`) it is the first of ours that is, on average, closer to the designers than the stretch (0.183 vs 0.195 where it is valid) and the valid candidate closest to the designer in 53% of pairs. Left to the selector alone it ships in 29% of held-out pairs and moves the pick distance 0.2522 → 0.2352 (+0.0169, family-bootstrap interval [+0.0132, +0.0209]) with no hard-check, fallback or floor-count change on our three sets, **but a judge round on its 144 changed picks went only 66 to 44 for it** (section 6b): the family reproduces what designers do, including small text and top-heavy stacks on tall canvases, and the rule score's fidelity term rewards exactly that. **Shipped with an admission guard** (a learned layout replaces the pick made without the family only if its smallest text is within 1 px and its utilization score within 0.05 of that pick; everything else is byte-identical to stage one): the kept picks were judged 54 to 17, 97 picks change on our three sets (all to `learned`; hard checks, fallbacks, floor counts and the mean smallest text unchanged), and on Crello the held-out gain is +0.0093 [+0.0062, +0.0127] with the changed picks' smallest text larger in 392 pairs and smaller in 36. ## 1. Dataset (`tools/crello/layout_dataset.py`) | | count | |---|---| | raw same-design pairs (Hugging Face) | 37,279 | | both templates in the extracted parquet | 31,153 (6,126 dropped: the pairs file also spans the test split, whose layouts were not extracted) | | same element count | 26,809 (4,344 dropped) | | same type sequence | 22,977 (3,832 dropped) | | same text-length sequence | 18,463 (4,514 dropped) | | at most 30 elements and at least one text | **18,449 usable pairs** (14 dropped), 1,316 families, 5,563 templates | | dropped while converting, analysing or featurising | **0** (no analysis error, no pair without a content unit) | | directed pairs (each pair read both ways) | 36,898 | | **rows** (one per content unit) | **240,227** (6.51 per pair); 120,039 as given, 120,188 swapped | | member elements of those units | 280,676 | | split by family (`pairs.py`, 70 / 30) | train 915 families, 160,997 rows; test 401 families, 79,230 rows (39,620 as given, 6,412 pairs) | | the 3,000-pair sample of `designer_eval.py` | all 3,000 pairs are in the dataset; 1,008 are in test families | The filters are `pairs.py`'s (`pairs_stats.json`); nothing new was dropped. Every usable pair is used, not just the 3,000 sample. Both directions are included (`rev` column): they read the same designer pair the other way, never straddle the split, and raised held-out quality slightly (exploration, one seed: unit distance 0.1555 as given only, 0.1522 with both). **Inputs.** `engine.resize._layout_features(analysis, target)`, the function the engine calls at run time, so training and serving cannot differ: the unit's source box and ink box as shares of the source canvas (8), kind flags (text, image, shape, pinned furniture, band, rotated, group), prominence, font size over the source short side, log text length, block index / count / relative position, flow axis, number of content units, source / target / log-ratio aspect, source and target short sides over 1000, rank by area and by font size, left / right alignment, sibling flag: 33 numbers, geometry and text length only (a rename test checks it). **Target.** The union of the designer's drawn bounds of the unit's member elements at the target size, as shares of the target canvas (cx, cy, w, h); the designer layout is `make_tasks.designer_layout` (same elements, the target template's geometry, text heights as our renderer measures the synthetic text). **Baseline.** The uniform stretch, which in these units is the source box itself. The designer's target font size per unit is stored too (`fs_t`) and was used to check text sizing, not as a target. Not modelled: backgrounds and decorations (the model places content units only; decorations follow the anchor's rules). ## 2. Models (`tools/crello/train_layout.py`; held-out families only) All models predict the correction to the stretch. Training uses 85% of the train families, early stopping and model selection use the other 15% (a hash of the family id), the 401 test families are only evaluated. Rows are weighted by 1 / sqrt(pairs in the family) (a handful of boilerplate families hold 20% of the pairs; exploration, one seed, unit distance: weight power 0 → 0.1589, 0.5 → 0.1517, 1 → 0.1559). Metrics on the 39,620 test rows (6,412 pairs): MAE of (cx, cy, w, h) as shares of the target canvas; **unit distance** = the engine's `dist` formula on unit boxes; **element distance** = the same on member elements of content units (predicted unit boxes mapped back to elements), pairs pooled and family-balanced. | model | MAE | MAE cx / cy / w / h | unit dist | element dist (pooled) | element dist (family-balanced) | sample test pairs (1,008) | |---|---|---|---|---|---|---| | (a) stretch | 0.0739 | 0.0684 / 0.0657 / 0.1028 / 0.0586 | 0.1886 | 0.1905 | 0.1260 | 0.1842 | | uniform scale min(sx, sy), centres kept | 0.0718 | 0.0684 / 0.0657 / 0.0969 / 0.0561 | 0.1843 | 0.1876 | 0.1250 | 0.1849 | | (b) ridge, squared loss (alpha 10) | 0.0705 | 0.0742 / 0.0685 / 0.0823 / 0.0570 | 0.1828 | 0.1822 | 0.1365 | 0.1774 | | (b') ridge model trained on the distance | 0.0669 | 0.0680 / 0.0653 / 0.0826 / 0.0518 | 0.1742 | 0.1757 | 0.1189 | 0.1692 | | (c) sklearn `MLPRegressor` 2 x 64 tanh, squared loss, 3 seeds | 0.0743 | 0.0914 / 0.0768 / 0.0757 / 0.0533 | 0.1976 (3 seeds 0.1974 ± 0.0003) | 0.1993 | 0.1612 | 0.1955 | | (d) the same + set context (mean of the other units' inputs) | 0.0810 | 0.1018 / 0.0833 / 0.0811 / 0.0579 | 0.2162 | 0.2181 | 0.1754 | 0.2139 | | numpy MLP 2 x 64 tanh, squared loss | 0.0646 | 0.0742 / 0.0672 / 0.0681 / 0.0490 | 0.1707 | 0.1711 | 0.1288 | 0.1667 | | **(c') numpy MLP 2 x 64 tanh on the distance, 5 seeds (shipped)** | **0.0573** | 0.0633 / 0.0611 / 0.0624 / 0.0425 | **0.1525** (5 seeds 0.1524 ± 0.0004) | **0.1545** | **0.1081** | **0.1488** | | (d') the same + set context | 0.0580 | 0.0646 / 0.0612 / 0.0637 / 0.0426 | 0.1541 (5 seeds ± 0.0004) | 0.1562 | 0.1096 | 0.1511 | What the table says: * **The loss is what matters.** Designers keep relative positions, so the stretch's centres are already good (MAE 0.068) and a squared-loss regression, which regresses to the mean, makes them worse (0.074 ridge, 0.091 sklearn MLP) while fixing the sizes. sklearn's `MLPRegressor` cannot train on anything else; a small numpy Adam loop on the metric itself (the engine's distance, smoothed) is what beats the stretch: every output improves except the centres, which it keeps at stretch quality (0.063, 0.061 against 0.068, 0.066). The squared-loss sklearn MLPs lose to the stretch, as does the one with context. * **Capacity and context.** The model is small on purpose: one hidden layer 0.1540, 2 x 32 0.1530, 2 x 64 0.1517, 2 x 128 0.1515, 3 x 64 0.1505, relu 0.1524 (exploration, validation protocol, one seed: unit distance). A 3-seed ensemble gives 0.1500 for three times the file. Twelve extra features (bounding box of all content, position within it, gaps to the nearest unit in each direction, overlaps, area share) gave 0.1506 against 0.1515, so the 33 features stay. A linear model on the metric is 0.1742: the gain needs the nonlinearity. **Set context does not help** (0.1541 against 0.1525): the simplest model that beats the stretch clearly is the 2 x 64 MLP without it, and that is what ships. * **Where it wins.** Better than the stretch in 67% of held-out pairs: 35% when the aspect changes by under 0.2 (it is not better than the stretch there: 0.0631 against 0.0606; the engine does not generate it below a 3% change anyway), 71% from 0.2 to 0.7, 84% above 0.7. By aspect-change bucket and target class (all 6,412 held-out pairs as given, element distance, pooled / family-balanced): | group | pairs | stretch | uniform | ridge | **mlp** | mlp beats stretch | |---|---|---|---|---|---|---| | all | 6412 | 0.1905 / 0.1260 | 0.1876 / 0.1250 | 0.1822 / 0.1365 | **0.1545 / 0.1081** | 67% | | aspect change 0-0.2 | 1557 | 0.0606 / 0.0633 | 0.0619 / 0.0644 | 0.0892 / 0.0930 | 0.0631 / 0.0656 | 35% | | aspect change 0.2-0.7 | 2707 | 0.1575 / 0.1341 | 0.1581 / 0.1336 | 0.1573 / 0.1383 | **0.1391 / 0.1165** | 71% | | aspect change 0.7+ | 2148 | 0.3263 / 0.3070 | 0.3157 / 0.3010 | 0.2808 / 0.2670 | **0.2402 / 0.2276** | 84% | | target strip | 544 | 0.2939 / 0.2228 | 0.2975 / 0.2234 | 0.2688 / 0.2151 | **0.2271 / 0.1780** | 73% | | target small | 1750 | 0.1686 / 0.1390 | 0.1659 / 0.1370 | 0.1662 / 0.1447 | **0.1432 / 0.1184** | 66% | | target square | 874 | 0.1803 / 0.1518 | 0.1726 / 0.1480 | 0.1732 / 0.1504 | **0.1566 / 0.1302** | 67% | | target vertical | 1693 | 0.1963 / 0.1422 | 0.1971 / 0.1443 | 0.1841 / 0.1466 | **0.1504 / 0.1146** | 62% | | target other | 1551 | 0.1783 / 0.1498 | 0.1714 / 0.1431 | 0.1727 / 0.1543 | **0.1452 / 0.1270** | 70% | Against our current candidates, on the 1,005 sample pairs in test families that have a recorded pick (element distance over the elements of content units, from `candidates.jsonl` of the branch without the model): stretch 0.1845, engine pick 0.2456, engine oracle (the valid candidate closest to the designer) 0.1748, **the model's raw boxes 0.1489**; it beats the stretch in 66% of these pairs, the pick in 86% and the oracle in 76%. By bucket (0-0.2 / 0.2-0.7 / 0.7+): 0.0595 / 0.1362 / 0.2332 against pick 0.0833 / 0.2420 / 0.3784 and oracle 0.0757 / 0.1561 / 0.2730. These are raw predictions (not fitted, checked or clamped), which is why the engine numbers below are smaller. Full per-class table: `output/crello/layout_train.md` (regenerated by the training command). The shipped file is the best of 5 seeds on validation (seed 0; test unit distance of the 5 seeds 0.1524 ± 0.0004), trained on train families only, 6,596 weights, 31.3 KB. ## 3. In the engine (`engine/resize.py`: end of section 2, section 5b) `_learned_candidates(an, target, ctx, existing)` (after `_proportional_candidates` in `generate_candidates`, same ±3% gate): `_layout_model()` (lazy, numpy, no pickle, cached per file state, validates names and shapes; the family is simply not generated if the file is absent, corrupt or inconsistent, so the output is byte-identical to the output without it; `CHAMELEON_LEARNED=0` and `TUNABLES["learned"]["enabled"]` switch it off) → `_layout_predict` (features → boxes in target px) → `_anchor_ladder(ctx, pred=…)`, the anchor's own rung ladder with `_anchor_try(pred=…)`: * free units are centred on the predicted centre; furniture, bands and satellites are placed as in `prop`; * **text** takes the predicted width (at least its longest word, at most the inner width), the height its wrapped text needs, and a size that starts at what `prop` gives (uniform scale lifted to the floor of the rung) and, at rungs above 0, grows to the largest whole size, up to the anchor's cap, whose wrapped text is at most 1.15 times as tall as the predicted box; * **images, groups and rotated elements** get the largest uniform scale that fits the predicted box and the margins (floor-lifted); a plain shape takes the predicted box; * then the anchor's margin clamp, ink push-apart, strictness per rung, satellites; decorations as in `prop`; * up to 2 layouts (`learned`, `learned:`, highest rung first), skipping layouts equal to any existing candidate, appended last, so every older candidate is unchanged; never raises (a failure logs `[learned] skipped` and yields none). **Admission guard (`docs/engine.md` B30, added after judge round 6, section 6b).** `resize()` chooses twice (`_choose`: `_select`, then the critic): over every candidate and over the candidates that are not learned. When the two picks differ, the learned pick ships only if `_learned_admissible` holds: its smallest text is at most `TUNABLES["learned"]["guard_px"]` (1) px under the family-off pick's and its utilization score (S5) at most `guard_util` (0.05) lower; otherwise the family-off pick ships. The log line ends in `learned over ` when a learned layout ships. So the family cannot change the output except through a learned layout that passes the guard: the 8 picks its presence moved through the critic's finalist set (section 5) revert, and without a shipped learned layout the output is byte-identical to stage one's (`tests/test_learned.py`). Why "box" text: designers' font sizes are the area-preserving size, median 1.0 x sqrt(sx sy) of the source size, against the uniform min(sx, sy) the anchor and `prop` use (designer / uniform: median 1.03 below a 0.2 aspect change, 1.24 from 0.2 to 0.7, 1.76 above 0.7). Taking the size `prop` gives (`text: "prop"`) made `learned` a re-positioned `prop` with small text (median candidate / designer font size 0.80); deriving it from the predicted box gives 0.995 (p10-p90 0.76-1.28, valid candidates above rung 0; `prop` 0.75, anchor 0.78, lanes 1.03 with a wider spread 0.55-1.48; the last-resort rung 0 keeps the `prop` size, 0.77). Measured on the 1,008 test-family pairs, pick distance 0.2522 before: `prop` text 0.2460 and picked in 15%; box text 0.2354 and 29%; box tolerance 1.0 / 1.15 / 1.35: 0.2387 / 0.2354 / 0.2346 (1.15 kept; these three were measured before the rung-0 refinement). Using the prop size at the last-resort rung 0 (crowded canvases, where box-sized text overlapped) raised the validity of generated candidates from 88% to 96% at the same pick distance. These three engine settings were chosen on the 3,000 Crello pairs; the gain is the same on train and test families (below), so little of it can be overfitting, but the 1,008 test pairs were looked at while choosing them. The model itself never saw them. The samples, synthetic and holdout sets were only used to check for regressions. ## 4. Crello designer pairs: engine without and with the model (`tools/crello/layout_eval.py`) Same 3,000 pairs, same engine (local macOS), `eval_base` = the branch without `layout.npz`, `eval_learned` = with it. `dist` is `designer_eval.py`'s (all non-background elements, decorations included). **Held-out (test) families, 1,008 pairs:** | group | pairs | generated | valid | closest valid to the designer | oracle dist before → after | picked | pick dist before → after | stretch dist | picks changed | |---|---|---|---|---|---|---|---|---|---| | all | 1008 | 94% | 96% | 52.8% | 0.1860 → 0.1670 | 28.8% (290) | 0.2522 → **0.2352** | 0.1930 | 319 | | aspect change 0-0.2 | 285 | 78% | 98% | 35.4% | 0.0846 → 0.0820 | 40.4% (115) | 0.0919 → 0.0876 | 0.0587 | 121 | | aspect change 0.2-0.7 | 368 | 100% | 95% | 56.5% | 0.1629 → 0.1477 | 36.1% (133) | 0.2419 → 0.2126 | 0.1541 | 150 | | aspect change 0.7+ | 355 | 100% | 95% | 62.8% | 0.2904 → 0.2548 | 11.8% (42) | 0.3915 → 0.3773 | 0.3410 | 48 | | target strip | 129 | 91% | 93% | 42.6% | 0.2508 → 0.2312 | 24.8% (32) | 0.3265 → 0.3101 | 0.2836 | 35 | | target small | 247 | 95% | 97% | 56.3% | 0.1735 → 0.1560 | 28.7% (71) | 0.2437 → 0.2260 | 0.1678 | 74 | | target square | 120 | 94% | 93% | 45.0% | 0.1880 → 0.1752 | 26.7% (32) | 0.2410 → 0.2283 | 0.1873 | 35 | | target vertical | 273 | 93% | 96% | 55.3% | 0.1743 → 0.1504 | 31.1% (85) | 0.2238 → 0.2054 | 0.1851 | 96 | | target other | 239 | 95% | 96% | 55.6% | 0.1760 → 0.1585 | 29.3% (70) | 0.2589 → 0.2421 | 0.1819 | 79 | Pick distance gain over the 1,008 test pairs +0.0169, family-bootstrap 95% interval [+0.0132, +0.0209] (232 families); train families (1,992 pairs) +0.0162 [+0.0132, +0.0194]; all 3,000 pairs 0.2489 → 0.2325 (+0.0164 [+0.0140, +0.0190]). Where `learned` is picked, the distance of those pairs goes 0.1909 → 0.1381 (test). Where a valid `learned` candidate exists (904 of 1,008 test pairs), the closest one averages 0.1831 against `prop` 0.2077, anchor 0.2169, lanes 0.334, the stretch 0.195 and the shipped pick 0.241; it is closer than `prop` in 69% of the pairs where both are valid and closer than the stretch in 47%. The selector still ships a layout 0.04 farther from the designers than the stretch (0.2352 against 0.1930), and the oracle (0.1670) shows how much of the gap sits in selection and how much in generation: the rule score prefers lanes layouts (better readability and utilisation, worse fidelity) to the closest `learned` layout in 273 of the 332 test pairs where `learned` is closest but not picked (`prop` 40, anchor 19). Other effects, all 3,000 pairs: shipped picks valid 2,994 → 2,994, fallbacks 10 → 9; candidates per pair 6.14 → 7.62 (valid 6.09 → 7.50); mean smallest text of the pick 26.40 → 26.82 px; texts below floor per pick 0.661 → 0.613; of the 934 changed picks the smallest text is larger in 432, smaller in 151, equal in 351 (mean 23.56 → 24.93 px). Picked families: lanes 1,699 → 1,411, prop 820 → 373, anchor 469 → 356, learned 0 → 849. **With the admission guard (shipped; `eval_guard`).** Test families: picked in 23.5% (237 pairs), pick distance 0.2522 → **0.2429** (+0.0093, family bootstrap [+0.0062, +0.0127]); by aspect change 0.0919 → 0.0898 (0 to 0.2, picked 37.5%), 0.2419 → 0.2294 (0.2 to 0.7, 25.5%), 0.3915 → 0.3798 (0.7 and up, 10.1%); train families +0.0082 [+0.0062, +0.0104]; all 3,000 pairs 0.2489 → 0.2403 (+0.0086 [+0.0068, +0.0104]). The guard keeps 699 of the 849 learned picks, gives back about 45% of the unguarded gain, and turns the family into a readability gain as well: of the 699 changed picks the smallest text is larger in 392, smaller in 36, equal in 271 (mean 21.98 → 24.63 px); all pairs: mean smallest text 26.40 → 27.01 px, texts below floor per pick 0.661 → 0.617, fallbacks 10 → 9, picked families lanes 1,699 → 1,549, prop 820 → 396, anchor 469 → 345, learned 699. ## 5. Our three sets (`tools/evaluate.py --no-render --no-invariance`; local macOS, critic on) Baseline = the branch with `CHAMELEON_LEARNED=0`, which reproduces `combined5/` exactly (all 1,162 picks and scores identical). | set | hard checks | fallbacks | all text at floor | texts below floor per pair | smallest text, mean px | mean rule score | picks changed | to `learned` (smaller / larger / equal smallest text) | |---|---|---|---|---|---|---|---|---| | samples (46) | 46/46 → 46/46 | 0 → 0 | 35 → 35 | 0.913 → 0.913 | 22.91 → 22.83 | 0.9215 → 0.9227 | 8 | 5 (2 / 0 / 3) | | synthetic (271) | 271/271 → 271/271 | 0 → 0 | 178 → 178 | 1.144 → 1.125 | 19.89 → 19.82 | 0.9056 → 0.9088 | 38 | 29 (7 / 1 / 21) | | holdout (845) | 845/845 → 845/845 | 0 → 0 | 438 → 438 | 1.611 → 1.593 | 13.17 → 13.09 | 0.8899 → 0.8940 | 124 | 102 (34 / 7 / 61) | **Shipped (with the admission guard, section 3; `evals/guard`):** | set | hard checks | fallbacks | all text at floor | texts below floor per pair | smallest text, mean px | mean rule score | picks changed | to `learned` (smaller / larger / equal smallest text) | |---|---|---|---|---|---|---|---|---| | samples (46) | 46/46 → 46/46 | 0 → 0 | 35 → 35 | 0.913 → 0.913 | 22.91 → 22.91 | 0.9215 → 0.9228 | 3 | 3 (0 / 0 / 3) | | synthetic (271) | 271/271 → 271/271 | 0 → 0 | 178 → 178 | 1.144 → 1.129 | 19.89 → 19.90 | 0.9056 → 0.9074 | 21 | 21 (2 / 1 / 18) | | holdout (845) | 845/845 → 845/845 | 0 → 0 | 438 → 438 | 1.611 → 1.594 | 13.17 → 13.18 | 0.8899 → 0.8926 | 73 | 73 (13 / 6 / 54) | Every pick that changes with the guard is a learned layout (the 8 side effects revert); the 15 with smaller smallest text lose 1 px each. The table below and the rest of this section describe the family without the guard, as it was judged. No hard-check, fallback or floor-count regression. Full run with the invariance checks (determinism, rename invariance, shuffle validity) and the self-check, family on: the same picks as the numbers above, 0 self-check warnings; determinism and shuffle validity hold in every pair; rename invariance fails in 2 pairs (05_wide 728x90, h27_panel 320x50), the same two with `CHAMELEON_LEARNED=0`: pre-existing on this branch, neither pick is a `learned` layout, not touched here. Of the 136 picks that move to `learned`, 126 are the rule score's and tie-breaks' (text smaller in 38, larger in 8, equal in 80) and 10 are the critic overriding to a learned finalist (smaller 5, equal 5). Adding the family also changes the critic's finalist set: 8 further picks move to a different non-learned layout (3 with smaller text: 08_scrim 300x250 7 → 3 px, h37_split 300x250 4 → 3 px and 2480x3508 27 → 21 px), and 26 picks keep their layout and only change whether the critic or the rule chose it. Every one of the 136: `layout_picks.md`. The picks are spread over every target class (the family is generated at every aspect change above 3%). Of the 43 picks with smaller smallest text, 25 lose 1 px (9 to 14 px mostly), 4 lose 2 px, 7 lose 3 to 4 px (five of them on canvases where both layouts are under 8 px: h01_stack 300x250 5 → 2, 06_story 160x600 7 → 3, 09_partial_panel 300x250 6 → 3, L1 160x600 13 → 10, h03_stack 1080x1350 14 → 11) and 7 lose 5 to 12 px, all in large layouts where the old pick was a one-column layout (h03_stack 2480x3508 38 → 26, h35_text 2480x3508 37 → 27, h39_stack 1080x1350 19 → 13, h15_text 1080x1080 20 → 14, h30_text 1080x1920 26 → 20, h28_left 2480x3508 27 → 21, 14_rotated 2480x3508 40 → 34). The 8 larger ones: h08_text at three sizes (24 → 30, 26 → 33, 36 → 37 px), h30_text 1000x1500 16 → 20, 09_partial_panel 2480x3508 27 → 31, h26_stack 2480x3508 31 → 33, h30_text 1080x1080 14 → 16, h15_text 1080x1920 20 → 22. **Runtime.** `resize()` per task, three alternating runs each, best of three: 300 Crello pairs 19.3 s → 22.2 s (64 → 74 ms mean, +15%), the 252 grid pairs of the samples and synthetic designs 30.0 s → 30.8 s (+3%); slowest task on those grids about 2 s either way. The machine was not always quiet (the first run of each took 20 s and 32 s), so read it as roughly +3 to +15%: one or two more candidates are fitted and scored per task. `designer_eval.py`'s summed engine time is dominated by machine state (the same `CHAMELEON_LEARNED=0` run took 2.6 min once and 7.6 min later; interleaved on 800 pairs 1.8 min without the model, 2.2 min with). With `CHAMELEON_LEARNED=0` the new code reproduces every candidate, score and pick of the original 3,000 Crello records. ## 6. Visual review of changed picks (before the judge round) Ten required, fifteen looked at, old pick (left) against new pick (right) rendered with the vendored renderer (`tools/ai_judge.py:render_candidate`), by me, one reviewer, no judge model: | design, size | old → new | what I see | |---|---|---| | L1 1080x1350 | lanes:1 (critic) → learned | same elements, centred stack, 37 px both; equal | | 03_badge_1 1080x1080 | lanes:1 (critic) → learned (critic) | bottle smaller, copy equal, 20 → 18 px; equal, slightly less filled | | L3 1080x1080 | prop → learned | near-identical, slightly more air between rows; equal | | 18_collage 1280x720 | lanes:2 → learned | the split (bottle left, copy right) becomes a centred stack whose bottle base touches the headline; cramped, **worse** | | 05_wide 1584x396 | lanes:3 → learned | larger headline and subhead (smallest text 10 → 9 px), tidy left copy, bottle centre, pills right; arguably **better** | | 06_story 160x600 | lanes:1 → learned (critic) | 7 px → 3 px, legal text vanishes, smaller product; **worse** | | 14_rotated 1080x1350 | anchor → learned | ribbon and rotated side label kept, a little more air; equal | | h15_text 1080x1080 | lanes:1 → learned | the one-column stack that fills the canvas becomes a block in the top half with 14 px pills and an empty lower half; **worse** | | h39_stack 1080x1350 | lanes:1 → learned | bigger bottle, content biased up, label 19 → 13 px; equal | | h03_stack 1080x1350 | lanes:1 → learned | content moved up, 14 → 11 px, ribbon over a circle; equal to slightly **worse** | | h10_stack 1080x1350 | lanes:1 → learned:0.85 | larger headline, fills a little better, 10 → 9 px; slightly **better** | | h04_split 1500x500 | lanes:2 → learned | keeps the source arrangement (headline and copy top left, bottle right), 13 → 12 px; equal | | h38_left 728x90 | lanes:1 → learned:0.55 (critic) | subhead 8 → 6 px under a two-line headline, rule score 0.832 → 0.782; **worse** | | h01_stack 300x250 | lanes:1 → learned | both unreadable, legal text 5 → 2 px; **worse** | | h23_split 1080x1920 | lanes:1+row2 (critic) → learned | bottle placed beside the copy block instead of floating low right; **better** | Three better, six equal, six worse. The pattern behind the worse ones: (1) at very narrow or very short canvases the uniform scale gives 2 to 3 px text and the lanes layouts it replaces reached 5 to 8 px; (2) the designers' habit of keeping relative positions leaves the lower half of a taller canvas empty, where the one-column layouts the judge preferred fill it; (3) a collage whose parts overlap in the source stays overlapped. The first and third are what the earlier judge rounds penalised. The rule score picks `learned` there because its fidelity term (weight 0.70, tuned on these same designer pairs) is higher (fidelity 0.50 → 0.91 on h15_text 1080x1080, 0.60 → 0.96 on 18_collage 1280x720), outweighing utilisation and readability. **What to do with it.** A guard that a `learned` pick may not lower the smallest text (offline simulation on the Crello picks, reverting to the old pick when the learned pick's smallest text is 1 px smaller or more) removes 47 of 290 learned picks on test families and keeps +0.0123 of the +0.0169 pick distance gain (a 2 px guard: 24 removed, +0.0139); on our sets it would remove the 43 picks with smaller text. That is a selector rule, which needs its own judge round (section 5 of `docs/crello-gains-2026-10-26.md` is the protocol: render old against new for every changed pick, both letter orders). The round was run (6b) and the rule, with a utilization axis added, is the guard. ## 6b. Judge round 6: old pick against new pick, and the admission guard (2026-10-27) Protocol of rounds 4 and 5 (`docs/crello-gains-2026-10-26.md` §5): every pick that changes on the three sets with the family on and no guard (144: 136 to a `learned` layout, 8 to another layout through the critic's finalist set) rendered old against new at 1:1 (`tools/judge/build_change_set6.py`), two sheets per pair with the letters swapped, each sheet judged once by a strong vision model (ten judges, the two orders of a pair by different judges, `tools/judge/make_lists6.py`; `JUDGE_PROMPT.md` unchanged); decided = the same winner in both orders (`tools/judge/aggregate6.py`; raw answers with reasons in `tools/judge/labels6/`). | set | comparisons | decided | new (learned family) wins | old wins | tie | inconsistent | mean confidence | |---|---|---|---|---|---|---|---| | samples | 5 | 2 | 1 | 1 | 0 | 3 | 2.7 | | synthetic | 32 | 23 | 18 | 5 | 1 | 8 | 2.7 | | holdout | 107 | 85 | 47 | 38 | 0 | 22 | 3.2 | | all | 144 | 110 | 66 | 44 | 1 | 33 | 3.1 | 66 to 44 (two-sided binomial p = 0.045) is a gain, far smaller than the critic's 67 to 13 (round 5) or the selection rules' 40 to 1 (round 4), and it splits on the two things the visual review above had named before the round: | subset of the decided comparisons | new wins | old wins | |---|---|---| | new pick is `learned` | 64 | 40 | | … chosen by the rule score / by the critic | 58 / 6 | 36 / 4 | | new pick is another layout (the critic's finalist set changed) | 2 | 4 | | smallest text equal | 44 | 16 | | smallest text larger | 3 | 3 | | smallest text 1 px smaller | 14 | 9 | | smallest text 2 px smaller | 2 | 2 | | smallest text 3 px or more smaller | 1 | 10 | | utilization score (S5) up by 0.05 or more | 38 | 6 | | utilization within 0.05 | 18 | 13 | | utilization down by 0.05 or more | 8 | 21 | | … of which tall targets (height > width) | 1 | 15 | The judges' reasons for the 44 old wins: top-heavy stacks that leave the lower half of a tall canvas empty (h15, h23, h24, h28, h30, h32, h34, h36, h39), text that goes unreadable on narrow or short canvases (06_story and L1 at 160x600, h01 and h18 at 300x250, h22 at 468x60), tag pills that touch or stagger (h07, h14, h15 1080x1920, h30), the legal line touching or crossing the CTA (h26 at four sizes: the source draws them overlapping, so every family keeps the pair as intentional layering and only the one-column layouts separate them), and the 16:9 collage that stays a portrait stack (18_collage twice). **The admission guard** (`_learned_admissible`, `TUNABLES["learned"]["guard_px"] = 1`, `guard_util = 0.05`; section 3). On this round it keeps 97 of the 144 changed picks, 71 of them decided at **54 to 17** (p = 1e-5), and reverts 47 that went 12 new, 27 old, 7 inconsistent, 1 tie. Alternatives on the same labels: the text guard alone keeps 61 to 28; text or a 0.10 utilization drop 59 to 25; 1 px or 0.10, 47 to 17. The two axes were named before the round (section 6: small text, top-heavy stacks) and the offline simulation of the text guard was in the report before any label existed; the thresholds were chosen on the round, so 54 to 17 is in-sample and a seventh round on designs neither the model nor the guard has seen is the honest check. What the guard costs on Crello is in section 4 (+0.0169 → +0.0093) and what it does on the three sets in section 5. ## 6c. Judge round 7: the guard out of sample (2026-10-28) The check section 6b asked for. 90 new designs from the holdout generator with a new seed (`tools/holdout7/`, `python tools/holdout7/build_holdout.py 90 20261028`: the same five style families as `tools/holdout/` with new copy, sizes and element mixes; nobody looked at them before the labels were in) on the holdout grid of about 21 sizes per design, 1,900 pairs. Three engine runs: family off (`CHAMELEON_LEARNED=0`), the shipped engine (guard on) and the family without its guard (`CHAMELEON_LEARNED_GUARD=0`, an evaluation-only switch added for this round). Hard checks 1,900 of 1,900 and fallbacks 0 in all three; all text at floor 996 → 1,001 (shipped) / 1,000 (no guard); mean smallest text 12.90 → 12.92 / 12.85 px; mean rule score 0.8866 → 0.8889 / 0.8900. Without the guard the family changes 168 picks (8.8%), all to learned layouts; the guard keeps 117 (6.2%) and reverts 51. Every one of the 168 was judged old against new, both letter orders, by twelve judges (the two orders of a pair by different judges; `tools/judge/build_change_set7.py` and `build_change_set7b.py`, the second batch of 60 designs added when the first 30 gave only 37 changed picks; `make_lists7*.py`; `aggregate7.py`; raw answers in `tools/judge/labels7/`). Decision rule, written down before any label existed: merge if the kept picks win at least 60% of the decided comparisons and the reverted picks win less often than the kept ones. | | comparisons | decided | new wins | old wins | new share | |---|---|---|---|---|---| | all (family without the guard) | 168 | 141 | 86 | 55 | 61% (p = 0.011) | | **kept by the guard (what ships)** | 117 | 98 | **71** | **27** | **72%** (p = 1e-5) | | reverted by the guard | 51 | 43 | 15 | 28 | 35% | Both conditions hold; the in-sample 76% of round 6 (54 to 17) becomes 72% out of sample. Kept picks by target class: small canvases (300x250, 336x280, 300x600, 160x600, 320x50, 468x60) 31 to 3, strips 7 to 1, squares 3 to 0, vertical 23 to 15, other 7 to 8. The breakdowns that produced the guard replicate on the utilization axis and only partly on the text axis: | subset of the decided comparisons | new wins | old wins | |---|---|---| | utilization score up by 0.05 or more | 61 | 13 | | within 0.05 | 17 | 20 | | down by 0.05 or more | 8 | 22 | | … of which tall targets | 3 | 13 | | smallest text equal | 59 | 20 | | smallest text larger | 6 | 9 | | 1 px smaller | 9 | 13 | | 2 px smaller | 8 | 3 | | 3 px or more smaller | 4 | 10 | | reverted for utilization alone | 3 | 15 | | reverted for text alone | 7 | 6 | | reverted for both | 5 | 7 | The utilization guard does the work (3 to 15 where it alone reverts); the text guard alone is a wash out of sample (7 to 6) and costs a few wins at exactly 2 px (8 to 3) while still removing the 3-px-and-more losses (4 to 10). It stays as judged: the shipped rule is the one these labels measured, and loosening it to 2 px would be a new in-sample choice. Caveats: the same generator as the old holdout (unseen instances, not unseen kinds of design); the judge is the model family whose labels trained the critic (so this is an independent check on the family and the guard, not on the critic); local CoreText text measurement. ## 7. Tests `tests/test_learned.py` (12 tests): features (shape, determinism, rename invariance, target enters only through aspect and size); inert without weights (absent, corrupt, wrong names, wrong shapes, non-finite: no candidates, `resize()` byte-identical to `CHAMELEON_LEARNED=0` on three samples at four sizes); the env switch and tunable; not generated within ±3%; a zero-output model reproduces the stretch's boxes through the whole plumbing; with the shipped file, `learned` exists on L2 at 1600x400 and 1080x1920, is complete, deterministic, on the canvas, unique, appended last with every older candidate unchanged, valid on L2 1080x1920 (at 1600x400 only the last-resort rung exists and it fails H8); the file is ≤ 200 KB, loads without pickle and predicts finite boxes on three samples at ten sizes; every learned candidate on that grid fits without errors and keeps every id; degenerate and non-finite designs never raise; a failing model is skipped. Four older tests that pin the older families' behaviour or compare with a models directory that is also missing `layout.npz` now set `CHAMELEON_LEARNED=0`; one (`test_proportional_candidates_exist...`) ignores `learned` candidates. Three guard tests (`LearnedGuardTests`): the thresholds on stand-in picks; on the three samples at ten sizes the output with the family differs from `CHAMELEON_LEARNED=0` only where the log says a learned layout shipped (and it ships at least once); with guards that always revert, the two outputs are identical everywhere. Results: all 66 tests pass (51 existing + 15 new; `tests/test_layout.py` and `python -m unittest discover -s tests -p "test_*.py"`, as CI runs them), and `python -m engine.main` on the three samples gives byte-identical JSON in two runs with no fallback or self-check warning. ## 8. Caveats * **The guard's thresholds were chosen on round 6 and checked out of sample in round 7** (section 6c: 90 fresh designs, kept picks 71 to 27, reverted 15 to 28). The text axis alone did not replicate (7 to 6); the utilization axis did (3 to 15). The Crello data says designers shrink text; the judge does not. * **Crello is not the organizers' briefs.** Geometry of VistaCreate templates (grey boxes for art, synthetic words, Solstice fonts); nothing here was checked on a brand banner that is not one of our 3 + 18 + 40 designs. * **The critic had never seen these layouts**; of the 136 unguarded learned picks, 10 are its overrides (judged 6 to 4). The 8 other picks its finalist set moved (judged 2 to 4) revert through the guard. * **Selection, not generation, is the bottleneck for designer closeness**: oracle 0.1670 against pick 0.2352 on test families. * **Local macOS only** (CoreText); the Docker (FreeType) run, which differs by up to ~8% in text width at small sizes (R15), was not repeated, and `tools/evaluate.py` numbers in CI may differ by a few picks. * **Trained on train families only** (915 of 1,316 families); retraining on all of them is one flag away but would remove the held-out numbers. * **Validity**: 95% of generated candidates pass every hard check (91% at aspect changes above 0.7); invalid ones are last-resort layouts with text overlapping text and are never shipped. * The sklearn MLP in the table is trained with squared loss because that is all `MLPRegressor` offers; it is the weakest row for a reason that has nothing to do with network size. ## 9. Commands ```bash .venv/bin/python tools/crello/layout_dataset.py OMP_NUM_THREADS=4 /opt/anaconda3/bin/python3 tools/crello/train_layout.py --export engine/models/layout.npz --report output/crello/layout_train.md # add --skip-sklearn to save 2 min CHAMELEON_LEARNED=0 .venv/bin/python tools/crello/designer_eval.py --workers 8 --out output/crello/eval_base .venv/bin/python tools/crello/designer_eval.py --workers 8 --out output/crello/eval_learned .venv/bin/python tools/crello/layout_eval.py --base output/crello/eval_base/candidates.jsonl --new output/crello/eval_learned/candidates.jsonl CHAMELEON_LEARNED=0 .venv/bin/python tools/evaluate.py --set samples --no-render --no-invariance --summary-md /tmp/base_samples.md # likewise synthetic, holdout .venv/bin/python tools/evaluate.py --set samples --no-render --no-invariance --summary-md /tmp/new_samples.md .venv/bin/python tools/crello/layout_picks.py --base --new --out tools/crello/layout_picks.md # the guard: CHAMELEON_LEARNED=0 vs the shipped engine is the "shipped" table of section 5; `eval_guard` in section 4 is designer_eval.py with the guard on .venv/bin/python tests/test_layout.py && .venv/bin/python -m unittest discover -s tests -p "test_*.py" ``` Files: `engine/resize.py` (features, `_learned_size`, `_learned_candidates`, `_anchor_try(pred=)`, loader `_layout_model`, `_layout_predict`), `engine/models/layout.npz`, `tools/crello/{layout_dataset,train_layout,layout_eval}.py`, `tests/test_learned.py`, this report, `LAYOUT_MODEL_CARD.md`, `layout_picks.md`, `docs/engine.md` (B29, section 6), `tools/crello/README.md`.