Buckets:
evalstate/birch-html / results /publish /models /gemini35flash /artifacts /benchmark-comparison.html
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8" /> | |
| <meta name="viewport" content="width=device-width, initial-scale=1" /> | |
| <title>Promote Candidate B — Birch HTML skill seed comparison</title> | |
| <style> | |
| /* Page-local refinements, Birch tokens only. */ | |
| .hero-callout { | |
| display: grid; | |
| gap: var(--space-3); | |
| grid-template-columns: 1fr; | |
| padding: var(--space-4); | |
| border: 1px solid var(--border-color); | |
| border-radius: 14px; | |
| background: var(--surface-warm); | |
| } | |
| @media (min-width: 720px) { | |
| .hero-callout { grid-template-columns: minmax(0, 1.4fr) minmax(0, 1fr); align-items: center; } | |
| } | |
| .decision-pill { | |
| display: inline-flex; align-items: center; gap: var(--space-1); | |
| padding: 2px 10px; border-radius: 999px; | |
| background: color-mix(in oklab, var(--success) 18%, var(--surface)); | |
| color: var(--success); font-weight: 600; font-size: 0.78rem; letter-spacing: 0.04em; | |
| text-transform: uppercase; | |
| } | |
| .delta-up { color: var(--success); font-weight: 600; } | |
| .delta-down { color: var(--danger); font-weight: 600; } | |
| .delta-warn { color: var(--accent); font-weight: 600; } | |
| .chart-svg svg { width: 100%; height: auto; } | |
| .formula { | |
| font-family: ui-monospace, SFMono-Regular, Menlo, monospace; | |
| font-size: 0.92rem; | |
| } | |
| .kpi-strong { font-variant-numeric: tabular-nums; } | |
| </style> | |
| <style data-birch-system> | |
| /* | |
| Birch HTML Design System | |
| Opinionated single-option stylesheet for reliable LLM-generated artifacts. | |
| Model: | |
| 1. Design tokens as CSS variables. | |
| 2. Layout primitives as classes. | |
| 3. Semantic components as classes with data-* variants. | |
| */ | |
| /* ==================== 1. TOKENS ==================== */ | |
| :root { | |
| color-scheme: light; | |
| /* Palette */ | |
| --color-ivory: #FAF9F5; | |
| --color-slate: #141413; | |
| --color-clay: #D97757; | |
| --color-clay-dark: #B85C3E; | |
| --color-oat: #E3DACC; | |
| --color-olive: #788C5D; | |
| --color-rust: #B04A3F; | |
| --color-sky: #6A8CAF; | |
| --color-white: #FFFFFF; | |
| --color-gray-50: #F7F5EE; | |
| --color-gray-100: #F0EEE6; | |
| --color-gray-150: #E8E4DA; | |
| --color-gray-200: #DED9CD; | |
| --color-gray-300: #D1CFC5; | |
| --color-gray-500: #87867F; | |
| --color-gray-700: #3D3D3A; | |
| --color-gray-800: #242421; | |
| /* Semantic colors */ | |
| --bg: var(--color-ivory); | |
| --surface: var(--color-white); | |
| --surface-tint: var(--color-gray-100); | |
| --surface-warm: var(--color-oat); | |
| --text: var(--color-slate); | |
| --text-muted: var(--color-gray-500); | |
| --text-soft: var(--color-gray-700); | |
| --muted: var(--text-muted); | |
| --border-color: var(--color-gray-300); | |
| --accent: var(--color-clay); | |
| --accent-strong: var(--color-clay-dark); | |
| --success: var(--color-olive); | |
| --warning: #C78E3F; | |
| --danger: var(--color-rust); | |
| --info: var(--color-sky); | |
| /* Typography */ | |
| --font-serif: ui-serif, Georgia, "Times New Roman", serif; | |
| --font-sans: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif; | |
| --font-mono: ui-monospace, "SF Mono", Menlo, Consolas, monospace; | |
| --text-xs: 11px; | |
| --text-sm: 12px; | |
| --text-md: 14px; | |
| --text-base: 15px; | |
| --text-lg: 17px; | |
| --text-xl: 21px; | |
| --text-2xl: 26px; | |
| --text-3xl: 36px; | |
| --text-4xl: 44px; | |
| --leading-tight: 1.15; | |
| --leading-title: 1.25; | |
| --leading-body: 1.55; | |
| --leading-loose: 1.7; | |
| /* Spacing */ | |
| --space-1: 4px; | |
| --space-2: 8px; | |
| --space-3: 12px; | |
| --space-4: 16px; | |
| --space-5: 24px; | |
| --space-6: 32px; | |
| --space-7: 48px; | |
| --space-8: 64px; | |
| --space-9: 96px; | |
| /* Shape */ | |
| --radius-xs: 4px; | |
| --radius-sm: 8px; | |
| --radius-md: 12px; | |
| --radius-lg: 16px; | |
| --radius-xl: 20px; | |
| --radius-pill: 999px; | |
| --border: 1.5px solid var(--border-color); | |
| --border-thin: 1px solid var(--border-color); | |
| --shadow-sm: 0 1px 2px rgba(20, 20, 19, 0.06); | |
| --shadow-md: 0 4px 14px rgba(20, 20, 19, 0.08); | |
| --shadow-lg: 0 12px 28px rgba(20, 20, 19, 0.12); | |
| /* Motion */ | |
| --ease: cubic-bezier(0.2, 0, 0, 1); | |
| --ease-spring: cubic-bezier(0.34, 1.56, 0.64, 1); | |
| --dur-fast: 120ms; | |
| --dur: 180ms; | |
| --hover-y: -2px; | |
| /* Page sizing */ | |
| --page-max: 1100px; | |
| --page-pad: 28px; | |
| --page-pad-block: 48px; | |
| /* Legacy aliases used by source examples */ | |
| --ivory: var(--color-ivory); | |
| --slate: var(--color-slate); | |
| --clay: var(--color-clay); | |
| --clay-d: var(--color-clay-dark); | |
| --oat: var(--color-oat); | |
| --olive: var(--color-olive); | |
| --rust: var(--color-rust); | |
| --sky: var(--color-sky); | |
| --white: var(--color-white); | |
| --gray-50: var(--color-gray-50); | |
| --gray-100: var(--color-gray-100); | |
| --gray-150: var(--color-gray-150); | |
| --gray-200: var(--color-gray-200); | |
| --gray-300: var(--color-gray-300); | |
| --gray-500: var(--color-gray-500); | |
| --gray-700: var(--color-gray-700); | |
| --gray-800: var(--color-gray-800); | |
| --serif: var(--font-serif); | |
| --sans: var(--font-sans); | |
| --mono: var(--font-mono); | |
| } | |
| /* ==================== BASE ==================== */ | |
| *, | |
| *::before, | |
| *::after { | |
| box-sizing: border-box; | |
| } | |
| html { | |
| background: var(--bg); | |
| color: var(--text); | |
| font-family: var(--font-sans); | |
| -webkit-text-size-adjust: 100%; | |
| } | |
| body { | |
| margin: 0; | |
| background: var(--bg); | |
| color: var(--text); | |
| font-family: var(--font-sans); | |
| font-size: var(--text-base); | |
| line-height: var(--leading-body); | |
| -webkit-font-smoothing: antialiased; | |
| text-rendering: optimizeLegibility; | |
| } | |
| img, | |
| svg { | |
| max-width: 100%; | |
| } | |
| a { | |
| color: inherit; | |
| text-decoration-color: rgba(217, 119, 87, 0.45); | |
| text-underline-offset: 3px; | |
| } | |
| a:hover { | |
| color: var(--accent); | |
| } | |
| h1, | |
| h2, | |
| h3, | |
| p { | |
| margin-block: 0; | |
| } | |
| h1, | |
| h2, | |
| h3 { | |
| font-family: var(--font-serif); | |
| font-weight: 500; | |
| letter-spacing: -0.01em; | |
| line-height: var(--leading-title); | |
| } | |
| h1 { | |
| font-size: clamp(32px, 5vw, var(--text-4xl)); | |
| } | |
| h2 { | |
| font-size: var(--text-2xl); | |
| } | |
| h3 { | |
| font-size: var(--text-xl); | |
| } | |
| code, | |
| kbd, | |
| pre { | |
| font-family: var(--font-mono); | |
| } | |
| code { | |
| font-size: 0.92em; | |
| white-space: normal; | |
| word-break: normal; | |
| } | |
| h1 code, | |
| h2 code, | |
| h3 code { | |
| overflow-wrap: anywhere; | |
| word-break: break-word; | |
| } | |
| button, | |
| input, | |
| textarea, | |
| select { | |
| font: inherit; | |
| } | |
| button { | |
| cursor: pointer; | |
| } | |
| ::selection { | |
| background: rgba(217, 119, 87, 0.24); | |
| } | |
| /* ==================== 2. LAYOUT PRIMITIVES ==================== */ | |
| .page { | |
| width: min(100% - (var(--page-pad) * 2), var(--page-max)); | |
| margin-inline: auto; | |
| padding-block: var(--page-pad-block) var(--space-9); | |
| } | |
| .page[data-size="narrow"] { | |
| --page-max: 820px; | |
| } | |
| .page[data-size="wide"] { | |
| --page-max: 1280px; | |
| } | |
| .page[data-size="full"] { | |
| --page-max: 1440px; | |
| } | |
| .section { | |
| margin-block: var(--section-gap, var(--space-5)); | |
| } | |
| .section[data-gap="lg"] { | |
| --section-gap: var(--space-6); | |
| } | |
| .section[data-gap="xl"] { | |
| --section-gap: var(--space-7); | |
| } | |
| .section:first-child { | |
| margin-block-start: 0; | |
| } | |
| .stack > .section { | |
| margin-block: 0; | |
| } | |
| .rule { | |
| border: 0; | |
| border-top: var(--border-thin); | |
| margin: var(--space-3) 0 var(--space-5); | |
| } | |
| .stack { | |
| display: flex; | |
| flex-direction: column; | |
| gap: var(--stack-gap, var(--space-4)); | |
| } | |
| .stack[data-gap="xs"] { --stack-gap: var(--space-2); } | |
| .stack[data-gap="sm"] { --stack-gap: var(--space-3); } | |
| .stack[data-gap="md"] { --stack-gap: var(--space-4); } | |
| .stack[data-gap="lg"] { --stack-gap: var(--space-6); } | |
| .stack[data-gap="xl"] { --stack-gap: var(--space-8); } | |
| .stack > :is(.button, .btn, .chip, .badge) { | |
| align-self: flex-start; | |
| } | |
| .cluster { | |
| display: flex; | |
| flex-wrap: wrap; | |
| align-items: center; | |
| gap: var(--cluster-gap, var(--space-3)); | |
| } | |
| .cluster[data-align="start"] { align-items: flex-start; } | |
| .cluster[data-align="end"] { align-items: flex-end; } | |
| .cluster[data-justify="between"] { justify-content: space-between; } | |
| .cluster[data-justify="end"] { justify-content: flex-end; } | |
| .grid { | |
| display: grid; | |
| gap: var(--grid-gap, var(--space-5)); | |
| } | |
| .grid[data-cols="2"] { | |
| grid-template-columns: repeat(2, minmax(0, 1fr)); | |
| } | |
| .grid[data-cols="3"] { | |
| grid-template-columns: repeat(3, minmax(0, 1fr)); | |
| } | |
| .grid[data-cols="4"] { | |
| grid-template-columns: repeat(4, minmax(0, 1fr)); | |
| } | |
| .auto-grid { | |
| display: grid; | |
| grid-template-columns: repeat(auto-fit, minmax(min(100%, var(--grid-min, 240px)), 1fr)); | |
| gap: var(--grid-gap, var(--space-5)); | |
| } | |
| .auto-grid > * { | |
| min-width: 0; | |
| } | |
| .split { | |
| display: grid; | |
| grid-template-columns: minmax(0, 1fr) minmax(280px, var(--split-side, 360px)); | |
| gap: var(--split-gap, var(--space-6)); | |
| align-items: start; | |
| } | |
| .split > * { | |
| min-width: 0; | |
| } | |
| .section-head { | |
| min-width: 0; | |
| } | |
| .section-rail { | |
| display: grid; | |
| grid-template-columns: minmax(0, 1fr) minmax(280px, var(--rail-width, 360px)); | |
| column-gap: var(--rail-gap, var(--space-6)); | |
| row-gap: var(--space-5); | |
| align-items: start; | |
| } | |
| .section-rail > .section-head { | |
| grid-column: 1 / -1; | |
| } | |
| .section-rail > .reference-panel, | |
| .section-rail > aside { | |
| align-self: start; | |
| } | |
| .sidebar-layout { | |
| display: grid; | |
| grid-template-columns: var(--sidebar-width, 240px) minmax(0, 1fr); | |
| gap: var(--sidebar-gap, var(--space-7)); | |
| align-items: start; | |
| } | |
| .sidebar-layout > :first-child, | |
| .sticky { | |
| position: sticky; | |
| top: var(--sticky-top, var(--space-5)); | |
| } | |
| .bleed { | |
| margin-inline: calc(var(--page-pad) * -1); | |
| } | |
| .scroll-x { | |
| overflow-x: auto; | |
| overscroll-behavior-inline: contain; | |
| } | |
| .center { | |
| display: grid; | |
| place-items: center; | |
| } | |
| .spacer { | |
| flex: 1; | |
| } | |
| @media (max-width: 860px) { | |
| :root { | |
| --page-pad: 20px; | |
| --page-pad-block: 40px; | |
| } | |
| .grid[data-cols], | |
| .split, | |
| .section-rail, | |
| .sidebar-layout { | |
| grid-template-columns: 1fr; | |
| } | |
| .sidebar-layout > :first-child, | |
| .sticky { | |
| position: static; | |
| } | |
| } | |
| /* ==================== TYPOGRAPHY UTILITIES ==================== */ | |
| .eyebrow { | |
| display: inline-flex; | |
| align-items: center; | |
| gap: var(--space-2); | |
| color: var(--accent); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-xs); | |
| font-weight: 700; | |
| letter-spacing: 0.08em; | |
| text-transform: uppercase; | |
| } | |
| .eyebrow::before { | |
| content: ""; | |
| width: 24px; | |
| height: 1.5px; | |
| background: currentColor; | |
| } | |
| .lede { | |
| max-width: 720px; | |
| color: var(--text-soft); | |
| font-size: var(--text-lg); | |
| line-height: 1.6; | |
| } | |
| .muted { | |
| color: var(--text-muted); | |
| } | |
| .caption { | |
| color: var(--text-muted); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-sm); | |
| } | |
| .stat-value { | |
| display: block; | |
| color: var(--text); | |
| font-family: var(--font-serif); | |
| font-size: clamp(30px, 3.2vw, 42px); | |
| font-variant-numeric: tabular-nums; | |
| font-weight: 550; | |
| letter-spacing: -0.025em; | |
| line-height: 1; | |
| } | |
| .stat-card { | |
| gap: var(--space-2); | |
| } | |
| .stat-card .caption { | |
| letter-spacing: 0.08em; | |
| text-transform: uppercase; | |
| } | |
| .stat-card .muted { | |
| max-width: 34ch; | |
| font-size: var(--text-sm); | |
| } | |
| .mono { | |
| font-family: var(--font-mono); | |
| } | |
| /* ==================== 3. SEMANTIC COMPONENTS ==================== */ | |
| .panel, | |
| .card { | |
| background: var(--surface); | |
| border: var(--border); | |
| border-radius: var(--radius-md); | |
| } | |
| .panel { | |
| padding: var(--panel-pad, var(--space-5)); | |
| } | |
| .card { | |
| padding: var(--card-pad, var(--space-5)); | |
| transition: | |
| transform var(--dur-fast) var(--ease), | |
| border-color var(--dur-fast) var(--ease), | |
| box-shadow var(--dur-fast) var(--ease), | |
| background-color var(--dur-fast) var(--ease); | |
| } | |
| .card[data-hover="lift"]:hover, | |
| .card[data-clickable]:hover { | |
| transform: translateY(var(--hover-y)); | |
| border-color: rgba(217, 119, 87, 0.55); | |
| box-shadow: var(--shadow-md); | |
| } | |
| .card[data-clickable] { | |
| cursor: pointer; | |
| } | |
| .card[data-variant="flat"] { | |
| background: transparent; | |
| border-color: transparent; | |
| box-shadow: none; | |
| } | |
| .card[data-variant="outlined"] { | |
| background: var(--surface); | |
| border: var(--border); | |
| } | |
| .card[data-variant="elevated"] { | |
| border-color: transparent; | |
| box-shadow: var(--shadow-md); | |
| } | |
| .card[data-variant="filled"], | |
| .card[data-tone="oat"] { | |
| background: var(--surface-warm); | |
| } | |
| .card[data-tone], | |
| .panel[data-tone], | |
| .callout[data-tone] { | |
| background: var(--surface); | |
| } | |
| .card[data-tone="danger"], | |
| .panel[data-tone="danger"], | |
| .callout[data-tone="danger"] { | |
| border-color: rgba(176, 74, 63, 0.36); | |
| } | |
| .card[data-tone="warning"], | |
| .panel[data-tone="warning"], | |
| .callout[data-tone="warning"] { | |
| border-color: rgba(199, 142, 63, 0.38); | |
| } | |
| .card[data-tone="success"], | |
| .panel[data-tone="success"], | |
| .callout[data-tone="success"] { | |
| border-color: rgba(120, 140, 93, 0.38); | |
| } | |
| .card[data-tone="info"], | |
| .panel[data-tone="info"], | |
| .callout[data-tone="info"] { | |
| border-color: rgba(106, 140, 175, 0.36); | |
| } | |
| .card[data-accent] { | |
| position: relative; | |
| overflow: hidden; | |
| } | |
| .card[data-accent]::before { | |
| content: ""; | |
| position: absolute; | |
| inset: 0 auto 0 0; | |
| width: 5px; | |
| background: var(--accent); | |
| } | |
| .card-head { | |
| display: flex; | |
| align-items: center; | |
| gap: var(--space-3); | |
| min-width: 0; | |
| max-width: 100%; | |
| } | |
| .card-head > :first-child { | |
| flex: 0 0 auto; | |
| } | |
| .card-titles { | |
| flex: 1 1 0; | |
| min-width: 0; | |
| max-width: 100%; | |
| } | |
| .card-title { | |
| max-width: 100%; | |
| margin: 0; | |
| font-family: var(--font-serif); | |
| font-size: var(--text-xl); | |
| font-weight: 550; | |
| line-height: var(--leading-title); | |
| overflow-wrap: anywhere; | |
| } | |
| .card-sub { | |
| margin: 2px 0 0; | |
| color: var(--text-muted); | |
| font-size: var(--text-sm); | |
| overflow-wrap: anywhere; | |
| } | |
| .toolbar { | |
| display: flex; | |
| flex-wrap: wrap; | |
| align-items: center; | |
| gap: var(--space-4); | |
| padding: var(--space-4) var(--space-5); | |
| background: var(--surface); | |
| border: var(--border); | |
| border-radius: var(--radius-md); | |
| } | |
| .toolbar[data-sticky] { | |
| position: sticky; | |
| top: 0; | |
| z-index: 10; | |
| } | |
| .chip, | |
| .badge { | |
| display: inline-flex; | |
| align-items: center; | |
| justify-content: center; | |
| gap: 6px; | |
| border-radius: var(--radius-pill); | |
| white-space: nowrap; | |
| font-family: var(--font-mono); | |
| font-size: var(--text-xs); | |
| font-weight: 650; | |
| line-height: 1; | |
| } | |
| .chip { | |
| min-height: 24px; | |
| padding: 0 10px; | |
| color: var(--text-soft); | |
| background: var(--surface-tint); | |
| border: 1px solid transparent; | |
| } | |
| .badge { | |
| min-height: 22px; | |
| padding: 0 9px; | |
| color: var(--text-muted); | |
| background: var(--surface); | |
| border: var(--border-thin); | |
| text-transform: uppercase; | |
| letter-spacing: 0.06em; | |
| } | |
| .chip[data-tone="clay"], | |
| .badge[data-tone="clay"] { | |
| color: var(--accent); | |
| background: rgba(217, 119, 87, 0.14); | |
| } | |
| .chip[data-tone="olive"], | |
| .badge[data-tone="olive"], | |
| .chip[data-tone="success"], | |
| .badge[data-tone="success"] { | |
| color: var(--success); | |
| background: rgba(120, 140, 93, 0.15); | |
| } | |
| .chip[data-tone="warning"], | |
| .badge[data-tone="warning"] { | |
| color: var(--warning); | |
| background: rgba(199, 142, 63, 0.16); | |
| } | |
| .chip[data-tone="rust"], | |
| .badge[data-tone="rust"], | |
| .chip[data-tone="danger"], | |
| .badge[data-tone="danger"] { | |
| color: var(--danger); | |
| background: rgba(176, 74, 63, 0.13); | |
| } | |
| .chip[data-tone="sky"], | |
| .badge[data-tone="sky"], | |
| .chip[data-tone="info"], | |
| .badge[data-tone="info"] { | |
| color: var(--info); | |
| background: rgba(106, 140, 175, 0.14); | |
| } | |
| .finding { | |
| border-left: 6px solid var(--border-color); | |
| } | |
| .finding[data-severity="blocker"], | |
| .finding[data-severity="critical"], | |
| .finding[data-severity="danger"], | |
| .finding[data-tone="danger"] { | |
| border-left-color: var(--danger); | |
| } | |
| .finding[data-severity="high"], | |
| .finding[data-severity="warning"], | |
| .finding[data-tone="warning"] { | |
| border-left-color: var(--warning); | |
| } | |
| .finding[data-severity="medium"], | |
| .finding[data-severity="info"], | |
| .finding[data-tone="info"] { | |
| border-left-color: var(--info); | |
| } | |
| .finding[data-severity="low"], | |
| .finding[data-severity="success"], | |
| .finding[data-tone="success"] { | |
| border-left-color: var(--success); | |
| } | |
| .button, | |
| .btn { | |
| appearance: none; | |
| display: inline-flex; | |
| align-items: center; | |
| justify-content: center; | |
| min-height: 36px; | |
| padding: 0 14px; | |
| border: 1.5px solid transparent; | |
| border-radius: var(--radius-sm); | |
| font-family: var(--font-sans); | |
| font-size: var(--text-md); | |
| font-weight: 650; | |
| line-height: 1; | |
| text-decoration: none; | |
| transition: | |
| transform var(--dur-fast) var(--ease), | |
| background-color var(--dur-fast) var(--ease), | |
| border-color var(--dur-fast) var(--ease), | |
| box-shadow var(--dur-fast) var(--ease); | |
| } | |
| .button:hover, | |
| .btn:hover { | |
| transform: translateY(-1px); | |
| } | |
| .button[data-variant="primary"], | |
| .btn-primary { | |
| color: var(--color-white); | |
| background: var(--color-slate); | |
| } | |
| .button[data-variant="primary"]:hover, | |
| .btn-primary:hover { | |
| background: var(--color-gray-700); | |
| } | |
| .button[data-variant="secondary"], | |
| .btn-secondary { | |
| color: var(--text); | |
| background: var(--surface-tint); | |
| border-color: var(--border-color); | |
| } | |
| .button[data-variant="ghost"], | |
| .btn-ghost { | |
| color: var(--text-soft); | |
| background: transparent; | |
| } | |
| .button[data-variant="ghost"]:hover, | |
| .btn-ghost:hover { | |
| color: var(--text); | |
| background: var(--surface-tint); | |
| } | |
| .button[data-variant="danger"], | |
| .btn-danger { | |
| color: var(--color-white); | |
| background: var(--danger); | |
| } | |
| .input { | |
| width: 100%; | |
| min-height: 38px; | |
| padding: 0 12px; | |
| color: var(--text); | |
| background: var(--surface); | |
| border: var(--border); | |
| border-radius: var(--radius-sm); | |
| outline: none; | |
| } | |
| .input:focus { | |
| border-color: var(--accent); | |
| box-shadow: 0 0 0 3px rgba(217, 119, 87, 0.16); | |
| } | |
| .avatar { | |
| display: inline-flex; | |
| align-items: center; | |
| justify-content: center; | |
| width: var(--avatar-size, 38px); | |
| height: var(--avatar-size, 38px); | |
| border-radius: 50%; | |
| color: var(--text-soft); | |
| background: var(--surface-warm); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-sm); | |
| font-weight: 700; | |
| letter-spacing: 0.02em; | |
| flex: 0 0 auto; | |
| } | |
| .code-block { | |
| margin: 0; | |
| padding: var(--space-4) var(--space-5); | |
| overflow-x: auto; | |
| color: #E8E6DC; | |
| background: var(--color-slate); | |
| border-radius: var(--radius-md); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-sm); | |
| line-height: var(--leading-loose); | |
| } | |
| .code-block[data-kind="command"], | |
| .code-block[data-wrap="true"], | |
| .command-block { | |
| max-width: 100%; | |
| min-width: 0; | |
| box-sizing: border-box; | |
| white-space: pre-wrap; | |
| overflow-wrap: anywhere; | |
| word-break: break-word; | |
| overflow-x: hidden; | |
| } | |
| .code-block[data-wrap="true"] code, | |
| .code-block[data-kind="command"] code { | |
| display: block; | |
| max-width: 100%; | |
| white-space: pre-wrap; | |
| overflow-wrap: anywhere; | |
| word-break: break-word; | |
| } | |
| .copyable { | |
| position: relative; | |
| } | |
| .copyable > .code-block { | |
| padding-inline-end: 88px; | |
| } | |
| .copy-button { | |
| appearance: none; | |
| position: absolute; | |
| inset-block-start: var(--space-2); | |
| inset-inline-end: var(--space-2); | |
| display: inline-flex; | |
| align-items: center; | |
| justify-content: center; | |
| min-height: 26px; | |
| padding: 0 9px; | |
| border: 1px solid rgba(250, 249, 245, 0.18); | |
| border-radius: var(--radius-sm); | |
| color: var(--color-gray-100); | |
| background: rgba(255, 255, 255, 0.08); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-xs); | |
| font-weight: 700; | |
| letter-spacing: 0.04em; | |
| text-transform: uppercase; | |
| transition: | |
| background-color var(--dur-fast) var(--ease), | |
| border-color var(--dur-fast) var(--ease), | |
| transform var(--dur-fast) var(--ease); | |
| } | |
| .copy-button:hover { | |
| transform: translateY(-1px); | |
| border-color: rgba(250, 249, 245, 0.35); | |
| background: rgba(255, 255, 255, 0.14); | |
| } | |
| .copy-button[data-copied="true"] { | |
| color: var(--color-white); | |
| background: rgba(120, 140, 93, 0.55); | |
| border-color: rgba(120, 140, 93, 0.75); | |
| } | |
| .reference-panel { | |
| --panel-pad: var(--space-5); | |
| } | |
| .callout { | |
| padding: var(--space-5); | |
| color: var(--text-soft); | |
| background: var(--surface-tint); | |
| border: var(--border); | |
| border-inline-start: 4px solid var(--accent); | |
| border-radius: var(--radius-md); | |
| } | |
| .callout[data-tone="success"] { | |
| border-inline-start-color: var(--success); | |
| background: rgba(120, 140, 93, 0.10); | |
| } | |
| .callout[data-tone="warning"] { | |
| border-inline-start-color: var(--warning); | |
| background: rgba(199, 142, 63, 0.11); | |
| } | |
| .callout[data-tone="danger"] { | |
| border-inline-start-color: var(--danger); | |
| background: rgba(176, 74, 63, 0.10); | |
| } | |
| .callout[data-tone="info"] { | |
| border-inline-start-color: var(--info); | |
| background: rgba(106, 140, 175, 0.11); | |
| } | |
| .callout-label { | |
| display: block; | |
| margin-bottom: var(--space-2); | |
| color: var(--text-muted); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-xs); | |
| font-weight: 700; | |
| letter-spacing: 0.09em; | |
| text-transform: uppercase; | |
| } | |
| .callout > :last-child { | |
| margin-bottom: 0; | |
| } | |
| .checklist { | |
| list-style: none; | |
| margin: 0; | |
| padding: 0; | |
| display: grid; | |
| gap: var(--space-3); | |
| } | |
| .checklist li { | |
| display: block ; | |
| position: relative ; | |
| min-width: 0; | |
| padding-inline-start: calc(18px + var(--space-3)); | |
| overflow-wrap: anywhere; | |
| } | |
| .checklist li::before { | |
| content: "✓"; | |
| position: absolute ; | |
| inset-block-start: 2px; | |
| inset-inline-start: 0; | |
| display: inline-flex; | |
| align-items: center; | |
| justify-content: center; | |
| width: 18px; | |
| height: 18px; | |
| border-radius: 50%; | |
| color: var(--success); | |
| background: rgba(120, 140, 93, 0.12); | |
| font-family: var(--font-mono); | |
| font-size: 12px; | |
| font-weight: 700; | |
| line-height: 1; | |
| } | |
| .checklist code { | |
| white-space: normal; | |
| overflow-wrap: anywhere; | |
| word-break: break-word; | |
| } | |
| .plain-list { | |
| margin: 0; | |
| padding-inline-start: 1.15em; | |
| color: var(--text-soft); | |
| } | |
| .plain-list li { | |
| overflow-wrap: anywhere; | |
| } | |
| .plain-list li + li { | |
| margin-top: var(--space-2); | |
| } | |
| .plain-list li::marker { | |
| color: var(--text-muted); | |
| } | |
| .insight-list, | |
| .takeaway-list { | |
| list-style: none; | |
| margin: 0; | |
| padding: 0; | |
| display: grid; | |
| gap: var(--space-3); | |
| } | |
| .insight-list li, | |
| .takeaway-list li { | |
| position: relative; | |
| display: block; | |
| padding-inline-start: calc(10px + var(--space-3)); | |
| color: var(--text-soft); | |
| } | |
| .insight-list li::before, | |
| .takeaway-list li::before { | |
| content: ""; | |
| position: absolute; | |
| inset-block-start: 0.65em; | |
| inset-inline-start: 0; | |
| width: 7px; | |
| height: 7px; | |
| border-radius: 50%; | |
| background: var(--accent); | |
| } | |
| .insight-list li > *, | |
| .takeaway-list li > * { | |
| min-width: 0; | |
| } | |
| .insight-list[data-tone="success"] li::before, | |
| .takeaway-list[data-tone="success"] li::before { | |
| background: var(--success); | |
| } | |
| .insight-list[data-tone="info"] li::before, | |
| .takeaway-list[data-tone="info"] li::before { | |
| background: var(--info); | |
| } | |
| .metric-list { | |
| --metric-label: 132px; | |
| --metric-value: 64px; | |
| display: grid; | |
| gap: var(--space-3); | |
| } | |
| .metric-row { | |
| display: grid; | |
| grid-template-columns: var(--metric-label) minmax(120px, 1fr) var(--metric-value); | |
| gap: var(--space-3); | |
| align-items: center; | |
| } | |
| .metric-row > :first-child { | |
| min-width: 0; | |
| } | |
| .metric-row > :last-child { | |
| justify-self: end; | |
| font-family: var(--font-mono); | |
| font-size: var(--text-sm); | |
| } | |
| .meter { | |
| height: 9px; | |
| overflow: hidden; | |
| border-radius: var(--radius-pill); | |
| background: var(--surface-tint); | |
| } | |
| .meter span { | |
| display: block; | |
| width: var(--value); | |
| height: 100%; | |
| background: var(--tone, var(--accent)); | |
| } | |
| .flow-list { | |
| list-style: none; | |
| margin: 0; | |
| padding: 0; | |
| display: grid; | |
| gap: var(--space-3); | |
| } | |
| .flow-step { | |
| position: relative; | |
| display: grid; | |
| grid-template-columns: 42px minmax(0, 1fr); | |
| gap: var(--space-3); | |
| align-items: start; | |
| padding: var(--space-4); | |
| background: var(--surface); | |
| border: var(--border); | |
| border-radius: var(--radius-md); | |
| } | |
| .flow-step > .flow-num { | |
| grid-column: 1; | |
| } | |
| .flow-step > :not(.flow-num) { | |
| grid-column: 2; | |
| min-width: 0; | |
| } | |
| .flow-step::after { | |
| content: ""; | |
| position: absolute; | |
| inset-block-start: calc(100% + 1px); | |
| inset-inline-start: 36px; | |
| width: 1.5px; | |
| height: var(--space-3); | |
| background: var(--border-color); | |
| } | |
| .flow-step:last-child::after { | |
| display: none; | |
| } | |
| .flow-num { | |
| display: inline-flex; | |
| align-items: center; | |
| justify-content: center; | |
| width: 34px; | |
| height: 34px; | |
| border-radius: var(--radius-pill); | |
| color: var(--text); | |
| background: var(--surface-warm); | |
| box-shadow: inset 0 0 0 1px rgba(20, 20, 19, 0.10); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-sm); | |
| font-weight: 700; | |
| } | |
| .flow-title { | |
| margin: 0 0 var(--space-1); | |
| font-family: var(--font-serif); | |
| font-size: var(--text-xl); | |
| font-weight: 550; | |
| line-height: var(--leading-title); | |
| } | |
| .flow-detail { | |
| margin: 0; | |
| color: var(--text-soft); | |
| } | |
| .flow-branch { | |
| display: grid; | |
| grid-template-columns: repeat(2, minmax(0, 1fr)); | |
| gap: var(--space-3); | |
| } | |
| .flow-branch .flow-step::after { | |
| display: none; | |
| } | |
| .flow-step[data-tone="success"] { | |
| border-color: rgba(120, 140, 93, 0.38); | |
| background: rgba(120, 140, 93, 0.08); | |
| } | |
| .flow-step[data-tone="warning"] { | |
| border-color: rgba(199, 142, 63, 0.42); | |
| background: rgba(199, 142, 63, 0.09); | |
| } | |
| @media (max-width: 620px) { | |
| .metric-row { | |
| grid-template-columns: 1fr max-content; | |
| } | |
| .metric-row .meter { | |
| grid-column: 1 / -1; | |
| order: 3; | |
| } | |
| .flow-branch { | |
| grid-template-columns: 1fr; | |
| } | |
| } | |
| /* ---------- Charts and numeric data ---------- */ | |
| .chart-panel { | |
| max-width: 100%; | |
| min-width: 0; | |
| overflow: hidden; | |
| } | |
| .chart-panel > * { | |
| min-width: 0; | |
| } | |
| .chart-panel svg, | |
| svg.chart-svg { | |
| display: block; | |
| width: 100%; | |
| height: auto; | |
| } | |
| .chart-caption { | |
| color: var(--text-muted); | |
| font-size: var(--text-sm); | |
| } | |
| .numeric-table-wrap { | |
| overflow-x: auto; | |
| background: var(--surface); | |
| border: var(--border); | |
| border-radius: var(--radius-md); | |
| } | |
| .numeric-table { | |
| width: 100%; | |
| min-width: 720px; | |
| border-collapse: collapse; | |
| } | |
| .numeric-table th, | |
| .numeric-table td { | |
| padding: var(--space-3) var(--space-4); | |
| border-bottom: 1px solid var(--color-gray-200); | |
| vertical-align: middle; | |
| } | |
| .numeric-table th { | |
| color: var(--text-muted); | |
| background: var(--surface-tint); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-xs); | |
| font-weight: 700; | |
| letter-spacing: 0.08em; | |
| text-align: left; | |
| text-transform: uppercase; | |
| } | |
| .numeric-table tr:last-child td { | |
| border-bottom: 0; | |
| } | |
| .numeric-table .metric, | |
| .numeric-table .num { | |
| text-align: right; | |
| font-variant-numeric: tabular-nums; | |
| white-space: nowrap; | |
| } | |
| .numeric-table .entity, | |
| .numeric-table .label-cell { | |
| font-weight: 650; | |
| } | |
| .numeric-table .note { | |
| color: var(--text-soft); | |
| } | |
| .numeric-table code { | |
| white-space: normal; | |
| overflow-wrap: anywhere; | |
| word-break: break-word; | |
| } | |
| .numeric-table .subtle { | |
| display: block; | |
| margin-top: 2px; | |
| color: var(--text-muted); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-xs); | |
| font-weight: 400; | |
| } | |
| @media (max-width: 620px) { | |
| .numeric-table { | |
| min-width: 0; | |
| } | |
| .numeric-table thead { | |
| display: none; | |
| } | |
| .numeric-table tbody, | |
| .numeric-table tr, | |
| .numeric-table td { | |
| display: block; | |
| } | |
| .numeric-table tr { | |
| padding: var(--space-3) var(--space-4); | |
| border-bottom: 1px solid var(--color-gray-200); | |
| } | |
| .numeric-table tr:last-child { | |
| border-bottom: 0; | |
| } | |
| .numeric-table td { | |
| padding: var(--space-1) 0; | |
| border-bottom: 0; | |
| } | |
| .numeric-table td[data-label] { | |
| display: flex; | |
| justify-content: space-between; | |
| gap: var(--space-4); | |
| } | |
| .numeric-table td[data-label]::before { | |
| content: attr(data-label); | |
| color: var(--text-muted); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-xs); | |
| text-transform: uppercase; | |
| } | |
| } | |
| /* ---------- Diff ---------- */ | |
| .diff { | |
| overflow-x: auto; | |
| color: #E8E6DC; | |
| background: var(--color-slate); | |
| border-radius: var(--radius-md); | |
| font-family: var(--font-mono); | |
| font-size: 12.5px; | |
| line-height: var(--leading-loose); | |
| } | |
| .diff-row { | |
| display: grid; | |
| grid-template-columns: 48px 18px minmax(max-content, 1fr); | |
| align-items: baseline; | |
| min-width: max-content; | |
| padding-inline-end: var(--space-4); | |
| white-space: pre; | |
| } | |
| .diff-row .ln { | |
| padding-inline-end: 14px; | |
| color: var(--color-gray-500); | |
| text-align: right; | |
| user-select: none; | |
| } | |
| .diff-row .mark { | |
| color: var(--color-gray-500); | |
| text-align: center; | |
| } | |
| .diff-row .code { | |
| color: #E8E6DC; | |
| } | |
| .diff-row.ctx .code, | |
| .diff-row[data-kind="ctx"] .code { | |
| color: #B8B6AC; | |
| } | |
| .diff-row.add, | |
| .diff-row[data-kind="add"] { | |
| background: rgba(120, 140, 93, 0.24); | |
| box-shadow: inset 3px 0 0 rgba(120, 140, 93, 0.78); | |
| } | |
| .diff-row.add .mark, | |
| .diff-row[data-kind="add"] .mark { | |
| color: #B9D394; | |
| font-weight: 800; | |
| } | |
| .diff-row.del, | |
| .diff-row[data-kind="del"] { | |
| background: rgba(176, 74, 63, 0.24); | |
| box-shadow: inset 3px 0 0 rgba(176, 74, 63, 0.82); | |
| } | |
| .diff-row.del .mark, | |
| .diff-row[data-kind="del"] .mark { | |
| color: #F19A8D; | |
| font-weight: 800; | |
| } | |
| .diff-row.hunk, | |
| .diff-row[data-kind="hunk"] { | |
| background: rgba(255, 255, 255, 0.045); | |
| } | |
| .diff-row.hunk .code, | |
| .diff-row[data-kind="hunk"] .code { | |
| color: var(--color-gray-500); | |
| } | |
| .diff[data-wrap="true"] { | |
| overflow-x: hidden; | |
| } | |
| .diff[data-wrap="true"] .diff-row { | |
| grid-template-columns: 42px 18px minmax(0, 1fr); | |
| min-width: 0; | |
| white-space: normal; | |
| } | |
| .diff[data-wrap="true"] .code { | |
| white-space: pre-wrap; | |
| overflow-wrap: anywhere; | |
| } | |
| /* ---------- Timeline ---------- */ | |
| .timeline { | |
| position: relative; | |
| display: grid; | |
| gap: 0; | |
| padding-inline-start: var(--space-6); | |
| } | |
| .timeline:is(ol, ul) { | |
| margin: 0; | |
| list-style: none; | |
| } | |
| .timeline::before { | |
| content: ""; | |
| position: absolute; | |
| inset-block: 4px 4px; | |
| inset-inline-start: 9px; | |
| width: 1.5px; | |
| background: var(--border-color); | |
| } | |
| .timeline-item, | |
| .tl-entry { | |
| position: relative; | |
| display: grid; | |
| grid-template-columns: minmax(84px, max-content) minmax(0, 1fr); | |
| gap: var(--space-4); | |
| padding-block: 0 var(--space-5); | |
| } | |
| .timeline-item:last-child, | |
| .tl-entry:last-child { | |
| padding-block-end: 0; | |
| } | |
| .timeline-item::before, | |
| .tl-entry::before { | |
| content: ""; | |
| position: absolute; | |
| inset-block-start: 4px; | |
| inset-inline-start: calc(4.25px - var(--space-6)); | |
| width: 11px; | |
| height: 11px; | |
| border: 2px solid var(--surface); | |
| border-radius: 50%; | |
| background: var(--accent); | |
| box-shadow: 0 0 0 1.5px var(--accent); | |
| } | |
| .timeline-item:not(:has(> .timeline-time, > .tl-time)), | |
| .tl-entry:not(:has(> .timeline-time, > .tl-time)) { | |
| display: block; | |
| } | |
| .timeline-time, | |
| .tl-time { | |
| color: var(--text-muted); | |
| font-family: var(--font-mono); | |
| font-size: var(--text-sm); | |
| white-space: nowrap; | |
| } | |
| .timeline-body, | |
| .tl-body { | |
| min-width: 0; | |
| color: var(--text-soft); | |
| } | |
| .timeline-body strong, | |
| .tl-body strong { | |
| color: var(--text); | |
| } | |
| @media (max-width: 620px) { | |
| .timeline-item, | |
| .tl-entry { | |
| grid-template-columns: 1fr; | |
| gap: var(--space-1); | |
| } | |
| } | |
| /* ---------- Flowchart ---------- */ | |
| .flowchart { | |
| display: block; | |
| width: 100%; | |
| height: auto; | |
| } | |
| svg.flow, | |
| svg.flowchart { | |
| display: block; | |
| width: 100%; | |
| height: auto; | |
| } | |
| .flow text, | |
| .flowchart text { | |
| fill: var(--text); | |
| font-family: var(--font-sans); | |
| font-size: 12px; | |
| } | |
| .flow .sub, | |
| .flowchart .sub { | |
| fill: var(--text-muted); | |
| font-size: 10px; | |
| } | |
| .flow-edge, | |
| .edge { | |
| fill: none; | |
| stroke: var(--text-muted); | |
| stroke-width: 1.5; | |
| } | |
| .flow-edge[data-kind="yes"], | |
| .edge.yes { | |
| stroke: var(--success); | |
| } | |
| .flow-edge[data-kind="no"], | |
| .edge.no { | |
| stroke: var(--danger); | |
| stroke-dasharray: 4 4; | |
| } | |
| /* SVG flow nodes */ | |
| .flow-node, | |
| .flow .node { | |
| cursor: pointer; | |
| transition: transform var(--dur-fast) var(--ease); | |
| } | |
| .flow-node:hover, | |
| .flow .node:hover { | |
| transform: translateY(-1px); | |
| } | |
| .flow-node rect, | |
| .flow .node rect { | |
| fill: var(--surface); | |
| stroke: var(--border-color); | |
| stroke-width: 1.5; | |
| rx: 8; | |
| } | |
| .flow-node path, | |
| .flow .node.gate path { | |
| fill: var(--surface); | |
| stroke: var(--border-color); | |
| stroke-width: 1.5; | |
| } | |
| .flow-node[data-shape="term"] rect, | |
| .flow .node.term rect { | |
| fill: var(--surface-tint); | |
| rx: 22; | |
| } | |
| .flow-node[data-kind="ok"] rect, | |
| .flow .node.ok rect { | |
| fill: rgba(120, 140, 93, 0.12); | |
| stroke: var(--success); | |
| } | |
| .flow-node[data-kind="bad"] rect, | |
| .flow .node.bad rect { | |
| fill: rgba(176, 74, 63, 0.10); | |
| stroke: var(--danger); | |
| } | |
| .flow-node.active rect, | |
| .flow-node.active path, | |
| .flow .node.active rect, | |
| .flow .node.active path { | |
| stroke: var(--accent); | |
| stroke-width: 2; | |
| } | |
| /* HTML flow nodes, for non-SVG diagrams */ | |
| div.flow-node { | |
| padding: var(--space-3) var(--space-4); | |
| background: var(--surface); | |
| border: var(--border); | |
| border-radius: var(--radius-sm); | |
| transition: | |
| transform var(--dur-fast) var(--ease), | |
| border-color var(--dur-fast) var(--ease), | |
| box-shadow var(--dur-fast) var(--ease); | |
| } | |
| div.flow-node:hover { | |
| transform: translateY(var(--hover-y)); | |
| border-color: rgba(217, 119, 87, 0.55); | |
| box-shadow: var(--shadow-sm); | |
| } | |
| div.flow-node[data-shape="term"] { | |
| border-radius: var(--radius-pill); | |
| background: var(--surface-tint); | |
| } | |
| div.flow-node[data-kind="ok"] { | |
| border-color: var(--success); | |
| background: rgba(120, 140, 93, 0.12); | |
| } | |
| div.flow-node[data-kind="bad"] { | |
| border-color: var(--danger); | |
| background: rgba(176, 74, 63, 0.10); | |
| } | |
| @media (prefers-reduced-motion: reduce) { | |
| *, | |
| *::before, | |
| *::after { | |
| scroll-behavior: auto ; | |
| transition-duration: 0.01ms ; | |
| animation-duration: 0.01ms ; | |
| animation-iteration-count: 1 ; | |
| } | |
| } | |
| /* Responsive safety net for generated artifacts. */ | |
| p, | |
| li { | |
| overflow-wrap: anywhere; | |
| } | |
| @media (max-width: 860px) { | |
| .split, | |
| .section-rail, | |
| .sidebar-layout { | |
| grid-template-columns: minmax(0, 1fr) ; | |
| } | |
| .split > *, | |
| .section-rail > *, | |
| .sidebar-layout > *, | |
| .chart-panel { | |
| width: 100%; | |
| max-width: 100%; | |
| min-width: 0; | |
| } | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <main class="page stack" data-gap="lg"> | |
| <!-- HERO: decision first --> | |
| <header class="stack" data-gap="sm"> | |
| <div class="eyebrow">Benchmark comparison · GEPA seed gating</div> | |
| <h1>Promote Candidate B as the next GEPA seed</h1> | |
| <p class="lede"> | |
| Candidate B lifts the final score from <strong>0.9175 → 0.9655</strong> | |
| (+0.048) and cuts checker warnings from <strong>5 → 2</strong> on the | |
| same four-eval suite, at the cost of <strong>+11.5% USD</strong> and | |
| <strong>+18.7% wall time</strong>. The cost is material but acceptable | |
| for seeding GEPA. | |
| </p> | |
| <div class="hero-callout"> | |
| <div class="stack" data-gap="sm"> | |
| <span class="decision-pill">Recommendation · Promote</span> | |
| <p><strong>Use Candidate B as the seed for the next single-model GEPA run.</strong> | |
| Then rerun on Kimi and Gemini before broad model-suite validation. | |
| Manually inspect the <em>code-review</em> artifact for long diff | |
| wrapping before committing.</p> | |
| <div class="cluster" data-gap="sm"> | |
| <span class="chip">model · codexresponses.gpt-5.5</span> | |
| <span class="chip">skill · birch-html candidate-b</span> | |
| <span class="chip">eval set · 4 artifacts</span> | |
| <span class="chip">2026-05-20 18:40 UTC</span> | |
| </div> | |
| </div> | |
| <aside class="reference-panel stack" data-gap="sm"> | |
| <div class="eyebrow">Headline deltas vs Seed A</div> | |
| <ul class="insight-list"> | |
| <li><span><strong>Final score +0.048.</strong> 0.9175 → 0.9655.</span></li> | |
| <li><span><strong>Checker warnings −3.</strong> 5 → 2, zero failures either side.</span></li> | |
| <li><span><strong>Cost +$0.22.</strong> $1.92 → $2.14 per full eval suite.</span></li> | |
| <li><span><strong>Wall time +77 s.</strong> 412 s → 489 s suite-wide.</span></li> | |
| </ul> | |
| </aside> | |
| </div> | |
| </header> | |
| <!-- KPI strip: evidence in the first viewport --> | |
| <section class="section stack" data-gap="md"> | |
| <div class="section-head"> | |
| <div class="eyebrow">Run KPIs · Candidate B</div> | |
| <h2>What the run produced</h2> | |
| </div> | |
| <div class="auto-grid" style="--grid-min: 160px;"> | |
| <div class="stat-card"> | |
| <div class="caption">Pass rate</div> | |
| <div class="stat-value kpi-strong">4 / 4</div> | |
| <div class="sub">generated_ok · 0 failures</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="caption">Final score</div> | |
| <div class="stat-value kpi-strong">0.966</div> | |
| <div class="sub"><span class="delta-up">+0.048</span> vs Seed A</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="caption">Checker warnings</div> | |
| <div class="stat-value kpi-strong">2</div> | |
| <div class="sub"><span class="delta-up">−3</span> · 0 failures</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="caption">Hygiene score</div> | |
| <div class="stat-value kpi-strong">0.99</div> | |
| <div class="sub"><span class="delta-down">−0.01</span> noise floor</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="caption">Wall time</div> | |
| <div class="stat-value kpi-strong">489 s</div> | |
| <div class="sub"><span class="delta-warn">+18.7%</span> vs 412 s</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="caption">Total tokens</div> | |
| <div class="stat-value kpi-strong">124.7k</div> | |
| <div class="sub"><span class="delta-warn">+7.5%</span> vs 116.0k</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="caption">Estimated cost</div> | |
| <div class="stat-value kpi-strong">$2.14</div> | |
| <div class="sub"><span class="delta-warn">+11.5%</span> vs $1.92</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="caption">Checker score</div> | |
| <div class="stat-value kpi-strong">0.94</div> | |
| <div class="sub"><span class="delta-up">+0.09</span> vs 0.85</div> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- PRIMARY TRADEOFF VISUAL --> | |
| <section class="section stack" data-gap="md"> | |
| <div class="section-head"> | |
| <div class="eyebrow">Primary tradeoff</div> | |
| <h2>Quality and warnings against cost, latency, and tokens</h2> | |
| </div> | |
| <div class="panel chart-panel stack" data-gap="sm"> | |
| <div class="chart-svg"> | |
| <svg role="img" aria-label="Final score and warnings vs cost, wall time, and tokens" preserveAspectRatio="xMidYMid meet" style="width:100%;height:auto;display:block;" xmlns:xlink="http://www.w3.org/1999/xlink" width="676.55952pt" height="258.95952pt" viewBox="0 0 676.55952 258.95952" xmlns="http://www.w3.org/2000/svg" version="1.1"> | |
| <defs> | |
| <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style> | |
| </defs> | |
| <g id="figure_1"> | |
| <g id="patch_1"> | |
| <path d="M 0 258.95952 | |
| L 676.55952 258.95952 | |
| L 676.55952 0 | |
| L 0 0 | |
| L 0 258.95952 | |
| z | |
| " style="fill: none"/> | |
| </g> | |
| <g id="axes_1"> | |
| <g id="patch_2"> | |
| <path d="M 43.597188 240.369207 | |
| L 323.623729 240.369207 | |
| L 323.623729 18.757969 | |
| L 43.597188 18.757969 | |
| L 43.597188 240.369207 | |
| z | |
| " style="fill: none"/> | |
| </g> | |
| <g id="matplotlib.axis_1"> | |
| <g id="xtick_1"> | |
| <g id="line2d_1"/> | |
| <g id="text_1"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #87867f" x="56.325667" y="254.207801" transform="rotate(-0 56.325667 254.207801)">Seed A</text> | |
| </g> | |
| </g> | |
| <g id="xtick_2"> | |
| <g id="line2d_2"/> | |
| <g id="text_2"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #87867f" x="310.89525" y="254.207801" transform="rotate(-0 310.89525 254.207801)">Candidate B</text> | |
| </g> | |
| </g> | |
| </g> | |
| <g id="matplotlib.axis_2"> | |
| <g id="ytick_1"> | |
| <g id="line2d_3"> | |
| <path d="M 43.597188 240.369207 | |
| L 323.623729 240.369207 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_4"/> | |
| <g id="text_3"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="243.788504" transform="rotate(-0 36.597188 243.788504)">0.88</text> | |
| </g> | |
| </g> | |
| <g id="ytick_2"> | |
| <g id="line2d_5"> | |
| <path d="M 43.597188 204.911409 | |
| L 323.623729 204.911409 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_6"/> | |
| <g id="text_4"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="208.330706" transform="rotate(-0 36.597188 208.330706)">0.90</text> | |
| </g> | |
| </g> | |
| <g id="ytick_3"> | |
| <g id="line2d_7"> | |
| <path d="M 43.597188 169.453611 | |
| L 323.623729 169.453611 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_8"/> | |
| <g id="text_5"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="172.872908" transform="rotate(-0 36.597188 172.872908)">0.92</text> | |
| </g> | |
| </g> | |
| <g id="ytick_4"> | |
| <g id="line2d_9"> | |
| <path d="M 43.597188 133.995813 | |
| L 323.623729 133.995813 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_10"/> | |
| <g id="text_6"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="137.41511" transform="rotate(-0 36.597188 137.41511)">0.94</text> | |
| </g> | |
| </g> | |
| <g id="ytick_5"> | |
| <g id="line2d_11"> | |
| <path d="M 43.597188 98.538015 | |
| L 323.623729 98.538015 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_12"/> | |
| <g id="text_7"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="101.957312" transform="rotate(-0 36.597188 101.957312)">0.96</text> | |
| </g> | |
| </g> | |
| <g id="ytick_6"> | |
| <g id="line2d_13"> | |
| <path d="M 43.597188 63.080216 | |
| L 323.623729 63.080216 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_14"/> | |
| <g id="text_8"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="66.499513" transform="rotate(-0 36.597188 66.499513)">0.98</text> | |
| </g> | |
| </g> | |
| <g id="ytick_7"> | |
| <g id="line2d_15"> | |
| <path d="M 43.597188 27.622418 | |
| L 323.623729 27.622418 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_16"/> | |
| <g id="text_9"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="31.041715" transform="rotate(-0 36.597188 31.041715)">1.00</text> | |
| </g> | |
| </g> | |
| <g id="text_10"> | |
| <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="10.478438" y="129.563588" transform="rotate(-90 10.478438 129.563588)">Final score (0-1)</text> | |
| </g> | |
| </g> | |
| <g id="line2d_17"> | |
| <path d="M 56.325667 173.885836 | |
| L 310.89525 88.78712 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #788c5d; stroke-width: 2.2; stroke-linecap: square"/> | |
| <defs> | |
| <path id="m4209fc29bb" d="M 0 3 | |
| C 0.795609 3 1.55874 2.683901 2.12132 2.12132 | |
| C 2.683901 1.55874 3 0.795609 3 0 | |
| C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 | |
| C 1.55874 -2.683901 0.795609 -3 0 -3 | |
| C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 | |
| C -2.683901 -1.55874 -3 -0.795609 -3 0 | |
| C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 | |
| C -1.55874 2.683901 -0.795609 3 0 3 | |
| z | |
| " style="stroke: #788c5d"/> | |
| </defs> | |
| <g clip-path="url(#pf3b598b886)"> | |
| <use xlink:href="#m4209fc29bb" x="56.325667" y="173.885836" style="fill: #788c5d; stroke: #788c5d"/> | |
| <use xlink:href="#m4209fc29bb" x="310.89525" y="88.78712" style="fill: #788c5d; stroke: #788c5d"/> | |
| </g> | |
| </g> | |
| <g id="patch_3"> | |
| <path d="M 43.597188 240.369207 | |
| L 43.597188 18.757969 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="patch_4"> | |
| <path d="M 43.597188 240.369207 | |
| L 323.623729 240.369207 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="text_11"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #141413" x="56.325667" y="163.885836" transform="rotate(-0 56.325667 163.885836)">0.917</text> | |
| </g> | |
| <g id="text_12"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #141413" x="310.89525" y="78.78712" transform="rotate(-0 310.89525 78.78712)">0.966</text> | |
| </g> | |
| <g id="text_13"> | |
| <text style="font-size: 13px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #141413" x="183.610458" y="12.757969" transform="rotate(-0 183.610458 12.757969)">Quality up, warnings down</text> | |
| </g> | |
| </g> | |
| <g id="axes_2"> | |
| <g id="patch_5"> | |
| <path d="M 393.652979 240.369207 | |
| L 673.67952 240.369207 | |
| L 673.67952 18.757969 | |
| L 393.652979 18.757969 | |
| L 393.652979 240.369207 | |
| z | |
| " style="fill: none"/> | |
| </g> | |
| <g id="matplotlib.axis_3"> | |
| <g id="xtick_3"> | |
| <g id="line2d_18"/> | |
| <g id="text_14"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #87867f" x="440.074491" y="254.207801" transform="rotate(-0 440.074491 254.207801)">Cost (USD)</text> | |
| </g> | |
| </g> | |
| <g id="xtick_4"> | |
| <g id="line2d_19"/> | |
| <g id="text_15"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #87867f" x="533.666249" y="254.207801" transform="rotate(-0 533.666249 254.207801)">Wall time (s)</text> | |
| </g> | |
| </g> | |
| <g id="xtick_5"> | |
| <g id="line2d_20"/> | |
| <g id="text_16"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #87867f" x="627.258008" y="254.207801" transform="rotate(-0 627.258008 254.207801)">Total tokens (k)</text> | |
| </g> | |
| </g> | |
| </g> | |
| <g id="matplotlib.axis_4"> | |
| <g id="ytick_8"> | |
| <g id="line2d_21"> | |
| <path d="M 393.652979 240.369207 | |
| L 673.67952 240.369207 | |
| " clip-path="url(#pa473b7c7fc)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_22"/> | |
| <g id="text_17"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="386.652979" y="243.788504" transform="rotate(-0 386.652979 243.788504)">0</text> | |
| </g> | |
| </g> | |
| <g id="ytick_9"> | |
| <g id="line2d_23"> | |
| <path d="M 393.652979 207.537913 | |
| L 673.67952 207.537913 | |
| " clip-path="url(#pa473b7c7fc)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_24"/> | |
| <g id="text_18"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="386.652979" y="210.95721" transform="rotate(-0 386.652979 210.95721)">20</text> | |
| </g> | |
| </g> | |
| <g id="ytick_10"> | |
| <g id="line2d_25"> | |
| <path d="M 393.652979 174.706618 | |
| L 673.67952 174.706618 | |
| " clip-path="url(#pa473b7c7fc)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_26"/> | |
| <g id="text_19"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="386.652979" y="178.125915" transform="rotate(-0 386.652979 178.125915)">40</text> | |
| </g> | |
| </g> | |
| <g id="ytick_11"> | |
| <g id="line2d_27"> | |
| <path d="M 393.652979 141.875324 | |
| L 673.67952 141.875324 | |
| " clip-path="url(#pa473b7c7fc)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_28"/> | |
| <g id="text_20"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="386.652979" y="145.29462" transform="rotate(-0 386.652979 145.29462)">60</text> | |
| </g> | |
| </g> | |
| <g id="ytick_12"> | |
| <g id="line2d_29"> | |
| <path d="M 393.652979 109.044029 | |
| L 673.67952 109.044029 | |
| " clip-path="url(#pa473b7c7fc)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_30"/> | |
| <g id="text_21"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="386.652979" y="112.463326" transform="rotate(-0 386.652979 112.463326)">80</text> | |
| </g> | |
| </g> | |
| <g id="ytick_13"> | |
| <g id="line2d_31"> | |
| <path d="M 393.652979 76.212734 | |
| L 673.67952 76.212734 | |
| " clip-path="url(#pa473b7c7fc)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_32"/> | |
| <g id="text_22"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="386.652979" y="79.632031" transform="rotate(-0 386.652979 79.632031)">100</text> | |
| </g> | |
| </g> | |
| <g id="ytick_14"> | |
| <g id="line2d_33"> | |
| <path d="M 393.652979 43.38144 | |
| L 673.67952 43.38144 | |
| " clip-path="url(#pa473b7c7fc)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_34"/> | |
| <g id="text_23"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="386.652979" y="46.800737" transform="rotate(-0 386.652979 46.800737)">120</text> | |
| </g> | |
| </g> | |
| <g id="text_24"> | |
| <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="363.394541" y="129.563588" transform="rotate(-90 363.394541 129.563588)">Indexed to Seed A = 100</text> | |
| </g> | |
| </g> | |
| <g id="patch_6"> | |
| <path d="M 406.381458 240.369207 | |
| L 440.074491 240.369207 | |
| L 440.074491 76.212734 | |
| L 406.381458 76.212734 | |
| z | |
| " clip-path="url(#pa473b7c7fc)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_7"> | |
| <path d="M 499.973216 240.369207 | |
| L 533.666249 240.369207 | |
| L 533.666249 76.212734 | |
| L 499.973216 76.212734 | |
| z | |
| " clip-path="url(#pa473b7c7fc)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_8"> | |
| <path d="M 593.564975 240.369207 | |
| L 627.258008 240.369207 | |
| L 627.258008 76.212734 | |
| L 593.564975 76.212734 | |
| z | |
| " clip-path="url(#pa473b7c7fc)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_9"> | |
| <path d="M 440.074491 240.369207 | |
| L 473.767524 240.369207 | |
| L 473.767524 57.403138 | |
| L 440.074491 57.403138 | |
| z | |
| " clip-path="url(#pa473b7c7fc)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_10"> | |
| <path d="M 533.666249 240.369207 | |
| L 567.359282 240.369207 | |
| L 567.359282 45.533005 | |
| L 533.666249 45.533005 | |
| z | |
| " clip-path="url(#pa473b7c7fc)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_11"> | |
| <path d="M 627.258008 240.369207 | |
| L 660.951041 240.369207 | |
| L 660.951041 63.900999 | |
| L 627.258008 63.900999 | |
| z | |
| " clip-path="url(#pa473b7c7fc)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_12"> | |
| <path d="M 393.652979 240.369207 | |
| L 393.652979 18.757969 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="patch_13"> | |
| <path d="M 393.652979 240.369207 | |
| L 673.67952 240.369207 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="text_25"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #b85c3e" x="456.921007" y="53.403138" transform="rotate(-0 456.921007 53.403138)">+11.5%</text> | |
| </g> | |
| <g id="text_26"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #b85c3e" x="550.512766" y="41.533005" transform="rotate(-0 550.512766 41.533005)">+18.7%</text> | |
| </g> | |
| <g id="text_27"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #b85c3e" x="644.104524" y="59.900999" transform="rotate(-0 644.104524 59.900999)">+7.5%</text> | |
| </g> | |
| <g id="text_28"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="423.227974" y="72.212734" transform="rotate(-0 423.227974 72.212734)">1.92</text> | |
| </g> | |
| <g id="text_29"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="516.819733" y="72.212734" transform="rotate(-0 516.819733 72.212734)">412</text> | |
| </g> | |
| <g id="text_30"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="610.411491" y="72.212734" transform="rotate(-0 610.411491 72.212734)">116</text> | |
| </g> | |
| <g id="text_31"> | |
| <text style="font-size: 13px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #141413" x="533.666249" y="12.757969" transform="rotate(-0 533.666249 12.757969)">Cost, latency, tokens trade-off</text> | |
| </g> | |
| <g id="legend_1"> | |
| <g id="patch_14"> | |
| <path d="M 590.347958 217.187176 | |
| L 602.947958 217.187176 | |
| L 602.947958 210.887176 | |
| L 590.347958 210.887176 | |
| z | |
| " style="fill: #6a8caf"/> | |
| </g> | |
| <g id="text_32"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #141413" x="606.997958" y="217.187176" transform="rotate(-0 606.997958 217.187176)">Seed A (100)</text> | |
| </g> | |
| <g id="patch_15"> | |
| <path d="M 590.347958 230.397489 | |
| L 602.947958 230.397489 | |
| L 602.947958 224.097489 | |
| L 590.347958 224.097489 | |
| z | |
| " style="fill: #d97757"/> | |
| </g> | |
| <g id="text_33"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #141413" x="606.997958" y="230.397489" transform="rotate(-0 606.997958 230.397489)">Candidate B</text> | |
| </g> | |
| </g> | |
| </g> | |
| <g id="axes_3"> | |
| <g id="matplotlib.axis_5"> | |
| <g id="ytick_15"> | |
| <g id="line2d_35"/> | |
| <g id="text_34"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="243.788504" transform="rotate(-0 329.623729 243.788504)">0</text> | |
| </g> | |
| </g> | |
| <g id="ytick_16"> | |
| <g id="line2d_36"/> | |
| <g id="text_35"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="212.129756" transform="rotate(-0 329.623729 212.129756)">1</text> | |
| </g> | |
| </g> | |
| <g id="ytick_17"> | |
| <g id="line2d_37"/> | |
| <g id="text_36"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="180.471008" transform="rotate(-0 329.623729 180.471008)">2</text> | |
| </g> | |
| </g> | |
| <g id="ytick_18"> | |
| <g id="line2d_38"/> | |
| <g id="text_37"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="148.812259" transform="rotate(-0 329.623729 148.812259)">3</text> | |
| </g> | |
| </g> | |
| <g id="ytick_19"> | |
| <g id="line2d_39"/> | |
| <g id="text_38"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="117.153511" transform="rotate(-0 329.623729 117.153511)">4</text> | |
| </g> | |
| </g> | |
| <g id="ytick_20"> | |
| <g id="line2d_40"/> | |
| <g id="text_39"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="85.494762" transform="rotate(-0 329.623729 85.494762)">5</text> | |
| </g> | |
| </g> | |
| <g id="ytick_21"> | |
| <g id="line2d_41"/> | |
| <g id="text_40"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="53.836014" transform="rotate(-0 329.623729 53.836014)">6</text> | |
| </g> | |
| </g> | |
| <g id="ytick_22"> | |
| <g id="line2d_42"/> | |
| <g id="text_41"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #b85c3e" x="329.623729" y="22.177266" transform="rotate(-0 329.623729 22.177266)">7</text> | |
| </g> | |
| </g> | |
| <g id="text_42"> | |
| <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #b85c3e" x="346.948416" y="129.563588" transform="rotate(-90 346.948416 129.563588)">Checker warnings</text> | |
| </g> | |
| </g> | |
| <g id="line2d_43"> | |
| <path d="M 56.325667 82.075466 | |
| L 310.89525 177.051711 | |
| " clip-path="url(#pf3b598b886)" style="fill: none; stroke: #d97757; stroke-width: 2.2; stroke-linecap: square"/> | |
| <defs> | |
| <path id="md2b1891308" d="M -3 3 | |
| L 3 3 | |
| L 3 -3 | |
| L -3 -3 | |
| z | |
| " style="stroke: #d97757; stroke-linejoin: miter"/> | |
| </defs> | |
| <g clip-path="url(#pf3b598b886)"> | |
| <use xlink:href="#md2b1891308" x="56.325667" y="82.075466" style="fill: #d97757; stroke: #d97757; stroke-linejoin: miter"/> | |
| <use xlink:href="#md2b1891308" x="310.89525" y="177.051711" style="fill: #d97757; stroke: #d97757; stroke-linejoin: miter"/> | |
| </g> | |
| </g> | |
| <g id="patch_16"> | |
| <path d="M 43.597188 240.369207 | |
| L 43.597188 18.757969 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="patch_17"> | |
| <path d="M 323.623729 240.369207 | |
| L 323.623729 18.757969 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="patch_18"> | |
| <path d="M 43.597188 240.369207 | |
| L 323.623729 240.369207 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="text_43"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #b85c3e" x="56.325667" y="98.075466" transform="rotate(-0 56.325667 98.075466)">5</text> | |
| </g> | |
| <g id="text_44"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #b85c3e" x="310.89525" y="193.051711" transform="rotate(-0 310.89525 193.051711)">2</text> | |
| </g> | |
| </g> | |
| </g> | |
| <defs> | |
| <clipPath id="pf3b598b886"> | |
| <rect x="43.597188" y="18.757969" width="280.026541" height="221.611239"/> | |
| </clipPath> | |
| <clipPath id="pa473b7c7fc"> | |
| <rect x="393.652979" y="18.757969" width="280.026541" height="221.611239"/> | |
| </clipPath> | |
| </defs> | |
| </svg> | |
| </div> | |
| <p class="chart-caption"> | |
| Left: final score rises from 0.917 to 0.966 while checker warnings | |
| drop from 5 to 2. Right: Candidate B costs 11.5% more, takes 18.7% | |
| longer, and consumes 7.5% more tokens (indexed to Seed A = 100). | |
| Source: <code>eval-runs/reports/skill-seed-gpt55/summary.json</code> · | |
| <code>eval-runs/reports/skill-candidate-b-gpt55/summary.json</code>. | |
| </p> | |
| </div> | |
| <!-- Exact values, close to the chart --> | |
| <div class="numeric-table-wrap"> | |
| <table class="numeric-table"> | |
| <caption class="caption">Aggregate run comparison · exact values</caption> | |
| <thead> | |
| <tr> | |
| <th scope="col">Metric</th> | |
| <th scope="col" class="metric">Seed A</th> | |
| <th scope="col" class="metric">Candidate B</th> | |
| <th scope="col" class="metric">Δ</th> | |
| <th scope="col">Direction</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr><th scope="row">Generated OK / total</th> | |
| <td class="metric">4 / 4</td><td class="metric">4 / 4</td> | |
| <td class="metric">0</td><td>flat</td></tr> | |
| <tr><th scope="row">Generation score</th> | |
| <td class="metric">1.000</td><td class="metric">1.000</td> | |
| <td class="metric">0.000</td><td>flat</td></tr> | |
| <tr><th scope="row">Checker failures</th> | |
| <td class="metric">0</td><td class="metric">0</td> | |
| <td class="metric">0</td><td>flat</td></tr> | |
| <tr><th scope="row">Checker warnings</th> | |
| <td class="metric">5</td><td class="metric">2</td> | |
| <td class="metric">−3</td><td><span class="delta-up">better</span></td></tr> | |
| <tr><th scope="row">Checker score</th> | |
| <td class="metric">0.850</td><td class="metric">0.940</td> | |
| <td class="metric">+0.090</td><td><span class="delta-up">better</span></td></tr> | |
| <tr><th scope="row">Hygiene score</th> | |
| <td class="metric">1.000</td><td class="metric">0.990</td> | |
| <td class="metric">−0.010</td><td><span class="delta-down">worse (noise)</span></td></tr> | |
| <tr><th scope="row">Final score</th> | |
| <td class="metric">0.9175</td><td class="metric">0.9655</td> | |
| <td class="metric">+0.0480</td><td><span class="delta-up">better</span></td></tr> | |
| <tr><th scope="row">Wall time (s)</th> | |
| <td class="metric">412</td><td class="metric">489</td> | |
| <td class="metric">+77 (+18.7%)</td><td><span class="delta-warn">worse</span></td></tr> | |
| <tr><th scope="row">Input tokens</th> | |
| <td class="metric">84,200</td><td class="metric">90,100</td> | |
| <td class="metric">+5,900 (+7.0%)</td><td><span class="delta-warn">worse</span></td></tr> | |
| <tr><th scope="row">Output tokens</th> | |
| <td class="metric">31,800</td><td class="metric">34,600</td> | |
| <td class="metric">+2,800 (+8.8%)</td><td><span class="delta-warn">worse</span></td></tr> | |
| <tr><th scope="row">Total tokens</th> | |
| <td class="metric">116,000</td><td class="metric">124,700</td> | |
| <td class="metric">+8,700 (+7.5%)</td><td><span class="delta-warn">worse</span></td></tr> | |
| <tr><th scope="row">Estimated cost (USD)</th> | |
| <td class="metric">$1.92</td><td class="metric">$2.14</td> | |
| <td class="metric">+$0.22 (+11.5%)</td><td><span class="delta-warn">worse</span></td></tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| </section> | |
| <!-- PER-EVAL DETAIL --> | |
| <section class="section stack" data-gap="md"> | |
| <div class="section-head"> | |
| <div class="eyebrow">Per-eval breakdown</div> | |
| <h2>Where the gains came from</h2> | |
| </div> | |
| <div class="panel chart-panel stack" data-gap="sm"> | |
| <div class="chart-svg"> | |
| <svg role="img" aria-label="Per-eval score, warnings, and wall time comparison" preserveAspectRatio="xMidYMid meet" style="width:100%;height:auto;display:block;" xmlns:xlink="http://www.w3.org/1999/xlink" width="734.15952pt" height="230.15952pt" viewBox="0 0 734.15952 230.15952" xmlns="http://www.w3.org/2000/svg" version="1.1"> | |
| <defs> | |
| <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style> | |
| </defs> | |
| <g id="figure_1"> | |
| <g id="patch_1"> | |
| <path d="M -0 230.15952 | |
| L 734.15952 230.15952 | |
| L 734.15952 0 | |
| L -0 0 | |
| L -0 230.15952 | |
| z | |
| " style="fill: none"/> | |
| </g> | |
| <g id="axes_1"> | |
| <g id="patch_2"> | |
| <path d="M 43.597188 182.951158 | |
| L 244.541708 182.951158 | |
| L 244.541708 18.757969 | |
| L 43.597188 18.757969 | |
| L 43.597188 182.951158 | |
| z | |
| " style="fill: none"/> | |
| </g> | |
| <g id="matplotlib.axis_1"> | |
| <g id="xtick_1"> | |
| <g id="line2d_1"/> | |
| <g id="text_1"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(12.406099 215.113542) rotate(-18)">numeric-data</text> | |
| </g> | |
| </g> | |
| <g id="xtick_2"> | |
| <g id="line2d_2"/> | |
| <g id="text_2"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(66.901253 213.362719) rotate(-18)">code-review</text> | |
| </g> | |
| </g> | |
| <g id="xtick_3"> | |
| <g id="line2d_3"/> | |
| <g id="text_3"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(93.049723 220.822293) rotate(-18)">module-explainer</text> | |
| </g> | |
| </g> | |
| <g id="xtick_4"> | |
| <g id="line2d_4"/> | |
| <g id="text_4"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(127.761713 225.49941) rotate(-18)">implementation-plan</text> | |
| </g> | |
| </g> | |
| </g> | |
| <g id="matplotlib.axis_2"> | |
| <g id="ytick_1"> | |
| <g id="line2d_5"> | |
| <path d="M 43.597188 172.689084 | |
| L 244.541708 172.689084 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_6"/> | |
| <g id="text_5"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="176.108381" transform="rotate(-0 36.597188 176.108381)">0.86</text> | |
| </g> | |
| </g> | |
| <g id="ytick_2"> | |
| <g id="line2d_7"> | |
| <path d="M 43.597188 152.164935 | |
| L 244.541708 152.164935 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_8"/> | |
| <g id="text_6"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="155.584232" transform="rotate(-0 36.597188 155.584232)">0.88</text> | |
| </g> | |
| </g> | |
| <g id="ytick_3"> | |
| <g id="line2d_9"> | |
| <path d="M 43.597188 131.640786 | |
| L 244.541708 131.640786 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_10"/> | |
| <g id="text_7"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="135.060083" transform="rotate(-0 36.597188 135.060083)">0.90</text> | |
| </g> | |
| </g> | |
| <g id="ytick_4"> | |
| <g id="line2d_11"> | |
| <path d="M 43.597188 111.116638 | |
| L 244.541708 111.116638 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_12"/> | |
| <g id="text_8"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="114.535935" transform="rotate(-0 36.597188 114.535935)">0.92</text> | |
| </g> | |
| </g> | |
| <g id="ytick_5"> | |
| <g id="line2d_13"> | |
| <path d="M 43.597188 90.592489 | |
| L 244.541708 90.592489 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_14"/> | |
| <g id="text_9"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="94.011786" transform="rotate(-0 36.597188 94.011786)">0.94</text> | |
| </g> | |
| </g> | |
| <g id="ytick_6"> | |
| <g id="line2d_15"> | |
| <path d="M 43.597188 70.06834 | |
| L 244.541708 70.06834 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_16"/> | |
| <g id="text_10"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="73.487637" transform="rotate(-0 36.597188 73.487637)">0.96</text> | |
| </g> | |
| </g> | |
| <g id="ytick_7"> | |
| <g id="line2d_17"> | |
| <path d="M 43.597188 49.544192 | |
| L 244.541708 49.544192 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_18"/> | |
| <g id="text_11"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="52.963489" transform="rotate(-0 36.597188 52.963489)">0.98</text> | |
| </g> | |
| </g> | |
| <g id="ytick_8"> | |
| <g id="line2d_19"> | |
| <path d="M 43.597188 29.020043 | |
| L 244.541708 29.020043 | |
| " clip-path="url(#p8c66debf8d)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_20"/> | |
| <g id="text_12"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="36.597188" y="32.43934" transform="rotate(-0 36.597188 32.43934)">1.00</text> | |
| </g> | |
| </g> | |
| <g id="text_13"> | |
| <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="10.478438" y="100.854563" transform="rotate(-90 10.478438 100.854563)">Per-eval score</text> | |
| </g> | |
| </g> | |
| <g id="patch_3"> | |
| <path d="M 52.731029 1055.227476 | |
| L 70.409433 1055.227476 | |
| L 70.409433 100.854563 | |
| L 52.731029 100.854563 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_4"> | |
| <path d="M 101.837706 1055.227476 | |
| L 119.516109 1055.227476 | |
| L 119.516109 121.378712 | |
| L 101.837706 121.378712 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_5"> | |
| <path d="M 150.944382 1055.227476 | |
| L 168.622786 1055.227476 | |
| L 168.622786 105.985601 | |
| L 150.944382 105.985601 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_6"> | |
| <path d="M 200.051059 1055.227476 | |
| L 217.729462 1055.227476 | |
| L 217.729462 126.509749 | |
| L 200.051059 126.509749 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_7"> | |
| <path d="M 70.409433 1055.227476 | |
| L 88.087836 1055.227476 | |
| L 88.087836 59.806266 | |
| L 70.409433 59.806266 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_8"> | |
| <path d="M 119.516109 1055.227476 | |
| L 137.194513 1055.227476 | |
| L 137.194513 75.199378 | |
| L 119.516109 75.199378 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_9"> | |
| <path d="M 168.622786 1055.227476 | |
| L 186.301189 1055.227476 | |
| L 186.301189 86.487659 | |
| L 168.622786 86.487659 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_10"> | |
| <path d="M 217.729462 1055.227476 | |
| L 235.407866 1055.227476 | |
| L 235.407866 36.203495 | |
| L 217.729462 36.203495 | |
| z | |
| " clip-path="url(#p8c66debf8d)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_11"> | |
| <path d="M 43.597188 182.951158 | |
| L 43.597188 18.757969 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="patch_12"> | |
| <path d="M 43.597187 182.951158 | |
| L 244.541708 182.951158 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="text_14"> | |
| <text style="font-size: 13px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #141413" x="144.069448" y="12.757969" transform="rotate(-0 144.069448 12.757969)">Score</text> | |
| </g> | |
| <g id="legend_1"> | |
| <g id="patch_13"> | |
| <path d="M 164.744051 159.769127 | |
| L 177.344051 159.769127 | |
| L 177.344051 153.469127 | |
| L 164.744051 153.469127 | |
| z | |
| " style="fill: #6a8caf"/> | |
| </g> | |
| <g id="text_15"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #141413" x="181.394051" y="159.769127" transform="rotate(-0 181.394051 159.769127)">Seed A</text> | |
| </g> | |
| <g id="patch_14"> | |
| <path d="M 164.744051 172.979439 | |
| L 177.344051 172.979439 | |
| L 177.344051 166.679439 | |
| L 164.744051 166.679439 | |
| z | |
| " style="fill: #d97757"/> | |
| </g> | |
| <g id="text_16"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #141413" x="181.394051" y="172.979439" transform="rotate(-0 181.394051 172.979439)">Candidate B</text> | |
| </g> | |
| </g> | |
| </g> | |
| <g id="axes_2"> | |
| <g id="patch_15"> | |
| <path d="M 285.533125 182.951158 | |
| L 486.477645 182.951158 | |
| L 486.477645 18.757969 | |
| L 285.533125 18.757969 | |
| L 285.533125 182.951158 | |
| z | |
| " style="fill: none"/> | |
| </g> | |
| <g id="matplotlib.axis_3"> | |
| <g id="xtick_5"> | |
| <g id="line2d_21"/> | |
| <g id="text_17"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(254.342036 215.113542) rotate(-18)">numeric-data</text> | |
| </g> | |
| </g> | |
| <g id="xtick_6"> | |
| <g id="line2d_22"/> | |
| <g id="text_18"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(308.837191 213.362719) rotate(-18)">code-review</text> | |
| </g> | |
| </g> | |
| <g id="xtick_7"> | |
| <g id="line2d_23"/> | |
| <g id="text_19"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(334.98566 220.822293) rotate(-18)">module-explainer</text> | |
| </g> | |
| </g> | |
| <g id="xtick_8"> | |
| <g id="line2d_24"/> | |
| <g id="text_20"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(369.697651 225.49941) rotate(-18)">implementation-plan</text> | |
| </g> | |
| </g> | |
| </g> | |
| <g id="matplotlib.axis_4"> | |
| <g id="ytick_9"> | |
| <g id="line2d_25"> | |
| <path d="M 285.533125 182.951158 | |
| L 486.477645 182.951158 | |
| " clip-path="url(#pf585a739e0)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_26"/> | |
| <g id="text_21"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="278.533125" y="186.370455" transform="rotate(-0 278.533125 186.370455)">0.0</text> | |
| </g> | |
| </g> | |
| <g id="ytick_10"> | |
| <g id="line2d_27"> | |
| <path d="M 285.533125 155.585627 | |
| L 486.477645 155.585627 | |
| " clip-path="url(#pf585a739e0)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_28"/> | |
| <g id="text_22"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="278.533125" y="159.004923" transform="rotate(-0 278.533125 159.004923)">0.5</text> | |
| </g> | |
| </g> | |
| <g id="ytick_11"> | |
| <g id="line2d_29"> | |
| <path d="M 285.533125 128.220095 | |
| L 486.477645 128.220095 | |
| " clip-path="url(#pf585a739e0)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_30"/> | |
| <g id="text_23"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="278.533125" y="131.639392" transform="rotate(-0 278.533125 131.639392)">1.0</text> | |
| </g> | |
| </g> | |
| <g id="ytick_12"> | |
| <g id="line2d_31"> | |
| <path d="M 285.533125 100.854563 | |
| L 486.477645 100.854563 | |
| " clip-path="url(#pf585a739e0)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_32"/> | |
| <g id="text_24"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="278.533125" y="104.27386" transform="rotate(-0 278.533125 104.27386)">1.5</text> | |
| </g> | |
| </g> | |
| <g id="ytick_13"> | |
| <g id="line2d_33"> | |
| <path d="M 285.533125 73.489032 | |
| L 486.477645 73.489032 | |
| " clip-path="url(#pf585a739e0)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_34"/> | |
| <g id="text_25"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="278.533125" y="76.908329" transform="rotate(-0 278.533125 76.908329)">2.0</text> | |
| </g> | |
| </g> | |
| <g id="ytick_14"> | |
| <g id="line2d_35"> | |
| <path d="M 285.533125 46.1235 | |
| L 486.477645 46.1235 | |
| " clip-path="url(#pf585a739e0)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_36"/> | |
| <g id="text_26"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="278.533125" y="49.542797" transform="rotate(-0 278.533125 49.542797)">2.5</text> | |
| </g> | |
| </g> | |
| <g id="ytick_15"> | |
| <g id="line2d_37"> | |
| <path d="M 285.533125 18.757969 | |
| L 486.477645 18.757969 | |
| " clip-path="url(#pf585a739e0)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_38"/> | |
| <g id="text_27"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="278.533125" y="22.177266" transform="rotate(-0 278.533125 22.177266)">3.0</text> | |
| </g> | |
| </g> | |
| <g id="text_28"> | |
| <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="258.140625" y="100.854563" transform="rotate(-90 258.140625 100.854563)">Checker warnings</text> | |
| </g> | |
| </g> | |
| <g id="patch_16"> | |
| <path d="M 294.666967 182.951158 | |
| L 312.34537 182.951158 | |
| L 312.34537 73.489032 | |
| L 294.666967 73.489032 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_17"> | |
| <path d="M 343.773643 182.951158 | |
| L 361.452047 182.951158 | |
| L 361.452047 128.220095 | |
| L 343.773643 128.220095 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_18"> | |
| <path d="M 392.88032 182.951158 | |
| L 410.558723 182.951158 | |
| L 410.558723 128.220095 | |
| L 392.88032 128.220095 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_19"> | |
| <path d="M 441.986996 182.951158 | |
| L 459.6654 182.951158 | |
| L 459.6654 128.220095 | |
| L 441.986996 128.220095 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_20"> | |
| <path d="M 312.34537 182.951158 | |
| L 330.023774 182.951158 | |
| L 330.023774 128.220095 | |
| L 312.34537 128.220095 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_21"> | |
| <path d="M 361.452047 182.951158 | |
| L 379.13045 182.951158 | |
| L 379.13045 182.951158 | |
| L 361.452047 182.951158 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_22"> | |
| <path d="M 410.558723 182.951158 | |
| L 428.237127 182.951158 | |
| L 428.237127 128.220095 | |
| L 410.558723 128.220095 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_23"> | |
| <path d="M 459.6654 182.951158 | |
| L 477.343803 182.951158 | |
| L 477.343803 182.951158 | |
| L 459.6654 182.951158 | |
| z | |
| " clip-path="url(#pf585a739e0)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_24"> | |
| <path d="M 285.533125 182.951158 | |
| L 285.533125 18.757969 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="patch_25"> | |
| <path d="M 285.533125 182.951158 | |
| L 486.477645 182.951158 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="text_29"> | |
| <text style="font-size: 13px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #141413" x="386.005385" y="12.757969" transform="rotate(-0 386.005385 12.757969)">Warnings</text> | |
| </g> | |
| </g> | |
| <g id="axes_3"> | |
| <g id="patch_26"> | |
| <path d="M 530.335 182.951158 | |
| L 731.27952 182.951158 | |
| L 731.27952 18.757969 | |
| L 530.335 18.757969 | |
| L 530.335 182.951158 | |
| z | |
| " style="fill: none"/> | |
| </g> | |
| <g id="matplotlib.axis_5"> | |
| <g id="xtick_9"> | |
| <g id="line2d_39"/> | |
| <g id="text_30"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(499.143911 215.113542) rotate(-18)">numeric-data</text> | |
| </g> | |
| </g> | |
| <g id="xtick_10"> | |
| <g id="line2d_40"/> | |
| <g id="text_31"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(553.639066 213.362719) rotate(-18)">code-review</text> | |
| </g> | |
| </g> | |
| <g id="xtick_11"> | |
| <g id="line2d_41"/> | |
| <g id="text_32"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(579.787535 220.822293) rotate(-18)">module-explainer</text> | |
| </g> | |
| </g> | |
| <g id="xtick_12"> | |
| <g id="line2d_42"/> | |
| <g id="text_33"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; fill: #87867f" transform="translate(614.499526 225.49941) rotate(-18)">implementation-plan</text> | |
| </g> | |
| </g> | |
| </g> | |
| <g id="matplotlib.axis_6"> | |
| <g id="ytick_16"> | |
| <g id="line2d_43"> | |
| <path d="M 530.335 182.951158 | |
| L 731.27952 182.951158 | |
| " clip-path="url(#p7cef315271)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_44"/> | |
| <g id="text_34"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="523.335" y="186.370455" transform="rotate(-0 523.335 186.370455)">0</text> | |
| </g> | |
| </g> | |
| <g id="ytick_17"> | |
| <g id="line2d_45"> | |
| <path d="M 530.335 159.258057 | |
| L 731.27952 159.258057 | |
| " clip-path="url(#p7cef315271)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_46"/> | |
| <g id="text_35"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="523.335" y="162.677354" transform="rotate(-0 523.335 162.677354)">20</text> | |
| </g> | |
| </g> | |
| <g id="ytick_18"> | |
| <g id="line2d_47"> | |
| <path d="M 530.335 135.564956 | |
| L 731.27952 135.564956 | |
| " clip-path="url(#p7cef315271)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_48"/> | |
| <g id="text_36"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="523.335" y="138.984253" transform="rotate(-0 523.335 138.984253)">40</text> | |
| </g> | |
| </g> | |
| <g id="ytick_19"> | |
| <g id="line2d_49"> | |
| <path d="M 530.335 111.871855 | |
| L 731.27952 111.871855 | |
| " clip-path="url(#p7cef315271)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_50"/> | |
| <g id="text_37"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="523.335" y="115.291152" transform="rotate(-0 523.335 115.291152)">60</text> | |
| </g> | |
| </g> | |
| <g id="ytick_20"> | |
| <g id="line2d_51"> | |
| <path d="M 530.335 88.178754 | |
| L 731.27952 88.178754 | |
| " clip-path="url(#p7cef315271)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_52"/> | |
| <g id="text_38"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="523.335" y="91.598051" transform="rotate(-0 523.335 91.598051)">80</text> | |
| </g> | |
| </g> | |
| <g id="ytick_21"> | |
| <g id="line2d_53"> | |
| <path d="M 530.335 64.485654 | |
| L 731.27952 64.485654 | |
| " clip-path="url(#p7cef315271)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_54"/> | |
| <g id="text_39"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="523.335" y="67.90495" transform="rotate(-0 523.335 67.90495)">100</text> | |
| </g> | |
| </g> | |
| <g id="ytick_22"> | |
| <g id="line2d_55"> | |
| <path d="M 530.335 40.792553 | |
| L 731.27952 40.792553 | |
| " clip-path="url(#p7cef315271)" style="fill: none; stroke: #ded9cd; stroke-width: 0.8; stroke-linecap: square"/> | |
| </g> | |
| <g id="line2d_56"/> | |
| <g id="text_40"> | |
| <text style="font-size: 9px; font-family: 'DejaVu Sans'; text-anchor: end; fill: #87867f" x="523.335" y="44.211849" transform="rotate(-0 523.335 44.211849)">120</text> | |
| </g> | |
| </g> | |
| <g id="text_41"> | |
| <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #3d3d3a" x="500.076563" y="100.854563" transform="rotate(-90 500.076563 100.854563)">Wall time (s)</text> | |
| </g> | |
| </g> | |
| <g id="patch_27"> | |
| <path d="M 539.468842 182.951158 | |
| L 557.147245 182.951158 | |
| L 557.147245 71.593584 | |
| L 539.468842 71.593584 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_28"> | |
| <path d="M 588.575518 182.951158 | |
| L 606.253922 182.951158 | |
| L 606.253922 62.116343 | |
| L 588.575518 62.116343 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_29"> | |
| <path d="M 637.682195 182.951158 | |
| L 655.360598 182.951158 | |
| L 655.360598 51.454448 | |
| L 637.682195 51.454448 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_30"> | |
| <path d="M 686.788871 182.951158 | |
| L 704.467275 182.951158 | |
| L 704.467275 58.562378 | |
| L 686.788871 58.562378 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #6a8caf"/> | |
| </g> | |
| <g id="patch_31"> | |
| <path d="M 557.147245 182.951158 | |
| L 574.825649 182.951158 | |
| L 574.825649 49.085138 | |
| L 557.147245 49.085138 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_32"> | |
| <path d="M 606.253922 182.951158 | |
| L 623.932325 182.951158 | |
| L 623.932325 39.607898 | |
| L 606.253922 39.607898 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_33"> | |
| <path d="M 655.360598 182.951158 | |
| L 673.039002 182.951158 | |
| L 673.039002 26.576692 | |
| L 655.360598 26.576692 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_34"> | |
| <path d="M 704.467275 182.951158 | |
| L 722.145678 182.951158 | |
| L 722.145678 37.238587 | |
| L 704.467275 37.238587 | |
| z | |
| " clip-path="url(#p7cef315271)" style="fill: #d97757"/> | |
| </g> | |
| <g id="patch_35"> | |
| <path d="M 530.335 182.951158 | |
| L 530.335 18.757969 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="patch_36"> | |
| <path d="M 530.335 182.951158 | |
| L 731.27952 182.951158 | |
| " style="fill: none; stroke: #d1cfc5; stroke-linejoin: miter; stroke-linecap: square"/> | |
| </g> | |
| <g id="text_42"> | |
| <text style="font-size: 13px; font-family: 'DejaVu Sans'; text-anchor: middle; fill: #141413" x="630.80726" y="12.757969" transform="rotate(-0 630.80726 12.757969)">Wall time</text> | |
| </g> | |
| </g> | |
| </g> | |
| <defs> | |
| <clipPath id="p8c66debf8d"> | |
| <rect x="43.597188" y="18.757969" width="200.94452" height="164.193189"/> | |
| </clipPath> | |
| <clipPath id="pf585a739e0"> | |
| <rect x="285.533125" y="18.757969" width="200.94452" height="164.193189"/> | |
| </clipPath> | |
| <clipPath id="p7cef315271"> | |
| <rect x="530.335" y="18.757969" width="200.94452" height="164.193189"/> | |
| </clipPath> | |
| </defs> | |
| </svg> | |
| </div> | |
| <p class="chart-caption"> | |
| Same eval ordering across all three panels. Score rises on every eval; | |
| warnings tie or drop; wall time grows roughly uniformly (≈+19% per | |
| eval). | |
| </p> | |
| </div> | |
| <div class="numeric-table-wrap"> | |
| <table class="numeric-table"> | |
| <caption class="caption">Per-eval scores, warnings, and wall time</caption> | |
| <thead> | |
| <tr> | |
| <th scope="col">Eval</th> | |
| <th scope="col" class="metric">Seed score</th> | |
| <th scope="col" class="metric">Cand. score</th> | |
| <th scope="col" class="metric">Δ score</th> | |
| <th scope="col" class="metric">Seed warn</th> | |
| <th scope="col" class="metric">Cand. warn</th> | |
| <th scope="col" class="metric">Seed s</th> | |
| <th scope="col" class="metric">Cand. s</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr><th scope="row">numeric-data</th> | |
| <td class="metric">0.930</td><td class="metric">0.970</td> | |
| <td class="metric"><span class="delta-up">+0.040</span></td> | |
| <td class="metric">2</td><td class="metric">1</td> | |
| <td class="metric">94</td><td class="metric">113</td></tr> | |
| <tr><th scope="row">code-review</th> | |
| <td class="metric">0.910</td><td class="metric">0.955</td> | |
| <td class="metric"><span class="delta-up">+0.045</span></td> | |
| <td class="metric">1</td><td class="metric">0</td> | |
| <td class="metric">102</td><td class="metric">121</td></tr> | |
| <tr><th scope="row">module-explainer</th> | |
| <td class="metric">0.925</td><td class="metric">0.944</td> | |
| <td class="metric"><span class="delta-up">+0.019</span></td> | |
| <td class="metric">1</td><td class="metric">1</td> | |
| <td class="metric">111</td><td class="metric">132</td></tr> | |
| <tr><th scope="row">implementation-plan</th> | |
| <td class="metric">0.905</td><td class="metric">0.993</td> | |
| <td class="metric"><span class="delta-up">+0.088</span></td> | |
| <td class="metric">1</td><td class="metric">0</td> | |
| <td class="metric">105</td><td class="metric">123</td></tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| <div class="grid" style="--grid-min: 240px;"> | |
| <div class="card stack" data-gap="xs"> | |
| <div class="card-head"> | |
| <div class="card-titles"> | |
| <span class="eyebrow">numeric-data</span> | |
| <span class="card-title">+0.040</span> | |
| </div> | |
| <span class="badge">warn 2→1</span> | |
| </div> | |
| <p>Candidate chart is clearer but uses more SVG text — watch label | |
| density on narrow viewports.</p> | |
| </div> | |
| <div class="card stack" data-gap="xs"> | |
| <div class="card-head"> | |
| <div class="card-titles"> | |
| <span class="eyebrow">code-review</span> | |
| <span class="card-title">+0.045</span> | |
| </div> | |
| <span class="badge">warn 1→0</span> | |
| </div> | |
| <p>No mechanical regression. Inspect finding-severity wording and | |
| long diff wrapping before committing the seed.</p> | |
| </div> | |
| <div class="card stack" data-gap="xs"> | |
| <div class="card-head"> | |
| <div class="card-titles"> | |
| <span class="eyebrow">module-explainer</span> | |
| <span class="card-title">+0.019</span> | |
| </div> | |
| <span class="badge">warn 1→1</span> | |
| </div> | |
| <p>Runtime-path narrative improved; mobile flow caption stays dense | |
| — the remaining warning sits here.</p> | |
| </div> | |
| <div class="card stack" data-gap="xs"> | |
| <div class="card-head"> | |
| <div class="card-titles"> | |
| <span class="eyebrow">implementation-plan</span> | |
| <span class="card-title">+0.088</span> | |
| </div> | |
| <span class="badge">warn 1→0</span> | |
| </div> | |
| <p>Plan quality is the biggest single-eval lift; no checker | |
| regression observed.</p> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- FINDINGS / REGRESSIONS --> | |
| <section class="section stack" data-gap="md"> | |
| <div class="section-head"> | |
| <div class="eyebrow">Findings</div> | |
| <h2>Improvements, regressions, and what to verify by hand</h2> | |
| </div> | |
| <div class="grid" style="--grid-min: 260px;"> | |
| <div class="panel stack" data-gap="sm"> | |
| <div class="eyebrow">Improvements</div> | |
| <ul class="checklist"> | |
| <li>All 4 artifacts still generate cleanly (4 / 4 generated_ok).</li> | |
| <li>Checker score lifts 0.85 → 0.94 (+0.09) on the same rubric.</li> | |
| <li>Warnings fall on 3 of 4 evals; tie on <em>module-explainer</em>.</li> | |
| <li><em>implementation-plan</em> contributes the largest per-eval | |
| score lift (+0.088).</li> | |
| </ul> | |
| </div> | |
| <div class="panel stack" data-gap="sm"> | |
| <div class="eyebrow">Regressions / cost</div> | |
| <ul class="plain-list"> | |
| <li>Wall time +77 s (+18.7%) — uniform per eval, not a single outlier.</li> | |
| <li>Total tokens +8.7k (+7.5%); cost +$0.22 (+11.5%).</li> | |
| <li>Hygiene score slips 1.00 → 0.99 — within noise floor but worth | |
| re-checking unsupported-class penalties.</li> | |
| <li><em>numeric-data</em> uses more SVG text — verify mobile chart | |
| readability.</li> | |
| </ul> | |
| </div> | |
| <div class="panel stack" data-gap="sm"> | |
| <div class="eyebrow">Manual gate before promotion</div> | |
| <ul class="plain-list"> | |
| <li>Inspect <em>code-review</em> long diff wrapping | |
| (<code>data-wrap="true"</code> on <code>.diff</code>).</li> | |
| <li>Eyeball <em>module-explainer</em> mobile flow caption density.</li> | |
| <li>Spot-check that hygiene drop is not a new unsupported class.</li> | |
| </ul> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- SCORING + PROVENANCE --> | |
| <section class="section stack" data-gap="md"> | |
| <div class="section-head"> | |
| <div class="eyebrow">Scoring & provenance</div> | |
| <h2>How the final score is computed</h2> | |
| </div> | |
| <div class="grid" style="--grid-min: 280px;"> | |
| <div class="panel stack" data-gap="sm"> | |
| <div class="eyebrow">Formula</div> | |
| <pre class="code-block formula" data-wrap="true"><code>final_score = 0.30 * generation_score | |
| + 0.55 * checker_score | |
| + 0.15 * hygiene_score | |
| generation_score = generated_ok / total_evals | |
| checker_score = max(0, 1 − 0.20 * checker_failures | |
| − 0.03 * checker_warnings) | |
| hygiene_score = avg(artifact compliance after | |
| unsupported-class, external-asset, | |
| and missing-CSS penalties)</code></pre> | |
| <p class="caption"> | |
| Worked example for Candidate B: 0.30·1.00 + 0.55·0.94 + 0.15·0.99 | |
| = 0.300 + 0.517 + 0.1485 = <strong>0.9655</strong>. | |
| </p> | |
| </div> | |
| <div class="panel stack" data-gap="sm"> | |
| <div class="eyebrow">Run provenance</div> | |
| <dl class="stack" data-gap="xs"> | |
| <div class="cluster" data-gap="sm"><dt class="caption">Model</dt><dd><code>codexresponses.gpt-5.5</code></dd></div> | |
| <div class="cluster" data-gap="sm"><dt class="caption">Skill versions</dt><dd><code>birch-html seed</code> · <code>birch-html candidate-b</code></dd></div> | |
| <div class="cluster" data-gap="sm"><dt class="caption">Eval set</dt><dd>numeric-data, code-review, module-explainer, implementation-plan</dd></div> | |
| <div class="cluster" data-gap="sm"><dt class="caption">Timestamp</dt><dd>2026-05-20T18:40:00Z</dd></div> | |
| </dl> | |
| <div class="eyebrow">Reports</div> | |
| <pre class="code-block" data-wrap="true"><code>eval-runs/reports/skill-seed-gpt55/summary.json | |
| eval-runs/reports/skill-candidate-b-gpt55/summary.json</code></pre> | |
| <div class="eyebrow">Reproduce</div> | |
| <pre class="code-block copyable" data-wrap="true"><code>python3 scripts/run_skill_evals.py \ | |
| --skill-dir birch-html \ | |
| --model codexresponses.gpt-5.5 \ | |
| --label skill-candidate-b-gpt55</code></pre> | |
| </div> | |
| </div> | |
| </section> | |
| <!-- CAVEATS --> | |
| <section class="section stack" data-gap="md"> | |
| <div class="section-head"> | |
| <div class="eyebrow">Caveats</div> | |
| <h2>What this comparison does <em>not</em> tell you</h2> | |
| </div> | |
| <div class="callout stack" data-gap="sm"> | |
| <span class="callout-label">Comparability and measurement limits</span> | |
| <ul class="plain-list"> | |
| <li><strong>Token counts are not apples-to-apples.</strong> Provider | |
| token schemas differ; cached, reasoning, and effective-token fields | |
| are not reconciled across runs.</li> | |
| <li><strong>Checker warnings are mechanical contract signals,</strong> | |
| not a full content-quality verdict — a clean checker run can still | |
| ship dense or unclear prose.</li> | |
| <li><strong>Same-vs-same screenshot pairs</strong> mostly validate | |
| geometry and rendering contracts; they do not score aesthetics.</li> | |
| <li><strong>Wall time mixes model latency with harness overhead;</strong> | |
| sub-20 s differences should be treated as noisy.</li> | |
| <li><strong>n = 1 per eval per candidate.</strong> No re-runs or | |
| seed sweep yet — promotion is a working hypothesis, not a stat.</li> | |
| </ul> | |
| </div> | |
| </section> | |
| <!-- NEXT EXPERIMENT --> | |
| <section class="section stack" data-gap="md"> | |
| <div class="section-head"> | |
| <div class="eyebrow">Next experiment</div> | |
| <h2>What to run after promotion</h2> | |
| </div> | |
| <ol class="flow-list"> | |
| <li class="flow-step"> | |
| <span class="flow-num">1</span> | |
| <div class="stack" data-gap="xs"> | |
| <span class="flow-title">Promote Candidate B as the GEPA seed</span> | |
| <span class="flow-detail">Single-model GEPA round on | |
| <code>codexresponses.gpt-5.5</code>, holding the eval set fixed | |
| at the four current artifacts.</span> | |
| </div> | |
| </li> | |
| <li class="flow-step"> | |
| <span class="flow-num">2</span> | |
| <div class="stack" data-gap="xs"> | |
| <span class="flow-title">Manual inspection gate</span> | |
| <span class="flow-detail">Before committing, open the | |
| <em>code-review</em> artifact and verify long diff wrapping with | |
| <code>data-wrap="true"</code>; eyeball the | |
| <em>module-explainer</em> mobile flow caption.</span> | |
| </div> | |
| </li> | |
| <li class="flow-step"> | |
| <span class="flow-num">3</span> | |
| <div class="stack" data-gap="xs"> | |
| <span class="flow-title">Cross-model rerun</span> | |
| <span class="flow-detail">Replay the GEPA-best candidate on Kimi | |
| and Gemini to check that the warning reduction is not | |
| model-specific.</span> | |
| </div> | |
| </li> | |
| <li class="flow-step"> | |
| <span class="flow-num">4</span> | |
| <div class="stack" data-gap="xs"> | |
| <span class="flow-title">Broad model-suite validation</span> | |
| <span class="flow-detail">Only after the cross-model spot-check | |
| passes: run the full model suite and re-evaluate cost / latency | |
| budgets against the +11.5% / +18.7% headroom seen here.</span> | |
| </div> | |
| </li> | |
| <li class="flow-step"> | |
| <span class="flow-num">5</span> | |
| <div class="stack" data-gap="xs"> | |
| <span class="flow-title">Tighten the checker</span> | |
| <span class="flow-detail">With warnings at 2, the next marginal | |
| gain probably lives in turning current warnings into failures | |
| for clearly-broken contracts (e.g. unwrapped wide diffs) so the | |
| score signal stays informative.</span> | |
| </div> | |
| </li> | |
| </ol> | |
| </section> | |
| <footer class="stack" data-gap="xs"> | |
| <p class="caption"> | |
| Source: <code>evals/benchmark-comparison/source.json</code> · | |
| Charts generated with <code>scripts/birch_mpl.py</code> + Matplotlib · | |
| Decision: <strong>promote Candidate B</strong>. | |
| </p> | |
| </footer> | |
| </main> | |
| </body> | |
| </html> | |
Xet Storage Details
- Size:
- 97.4 kB
- Xet hash:
- f7b735edb291b4eac52586605af6be946363f84c81bbd5c44c1ced7800fb7b99
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.