| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <title>📈 Theoretical Model Growth — Self-Evolving Neural Network</title> |
| <style> |
| :root { --bg:#0d1117; --surface:#161b22; --border:#30363d; --text:#e6edf3; |
| --dim:#8b949e; --accent:#58a6ff; --green:#3fb950; --red:#f85149; --yellow:#d29922; } |
| * { margin:0; padding:0; box-sizing:border-box; } |
| body { background:var(--bg); color:var(--text); font-family:'SF Mono','Fira Code',monospace; |
| padding:24px; min-height:100vh; } |
| .wrap { max-width:1060px; margin:0 auto; } |
| h1 { color:var(--green); font-size:1.5em; margin-bottom:6px; } |
| .sub { color:var(--dim); font-size:.85em; margin-bottom:20px; } |
| .card { background:var(--surface); border:1px solid var(--border); border-radius:10px; |
| padding:20px; margin-bottom:20px; } |
| .svgbox { width:100%; overflow-x:auto; } |
| .svgbox svg { width:100%; height:auto; min-width:760px; } |
| table { width:100%; border-collapse:collapse; font-size:.85em; } |
| th, td { padding:7px 12px; text-align:right; border-bottom:1px solid var(--border); } |
| th { color:var(--accent); } |
| th:first-child, td:first-child { text-align:center; } |
| td:nth-child(2) { color:var(--green); font-weight:bold; } |
| td:nth-child(3) { color:var(--dim); } |
| .capnote { margin-top:14px; color:var(--dim); font-size:.8em; line-height:1.7; } |
| .capnote b { color:var(--red); } |
| .grid2 { display:grid; grid-template-columns:1fr 1fr; gap:14px; margin-top:14px; } |
| .stat { background:var(--bg); border:1px solid var(--border); border-radius:8px; padding:14px; } |
| .stat .k { color:var(--dim); font-size:.75em; text-transform:uppercase; letter-spacing:1px; } |
| .stat .v { font-size:1.35em; font-weight:bold; margin-top:6px; } |
| .stat .v.green { color:var(--green); } .stat .v.red { color:var(--red); } |
| @media (max-width:700px) { .grid2 { grid-template-columns:1fr; } } |
| </style> |
| </head> |
| <body> |
| <div class="wrap"> |
| <h1>📈 How far can the model grow? (theoretical)</h1> |
| <div class="sub">Gated architecture · input 10 features · top-3 gate on layer 1 · remaining 7 join every layer after · log-scaled parameter axis · counts measured from real model builds</div> |
|
|
| <div class="card"> |
| <div class="svgbox"><svg xmlns="http://www.w3.org/2000/svg" width="1000" height="640" viewBox="0 0 1000 640"> |
| <line x1="90.0" y1="550.0" x2="970.0" y2="550.0" stroke="#21262d" stroke-width="1"/> |
| <text x="78.0" y="554.0" fill="#8b949e" font-size="12" font-family="monospace" font-weight="normal" text-anchor="end">100</text> |
| <line x1="90.0" y1="430.0" x2="970.0" y2="430.0" stroke="#21262d" stroke-width="1"/> |
| <text x="78.0" y="434.0" fill="#8b949e" font-size="12" font-family="monospace" font-weight="normal" text-anchor="end">1K</text> |
| <line x1="90.0" y1="310.0" x2="970.0" y2="310.0" stroke="#21262d" stroke-width="1"/> |
| <text x="78.0" y="314.0" fill="#8b949e" font-size="12" font-family="monospace" font-weight="normal" text-anchor="end">10K</text> |
| <line x1="90.0" y1="190.0" x2="970.0" y2="190.0" stroke="#21262d" stroke-width="1"/> |
| <text x="78.0" y="194.0" fill="#8b949e" font-size="12" font-family="monospace" font-weight="normal" text-anchor="end">100K</text> |
| <line x1="90.0" y1="70.0" x2="970.0" y2="70.0" stroke="#21262d" stroke-width="1"/> |
| <text x="78.0" y="74.0" fill="#8b949e" font-size="12" font-family="monospace" font-weight="normal" text-anchor="end">1M</text> |
| <line x1="90.0" y1="550.0" x2="90.0" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="90.0" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">1</text> |
| <line x1="187.8" y1="550.0" x2="187.8" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="187.8" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">2</text> |
| <line x1="285.6" y1="550.0" x2="285.6" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="285.6" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">3</text> |
| <line x1="383.3" y1="550.0" x2="383.3" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="383.3" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">4</text> |
| <line x1="481.1" y1="550.0" x2="481.1" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="481.1" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">5</text> |
| <line x1="578.9" y1="550.0" x2="578.9" y2="556.0" stroke="#f85149" stroke-width="2"/> |
| <text x="578.9" y="572.0" fill="#f85149" font-size="14" font-family="monospace" font-weight="bold" text-anchor="middle">6</text> |
| <line x1="676.7" y1="550.0" x2="676.7" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="676.7" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">7</text> |
| <line x1="774.4" y1="550.0" x2="774.4" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="774.4" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">8</text> |
| <line x1="872.2" y1="550.0" x2="872.2" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="872.2" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">9</text> |
| <line x1="970.0" y1="550.0" x2="970.0" y2="556.0" stroke="#8b949e" stroke-width="1"/> |
| <text x="970.0" y="572.0" fill="#e6edf3" font-size="12" font-family="monospace" font-weight="normal" text-anchor="middle">10</text> |
| <line x1="578.9" y1="70.0" x2="578.9" y2="550.0" stroke="#f85149" stroke-width="1.5" stroke-dasharray="6 5" opacity="0.55"/> |
| <polyline points="90.0,386.3 187.8,207.9 285.6,172.6 383.3,151.8 481.1,137.0 578.9,125.4" fill="none" stroke="#3fb950" stroke-width="3" stroke-linejoin="round" stroke-linecap="round" opacity="1.0"/> |
| <polyline points="90.0,550.0 187.8,469.1 285.6,438.1 383.3,418.8 481.1,404.7 578.9,393.7" fill="none" stroke="#8b949e" stroke-width="2" stroke-linejoin="round" stroke-linecap="round" stroke-dasharray="4 4" opacity="1.0"/> |
| <polyline points="676.7,116.0 774.4,108.0 872.2,101.0 970.0,94.9" fill="none" stroke="#58a6ff" stroke-width="2.5" stroke-linejoin="round" stroke-linecap="round" stroke-dasharray="7 5" opacity="0.8"/> |
| <circle cx="90.0" cy="386.3" r="5" fill="#3fb950" stroke="#0d1117" stroke-width="2"/> |
| <circle cx="187.8" cy="207.9" r="5" fill="#3fb950" stroke="#0d1117" stroke-width="2"/> |
| <circle cx="285.6" cy="172.6" r="5" fill="#3fb950" stroke="#0d1117" stroke-width="2"/> |
| <circle cx="383.3" cy="151.8" r="5" fill="#3fb950" stroke="#0d1117" stroke-width="2"/> |
| <circle cx="481.1" cy="137.0" r="5" fill="#3fb950" stroke="#0d1117" stroke-width="2"/> |
| <circle cx="578.9" cy="125.4" r="5" fill="#3fb950" stroke="#0d1117" stroke-width="2"/> |
| <circle cx="676.7" cy="116.0" r="3.5" fill="#58a6ff" stroke="#0d1117" stroke-width="1.5"/> |
| <circle cx="774.4" cy="108.0" r="3.5" fill="#58a6ff" stroke="#0d1117" stroke-width="1.5"/> |
| <circle cx="872.2" cy="101.0" r="3.5" fill="#58a6ff" stroke="#0d1117" stroke-width="1.5"/> |
| <circle cx="970.0" cy="94.9" r="3.5" fill="#58a6ff" stroke="#0d1117" stroke-width="1.5"/> |
| <circle cx="187.8" cy="355.0" r="7" fill="#d29922" stroke="#0d1117" stroke-width="2.5"/> |
| <text x="201.8" y="347.0" fill="#d29922" font-size="13" font-family="monospace" font-weight="bold" text-anchor="start">current best: 4,216 params (2 layers)</text> |
| <circle cx="578.9" cy="125.4" r="8" fill="#f85149" stroke="#0d1117" stroke-width="3"/> |
| <text x="564.9" y="109.4" fill="#f85149" font-size="13" font-family="monospace" font-weight="bold" text-anchor="end">theoretical ceiling: 345,352 params</text> |
| <text x="530.0" y="610.0" fill="#8b949e" font-size="13" font-family="monospace" font-weight="normal" text-anchor="middle">layers in genome (current cap: 6 — dashed line beyond is extrapolated)</text> |
| <text x="18.0" y="310.0" fill="#8b949e" font-size="13" font-family="monospace" font-weight="normal" text-anchor="middle">parameters (log scale)</text> |
| <rect x="700" y="76" width="256" height="86" rx="6" fill="#161b22" stroke="#30363d"/> |
| <line x1="712" y1="92" x2="740" y2="92" stroke="#3fb950" stroke-width="3"/> |
| <text x="748.0" y="96.0" fill="#e6edf3" font-size="11" font-family="monospace" font-weight="normal" text-anchor="start">MAX growth: 256 units/lyr + BN</text> |
| <line x1="712" y1="116" x2="740" y2="116" stroke="#8b949e" stroke-width="2.5" stroke-dasharray="5 4"/> |
| <text x="748.0" y="120.0" fill="#e6edf3" font-size="11" font-family="monospace" font-weight="normal" text-anchor="start">MIN growth: 16 units/lyr, no BN</text> |
| <line x1="712" y1="140" x2="740" y2="140" stroke="#58a6ff" stroke-width="2.5" stroke-dasharray="5 4"/> |
| <text x="748.0" y="144.0" fill="#e6edf3" font-size="11" font-family="monospace" font-weight="normal" text-anchor="start">extrapolated beyond cap</text> |
| </svg></div> |
| </div> |
|
|
| <div class="card"> |
| <table> |
| <thead><tr><th>Layers</th><th>MAX params (256u + BN)</th><th>MIN params (16u, no BN)</th><th>Note</th></tr></thead> |
| <tbody><tr><td>1</td><td>2,312</td><td>88</td><td></td></tr><tr><td>2</td><td>70,920</td><td>472</td><td></td></tr><tr><td>3</td><td>139,528</td><td>856</td><td></td></tr><tr><td>4</td><td>208,136</td><td>1,240</td><td></td></tr><tr><td>5</td><td>276,744</td><td>1,624</td><td></td></tr><tr><td>6</td><td>345,352</td><td>2,008</td><td>current cap</td></tr><tr style='opacity:.55'><td>7*</td><td>413,960</td><td>—</td><td>if cap lifted</td></tr><tr style='opacity:.55'><td>8*</td><td>482,568</td><td>—</td><td>if cap lifted</td></tr><tr style='opacity:.55'><td>9*</td><td>551,176</td><td>—</td><td>if cap lifted</td></tr><tr style='opacity:.55'><td>10*</td><td>619,784</td><td>—</td><td>if cap lifted</td></tr></tbody> |
| </table> |
| <div class="capnote"> |
| <b>* rows 7–10 are extrapolated:</b> the genome pool currently caps architectures at |
| <b>6 layers × 256 units</b> (mutation can only add layers up to that limit). If that cap were |
| lifted, the ceiling keeps climbing linearly (~68,608 params per extra 256-unit layer). |
| Beyond the data: on <b>CPU</b> (this machine) a ~1M-param model takes ~48s per 10-epoch eval vs |
| <1s for the current best — so the cap is a <i>practical</i> guardrail, not a hard limit. |
| </div> |
| </div> |
|
|
| <div class="grid2"> |
| <div class="stat"><div class="k">Current best (saved)</div><div class="v green">4,216 params · 2 layers</div></div> |
| <div class="stat"><div class="k">Theoretical ceiling @ current cap</div><div class="v red">345,352 params · 6 layers</div></div> |
| </div> |
| </div> |
| </body> |
| </html> |
