📈 How far can the model grow? (theoretical)

Gated architecture · input 10 features · top-3 gate on layer 1 · remaining 7 join every layer after · log-scaled parameter axis · counts measured from real model builds
100 1K 10K 100K 1M 1 2 3 4 5 6 7 8 9 10 current best: 4,216 params (2 layers) theoretical ceiling: 345,352 params layers in genome (current cap: 6 — dashed line beyond is extrapolated) parameters (log scale) MAX growth: 256 units/lyr + BN MIN growth: 16 units/lyr, no BN extrapolated beyond cap
LayersMAX params (256u + BN)MIN params (16u, no BN)Note
12,31288
270,920472
3139,528856
4208,1361,240
5276,7441,624
6345,3522,008current cap
7*413,960—if cap lifted
8*482,568—if cap lifted
9*551,176—if cap lifted
10*619,784—if cap lifted
* rows 7–10 are extrapolated: the genome pool currently caps architectures at 6 layers × 256 units (mutation can only add layers up to that limit). If that cap were lifted, the ceiling keeps climbing linearly (~68,608 params per extra 256-unit layer). Beyond the data: on CPU (this machine) a ~1M-param model takes ~48s per 10-epoch eval vs <1s for the current best — so the cap is a practical guardrail, not a hard limit.
Current best (saved)
4,216 params · 2 layers
Theoretical ceiling @ current cap
345,352 params · 6 layers