| license: apache-2.0 | |
| # Foundry-4 | |
| Kernel-specialized hybrid LM: 5.02B total / ~0.58B active params. | |
| 18 GDN (gated DeltaNet, fla) + 6 RoPE attention layers, 384-expert | |
| sigmoid-routed dropless MoE (top-8, packed Triton segmm), KernelGram | |
| n-gram tables, 4-iteration shared-weight latent ponder loop, JEV-style | |
| judge head. | |
| Trained 2000 steps on a curated kernel-optimization corpus | |
| (CUDA/Triton episodes with measured RESULT/BEST/HISTORY feedback) plus | |
| GPU-verified synthetic optimization ladders (synth4.py). | |
| Code: model4.py / gdn.py / segmm.py in this repo. | |