metadata
license: apache-2.0
Foundry-4
Kernel-specialized hybrid LM: 5.02B total / ~0.58B active params. 18 GDN (gated DeltaNet, fla) + 6 RoPE attention layers, 384-expert sigmoid-routed dropless MoE (top-8, packed Triton segmm), KernelGram n-gram tables, 4-iteration shared-weight latent ponder loop, JEV-style judge head.
Trained 2000 steps on a curated kernel-optimization corpus (CUDA/Triton episodes with measured RESULT/BEST/HISTORY feedback) plus GPU-verified synthetic optimization ladders (synth4.py).
Code: model4.py / gdn.py / segmm.py in this repo.