File size: 180 Bytes
afafadc | 1 2 3 4 5 | layer 20: n_tok=1 n_expert_used=8 n_expert=128 n_embd=1536 n_ff=512 distinct experts: 8 max|got-ref| = 0.0261884 max|ref| = 1.9054 max_rel = 0.0137443 corr = 0.99985581 PASS |
afafadc | 1 2 3 4 5 | layer 20: n_tok=1 n_expert_used=8 n_expert=128 n_embd=1536 n_ff=512 distinct experts: 8 max|got-ref| = 0.0261884 max|ref| = 1.9054 max_rel = 0.0137443 corr = 0.99985581 PASS |