zkolter's picture
Update params.json
fea2dfb verified
raw
history blame contribute delete
148 Bytes
{
"dim": 2048,
"n_layers": 16,
"n_heads": 32,
"vocab_size": 128256,
"ffn_dim_multiplier": 4,
"norm_eps": 1e-05,
"max_seq_len": 4096
}