RinKana's picture
MiniGPT-30M with RMSNorm+RoPE+SwiGLU after 3 epochs (manual load only)
b10e977 verified
{
"vocab_size": 50257,
"n_positions": 512,
"n_layer": 6,
"n_head": 8,
"n_embd": 384,
"architectures": [
"MiniGPT"
],
"model_type": "gpt2"
}