Rzoro
/

erebus-small

Text Generation

foundation-model

Model card Files Files and versions

erebus-small / config.json

Rzoro's picture

Add Erebus-small checkpoint at step 42000 (42% trained, loss~4.1)

0ed640a verified 15 days ago

history blame contribute delete

135 Bytes

	{
	"vocab_size": 50257,
	"d_model": 768,
	"n_heads": 12,
	"n_layers": 12,
	"d_ff": 3072,
	"max_seq_len": 1024,
	"dropout": 0.1
	}