Rzoro
/

erebus-medium

Text Generation

foundation-model

Model card Files Files and versions

erebus-medium / config.json

Rzoro's picture

Add Erebus-medium checkpoint at step 20000 (20% trained, loss~8.79)

ba0c987 verified 2 days ago

history blame contribute delete

136 Bytes

	{
	"vocab_size": 50257,
	"d_model": 1024,
	"n_heads": 16,
	"n_layers": 24,
	"d_ff": 4096,
	"max_seq_len": 1024,
	"dropout": 0.1
	}