manasred
/

picochat

Text Generation

Model card Files Files and versions

picochat / config.json

manasred's picture

Update config for depth-4 architecture

16c4152 verified 22 days ago

history blame contribute delete

142 Bytes

	{
	"sequence_len": 2048,
	"vocab_size": 4096,
	"n_layer": 4,
	"n_head": 4,
	"n_kv_head": 2,
	"n_embd": 256,
	"window_pattern": "SSSL"
	}