levos06
/

vel_17M

Text Generation

Model card Files Files and versions

vel_17M / config.json

levos06's picture

Upload model files

3719d13 verified 10 days ago

history blame contribute delete

198 Bytes

	{
	"vocab_size": 50257,
	"dim": 256,
	"n_layers": 4,
	"n_heads": 4,
	"max_seq_len": 512,
	"architecture": "DeepSeekTransformer",
	"components": [
	"RMSNorm",
	"RoPE",
	"SwiGLU"
	]
	}