POG-E4B-v1 / v21 /adapter_config.json

v2 program: v2.1 multi-dataset variant, GGUF layer-trim early-exit (bit-exact), server-mode extractor, 5-task BEIR sweep, binary/int8 compression study, full ablation writeup

d27cbf8 verified about 1 month ago

Raw

History Blame Contribute Delete

300 Bytes

	{
	"d_model": 2560,
	"n_layers": 3,
	"n_slots": 6,
	"d_attn": 512,
	"d_gate": 256,
	"d_ff": 3072,
	"d_out": 2560,
	"matryoshka_dims": [
	256,
	512,
	1024,
	2560
	],
	"backbone": "unsloth/gemma-4-E4B-it-qat-GGUF (UD-Q4_K_XL)",
	"intercept_layers": [
	28,
	32,
	36
	]
	}