SOTAagi2030
/

Keystone-Trail-Code-Tokenizer-345

tokenizers

emergency-communications

tokenizer

Model card Files Files and versions

xet

Community

Keystone-Trail-Code-Tokenizer-345 / tokenizer /tokenizer-config.json

SOTAagi2030

Add Keystone trail-code tokenizer release KT-2026-04

ff44873 verified 10 days ago

Raw History Blame Contribute Delete

194 Bytes

{
  "release_id": "KT-2026-04",
  "model_max_length": 128,
  "reserved_tokens": [
    "[PAD]",
    "[UNK]",
    "[BOS]",
    "[EOS]"
  ],
  "ordinary_token_budget": 6,
  "vocabulary_size": 10
}

	{
	"release_id": "KT-2026-04",
	"model_max_length": 128,
	"reserved_tokens": [
	"[PAD]",
	"[UNK]",
	"[BOS]",
	"[EOS]"
	],
	"ordinary_token_budget": 6,
	"vocabulary_size": 10
	}