einsof2-deepseek-v4-stiefel-adapter / tokenizer_config.json
Polydim-Einsof's picture
Upload EINSOF-II DeepSeek-V4.1 Flash 16B MoE Stiefel Polar LoRA Adapter with order 4 Stiefel retraction (Exit Code 0)
fec0b49 verified
Raw History Blame Contribute Delete
396 Bytes
{
"backend": "tokenizers",
"bos_token": "<|begin▁of▁sentence|>",
"clean_up_tokenization_spaces": false,
"eos_token": "<|end▁of▁sentence|>",
"is_local": false,
"legacy": true,
"local_files_only": false,
"model_max_length": 16384,
"pad_token": "<|end▁of▁sentence|>",
"sp_model_kwargs": {},
"tokenizer_class": "TokenizersBackend",
"unk_token": null
}