RyeAI
/

DA-BPE-LLAMA3.2-TOKEN

BPE Tokenization

Model card Files Files and versions

DA-BPE-LLAMA3.2-TOKEN / special_tokens_map.json

MikkelWK's picture

Rename DA BPE Llama 3.2 Special Tokens Map.json to special_tokens_map.json

5c50ea3 verified over 1 year ago

history blame contribute delete

141 Bytes

	{
	"pad_token": {
	"content": "[PAD]",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false
	}
	}