Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
theformatisvalid
/
tokenizers-training
like
0
Sleeping
App
Files
Files
Community
Fetching metadata from the HF Docker repository...
main
tokenizers-training
/
src
/
tokenizers
/
bpe_v8k_f2
/
tokenizer_config.json
theformatisvalid
Upload 12 files
7cc2359
verified
4 months ago
raw
Copy download link
history
blame
contribute
delete
Safe
123 Bytes
{
"unk_token"
:
"<UNK>"
,
"cls_token"
:
null
,
"sep_token"
:
null
,
"mask_token"
:
null
,
"model_max_length"
:
512
}