timaeus
/

GPT2-tokenizer

Model card Files Files and versions

3.36 MB

2 contributors

History: 1 commit

svwingerden's picture

georgeyw's picture

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx

138dbbe verified about 1 year ago

.gitattributes
1.52 kB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx about 1 year ago
added_tokens.json
29 Bytes

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx about 1 year ago
merges.txt
456 kB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx about 1 year ago
special_tokens_map.json
131 Bytes

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx about 1 year ago
tokenizer.json
2.11 MB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx about 1 year ago
tokenizer_config.json
559 Bytes

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx about 1 year ago
vocab.json
798 kB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx about 1 year ago