ynnning
/

atlas-tokenizer

Model card Files Files and versions

atlas-tokenizer

546 kB

Ctrl+K

Ctrl+K

1 contributor

History: 3 commits

ynnning's picture

Upload token_bytes.pt with huggingface_hub

7230982 verified 3 months ago

.gitattributes

1.52 kB
initial commit 3 months ago
token_bytes.pt

133 kB
xet

Upload token_bytes.pt with huggingface_hub 3 months ago
tokenizer.pkl
Detected Pickle imports (1)
- "tiktoken.core.Encoding"
How to fix it?
412 kB
xet

Upload tokenizer.pkl with huggingface_hub 3 months ago