tomzhengy
/

nanochat-tokenizer

Model card Files Files and versions

nanochat-tokenizer

1.11 MB

Ctrl+K

Ctrl+K

1 contributor

History: 2 commits

tomzhengy's picture

Upload folder using huggingface_hub

1ac793d verified 4 months ago

.gitattributes

1.52 kB
initial commit 4 months ago
token_bytes.pt
Detected Pickle imports (3)
- "torch.IntStorage",
- "torch._utils._rebuild_tensor_v2",
- "collections.OrderedDict"
What is a pickle import?
264 kB
xet

Upload folder using huggingface_hub 4 months ago
tokenizer.pkl
Detected Pickle imports (1)
- "tiktoken.core.Encoding"
How to fix it?
847 kB
xet

Upload folder using huggingface_hub 4 months ago