icarus112
/

prod9_b16_checkpoint

Model card Files Files and versions

prod9_b16_checkpoint / bootstrap /prod9_tokenizer

1.11 MB

Ctrl+K

Ctrl+K

1 contributor

History: 4 commits

icarus112's picture

Fix: prod9 65536 vocab tokenizer (token_bytes.pt)

61c891d verified about 1 month ago

token_bytes.pt
Detected Pickle imports (3)
- "torch.IntStorage",
- "torch._utils._rebuild_tensor_v2",
- "collections.OrderedDict"
What is a pickle import?
264 kB
xet

Fix: prod9 65536 vocab tokenizer (token_bytes.pt) about 1 month ago
tokenizer.pkl
Detected Pickle imports (1)
- "tiktoken.core.Encoding"
How to fix it?
844 kB
xet

Fix: prod9 65536 vocab tokenizer (tokenizer.pkl) about 1 month ago