Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Log In
Sign Up
FoundationVision
/
unitok_tokenizer
like
10
Follow
FoundationVision
112
Feature Extraction
arxiv:
2502.20321
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
1
d013ff3
unitok_tokenizer
10.2 GB
3 contributors
History:
2 commits
machuofan
init
d013ff3
about 1 year ago
external
init
about 1 year ago
.gitattributes
Safe
1.52 kB
initial commit
about 1 year ago
README.md
Safe
31 Bytes
initial commit
about 1 year ago
unitok_tokenizer.pth
pickle
Detected Pickle imports (4)
"torch._utils._rebuild_tensor_v2"
,
"collections.OrderedDict"
,
"torch.FloatStorage"
,
"torch.bfloat16"
How to fix it?
9.93 GB
xet
init
about 1 year ago