Hugging Face's logo

Buckets:
hf-doc-build
/
doc-dev

Files
xet
hf-doc-build/doc-dev / tokenizers /pr_1968 /en /api
151 GB
2,492,593 files
Updated 8 minutes ago
NameSize
added-tokens.html54.6 kB
xet
added-tokens.md2.56 kB
xet
decoders.html80.8 kB
xet
decoders.md4.78 kB
xet
encode-inputs.html20.3 kB
xet
encode-inputs.md2.08 kB
xet
encoding.html159 kB
xet
encoding.md9.37 kB
xet
input-sequences.html18.8 kB
xet
input-sequences.md1.17 kB
xet
models.html134 kB
xet
models.md8.51 kB
xet
normalizers.html90.4 kB
xet
normalizers.md4.48 kB
xet
post-processors.html56.9 kB
xet
post-processors.md5.02 kB
xet
pre-tokenizers.html108 kB
xet
pre-tokenizers.md8.19 kB
xet
tokenizer.html287 kB
xet
tokenizer.md18.8 kB
xet
trainers.html76.4 kB
xet
trainers.md4.31 kB
xet
visualizer.html36.2 kB
xet
visualizer.md2.15 kB
xet
Total size
151 GB
Files
2,492,593
Last updated
May 16
Pre-warmed CDN
US EU US EU

Contributors