Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
hf-internal-testing
/
llama3-tokenizer-tiktoken
like
0
Follow
Hugging Face Internal Testing Organization
170
test
tiktoken
tokenizer
Model card
Files
Files and versions
xet
Community
Copy to bucket
new
main
llama3-tokenizer-tiktoken
2.19 MB
Ctrl+K
Ctrl+K
1 contributor
History:
4 commits
itazap
HF Staff
Delete tokenizer.model
a1d81e8
verified
4 months ago
.gitattributes
Safe
1.52 kB
initial commit
4 months ago
README.md
845 Bytes
Add tiktoken tokenizer files from Meta-Llama-3-8B-Instruct/original
4 months ago
tiktoken.model
Safe
2.18 MB
xet
Add tiktoken tokenizer files from Meta-Llama-3-8B-Instruct/original
4 months ago
