Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
wikilangs
/
tok
like
0
Follow
WikiLangs
5
Text Generation
fastText
omarkamali/wikipedia-monthly
Toki Pona
wikilangs
nlp
tokenizer
embeddings
n-gram
markov
wikipedia
feature-extraction
sentence-similarity
tokenization
n-grams
markov-chain
text-mining
babelvec
vocabulous
vocabulary
monolingual
family-constructed_auxlang
License:
mit
Model card
Files
Files and versions
xet
Community
Use this model
main
tok
/
models
/
tokenizer
4.74 MB
1 contributor
History:
1 commit
omarkamali
Upload all models and assets for tok (latest)
1726dc8
verified
2 days ago
tok_tokenizer_16k.model
498 kB
xet
Upload all models and assets for tok (latest)
2 days ago
tok_tokenizer_16k.vocab
223 kB
Upload all models and assets for tok (latest)
2 days ago
tok_tokenizer_32k.model
780 kB
xet
Upload all models and assets for tok (latest)
2 days ago
tok_tokenizer_32k.vocab
489 kB
Upload all models and assets for tok (latest)
2 days ago
tok_tokenizer_64k.model
1.31 MB
xet
Upload all models and assets for tok (latest)
2 days ago
tok_tokenizer_64k.vocab
983 kB
Upload all models and assets for tok (latest)
2 days ago
tok_tokenizer_8k.model
363 kB
xet
Upload all models and assets for tok (latest)
2 days ago
tok_tokenizer_8k.vocab
98.2 kB
Upload all models and assets for tok (latest)
2 days ago
