Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets new
  • Docs
  • Enterprise
  • Pricing
    • Website
      • Tasks
      • HuggingChat
      • Collections
      • Languages
      • Organizations
    • Community
      • Blog
      • Posts
      • Daily Papers
      • Hardware
      • Learn
      • Discord
      • Forum
      • GitHub
    • Solutions
      • Team & Enterprise
      • Hugging Face PRO
      • Enterprise Support
      • Inference Providers
      • Inference Endpoints
      • Storage Buckets

  • Log In
  • Sign Up

janakhpon
/
mon_tokenizer

Mon
Burmese
English
tokenizers
tokenizer
unigram
mon
burmese
myanmar
low-resource
Model card Files Files and versions
xet
Community
mon_tokenizer
4.87 MB
Ctrl+K
Ctrl+K
  • 1 contributor
History: 16 commits
janakhpon's picture
janakhpon
chore: keep internal engineering docs off the Hub
8fd15e7 10 days ago
  • .gitattributes
    315 Bytes
    feat: restructure and upgrade to 32k vocab model (v2) 4 months ago
  • .gitignore
    784 Bytes
    chore: keep internal engineering docs off the Hub 10 days ago
  • README.md
    5.76 kB
    feat: migrate the artifact from SentencePiece to tokenizers JSON 11 days ago
  • model_card.json
    2.65 kB
    feat: migrate the artifact from SentencePiece to tokenizers JSON 11 days ago
  • special_tokens_map.json
    96 Bytes
    feat: migrate the artifact from SentencePiece to tokenizers JSON 11 days ago
  • tokenizer.json
    4.86 MB
    feat: migrate the artifact from SentencePiece to tokenizers JSON 11 days ago
  • tokenizer_config.json
    240 Bytes
    feat: migrate the artifact from SentencePiece to tokenizers JSON 11 days ago