Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

Morton-Li
/
QiTianTokenizer-Medium

tokenizer
bilingual
chinese
english
multilingual
Model card Files Files and versions
xet
Community
QiTianTokenizer-Medium
5.06 MB
  • 1 contributor
History: 5 commits
Morton-Li's picture
Morton-Li
Version iteration
ffd1e26 24 days ago
  • .gitattributes
    1.52 kB
    initial commit 4 months ago
  • README.md
    7.86 kB
    Version iteration 24 days ago
  • chat_template.jinja
    306 Bytes
    Version iteration 24 days ago
  • tokenizer.json
    5.05 MB
    Version iteration 24 days ago
  • tokenizer.py
    202 Bytes
    添加 QiTianTokenizerFast 类及其配置,移除 QiTianTokenizer 类的引用并完善 QiTianTokenizerFast 的配置 3 months ago
  • tokenizer_config.json
    637 Bytes
    Version iteration 24 days ago