Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

BeaverAI
/
gemma-2-tokenizer

Model card Files Files and versions
xet
Community
gemma-2-tokenizer / README.md
TheDrummer's picture
TheDrummer
Create README.md
157c743 verified over 1 year ago
preview code
|
raw
history blame contribute delete
248 Bytes

Tokenizer files for this script:

https://huggingface.co/docs/transformers.js/en/api/tokenizers#pretrainedtokenizerapplychattemplateconversation-options--code-string-code--code-tensor-code--code-array--number--code--code-array--array--number---code