gemma-2-tokenizer / README.md
TheDrummer's picture
Create README.md
157c743 verified
Tokenizer files for this script:
https://huggingface.co/docs/transformers.js/en/api/tokenizers#pretrainedtokenizerapplychattemplateconversation-options--code-string-code--code-tensor-code--code-array--number--code--code-array--array--number---code