tokenization-trial / README.md
ClaudeHu05's picture
Create README.md
9ef065d verified

This model repository will only contain files needed for tokenization (vocab.json, tokenizer_config.sjon, etc). The goal is to get familiar with huggingface tokenizers and test tokenizers for tabs.