Commit History

  1. Upload tokenizer.json

    Verified signature
    70ef62f

  1. Refactor README.md to remove licensing section and unsupported tasks, streamlining content for clarity

    2e71ee6
  2. Update README.md to change tokenizer model reference and remove licensing, citation, and contribution sections

    72772cb
  3. Add README.md for TR Tokenizer and include usage examples

    833de38
  4. Upload tokenizer_config.json

    Verified signature
    1a1d524
  5. Upload test_bpe_tokenizer.json

    Verified signature
    9d5be2e
  6. initial commit

    Verified signature
    e8dea55