Transformers

Commit History

  1. FIX: Complete compilation of 32k vocabulary into tokenizer.json

    Verified signature
    c5ecc2b

  1. FIX: Complete compilation of 32k vocabulary into tokenizer.json

    Verified signature
    f483284
  2. Initial deploy of optimized 32k code/math SentencePiece tokenizer architecture

    Verified signature
    11f7ba1
  3. initial commit

    Verified signature
    54ce7d5