FORGE-500M-Chat / tokenizer

Commit History

5 days ago

  1. Add tokenizer component: streaming_tokenizer.py

    Verified signature
    6b836f4
  2. Add tokenizer component: crayon_wrapper.py

    Verified signature
    6684ccb
  3. Add tokenizer component: data_collator.py

    Verified signature
    5288f34
  4. Add tokenizer component: __init__.py

    Verified signature
    2826092
  5. Release FORGE-500M-Chat: Complete Base + SFT + DPO aligned model with full training journey log

    Verified signature
    9527e5e