TFM-tokenizer / tokenizer.json

AUGUST-GKD

Add special token for table tokenization

fa6d40e 25 days ago

17.2 MB

Invalid JSON: Unexpected token 'v', "version ht"... is not valid JSON

This file is stored with Xet . It is too big to display, but you can still download it.

( Raw pointer file )

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.