almaghrabima
/

SARF-Tokenizer

Model card Files Files and versions

SARF-Tokenizer / tokenizer_benchmark.py

Commit History

Update: rank by parity+efficiency, add Falcon-H1-7B

5362025

almaghrabima commited on 3 days ago

Add benchmark results: 13 tokenizer comparison

301b160

almaghrabima commited on 3 days ago