Commit History

add token_bytes.pt at root
1fbde5e
verified

procmarco commited on

add tokenizer.pkl at root
08e1e2d
verified

procmarco commited on

48k IT/EN/code BPE (40/40/20), frontier-aligned special tokens
b4e272e
verified

procmarco commited on

initial commit
9929537
verified

procmarco commited on