johnbean393
/

chiboard-1.1-tokenizer

Model card Files Files and versions

chiboard-1.1-tokenizer / tokenizer.json

Commit History

Oct 4 14 days ago

00:28
Use the Qwen3.5 base pre-tokenizer and decoder (regex [\p{L}\p{M}]+); tokenizer_class TokenizersBackend so transformers loads tokenizer.json verbatim; supersedes 177cdc85 (T1's Qwen2 pre-tokenizer)

johnbean393

Verified signature
dc692bd

Oct 3 15 days ago

23:31
Chiboard 1.1 tokenizer v3: T1 + <|chiboard_screen_ref|>=248081, <|chiboard_screen|>=248082

johnbean393

Verified signature
177cdc8