Commit History
13 days ago
-
tokenizer_config: return_tensors='pt' call default (TRF019)
Verified signature5eba794 -
Plain TokenizersBackend: the capitalization as normalizer steps
Verified signatureThe normalizer of `tokenizer.json` now also upper-cases the first character (one step per lowercase letter of the vocabulary), so the tokenizer loads as a plain `TokenizersBackend`; `tokenizer_config.json` drops the `PocketTTSTokenizer` settings. Token ids of prepared prompts are unchanged.8eec842 -
Text preparation as tokenizers normalizers in tokenizer.json
Verified signatureThe normalizer of `tokenizer.json` holds the original's `prepare_text_prompt` steps but the capitalization (done by `PocketTTSTokenizer`). `capitalize_first_letter` and `append_terminal_punctuation` are no longer settings (on in every released config). Token ids are unchanged.c14ddeb -
Text preparation in PocketTTSTokenizer: tokenizer_class + language settings, layout-only chat template
Verified signature2db240a
19 days ago
-
Convert from kyutai/pocket-tts
Verified signaturebdd62cb