Zero-Shot Image Classification
Transformers
Safetensors
siglip
vision

"tokenizer_class" in tokenizer_config.json should be "Siglip2Tokenizer" instead of "GemmaTokenizer" to get correct lowercasing behaviour

#22
by bellinghesam - opened

https://huggingface.co/google/siglip2-base-patch16-224/blob/main/tokenizer_config.json#L2017

All SigLIP2 checkpoints currently ship with "tokenizer_class": "GemmaTokenizer" in tokenizer_config.json. This causes AutoTokenizer.from_pretrained(...) and AutoProcessor.from_pretrained(...) to load a plain GemmaTokenizer, which does not lowercase input text.
This is inconsistent with how the model was trained. From the SigLIP2 paper:

"We set the text length to 64 and use the multilingual Gemma tokenizer [22] with vocabulary size 256k, transforming the text to lower case before tokenization."

Since the default loaded tokenizer doesn't lowercase, users get degraded/incorrect results out of the box unless they manually lowercase every input themselves β€” this exact symptom is reported in discussion #7.
This has already been fixed on the transformers side. transformers now ships a dedicated Siglip2Tokenizer class that applies lowercasing at the tokenizer level, and tokenization_auto.py maps "siglip2" model types to it. However, since each checkpoint's tokenizer_config.json explicitly hardcodes "tokenizer_class": "GemmaTokenizer", this explicit value takes precedence over the library's auto-resolution, so the fix never actually takes effect for users loading these checkpoints.
For comparison, the original SigLIP1 checkpoints correctly reference their model-specific tokenizer β€” e.g. google/siglip-so400m-patch14-384's tokenizer_config.json sets "tokenizer_class": "SiglipTokenizer".
Requested fix: update tokenizer_class in tokenizer_config.json from "GemmaTokenizer" to "Siglip2Tokenizer" across all google/siglip2-* checkpoints, so that AutoTokenizer/AutoProcessor correctly apply lowercasing by default, matching both the paper's training procedure and the now-fixed transformers library behavior.

bellinghesam changed discussion title from "tokenizer_class" in tokenizer_config.json should be "Siglip2Tokenizer" instead of "GemmaTokenizerFast" to get correct lowercasing behaviour to "tokenizer_class" in tokenizer_config.json should be "Siglip2Tokenizer" instead of "GemmaTokenizer" to get correct lowercasing behaviour

Sign up or log in to comment