guarin's picture
guarin HF Staff
Use tiny Qwen image and video sizes for fast processor tests
e2f2af6 verified
|
Raw History Blame Contribute Delete
1.18 kB
metadata
library_name: transformers
tags:
  - testing
  - image-text-to-text

Tiny Xiaomi-Lens-0 processor

Processor-only test fixture using the tiny BPE tokenizer from hf-internal-testing/tiny-processor-qwen3_vl and the processor configuration, tokenizer settings, and chat template from guarin/Xiaomi-Lens-0.

Xiaomi control tokens are added to the tiny vocabulary. Token IDs are local to this fixture and are not compatible with the full model weights. Image and video size limits (784–3136 pixels) and patch size (4) use the tiny Qwen fixture settings for fast unit tests. The image min_pixels/max_pixels aliases match those limits. Other processor settings, the chat template, and tokenizer settings come from Xiaomi-Lens-0. No model weights are included. Requires a Transformers version supporting XiaomiLens0Processor.

from transformers import AutoProcessor
processor = AutoProcessor.from_pretrained("hf-internal-testing/tiny-processor-xiaomi_lens0")