[BUG] gemma-4 zero3 from_pretrained

ISSUE #45397: [BUG] gemma-4 zero3 from_pretrained
State: closed  |  Labels: bug
Author: Jintao-Huang  |  Created: 2026-04-13T09:25:04Z

### System Info

https://github.com/modelscope/ms-swift/issues/9078

google/gemma-4-E4B-it

zero2 works fine, zero3 does not.

zero2:

zero3:

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

-

--- Comment by saslifat-gif at 2026-04-13T11:20:04Z ---
Hi, I'm investigating this issue.

Could you share:
1. Your DeepSpeed ZeRO-3 config file
2. The full training script
3. Transformers version (`pip show transformers`)

I noticed the MISSING keys are all in vision_tower and audio_tower.
Looking into whether this is related to how Gemma4Model initializes
these submodels under ZeRO-3's parameter gathering context.

--- Comment by Jintao-Huang at 2026-04-13T13:04:17Z ---
https://github.com/modelscope/ms-swift/blob/main/swift/config/zero3.json

transformers 5.5.3

--- Comment by Jintao-Huang at 2026-04-13T13:08:50Z ---
I think it has nothing to do with the trainer and should be easy to reproduce — just load the weights (with zero3).

google/gemma-4-26B-A4B-it

--- Comment by saslifat-gif at 2026-04-13T15:14:38Z ---
Hi @Jintao-Huang,
Sorry for the late reply! I was actually diving deep into the code to reproduce and fix this issue right when you commented.
You were spot on—it has nothing to do with the trainer. I found that the root cause is in the ZeRO-3 loading logic where named_buffers were being skipped.
I've just opened a PR (#45402) with a fix and verified it works. Let's wait for the maintainers to review it. Thanks for the confirmation!


ISSUE #45390: CLIPTextModel / CLIPVisionModel fail to load old checkpoints after architecture flattening
State: closed  |  Labels:
Author: sayakpaul  |  Created: 2026-04-13T04:50:21Z

## Description

After the recent refactoring that flattened `CLIPTextModel` (removed the `self.text_model` wrapper) and `CLIPVisionModel` (removed the `self.vision_model` wrapper), old checkpoints that were saved with the nested structure can no longer be loaded correctly.

All weights end up randomly initialized because the checkpoint keys (e.g. `text_model.embeddings.token_embedding.weight`) don't match the new model's state dict keys (e.g. `embeddings.token_embedding.weight`).

## Minimal reproducer

```python
import torch
from transformers import CLIPTextModel, CLIPTextConfig

# Any old-format CLIP checkpoint works; this one ships with diffusers tests
model_path = "hf-internal-testing/tiny-stable-diffusion-torch"

# Download so it's cached
from huggingface_hub import hf_hub_download
ckpt_dir = hf_hub_download(model_path, "text_encoder/pytorch_model.bin")

# Show the checkpoint has text_model.* keys
sd = torch.load(ckpt_dir, map_location="cpu", weights_only=True)
print("Checkpoint key example:", list(sd.keys())[1])
# → text_model.embeddings.token_embedding.weight

expected_sum = sd["text_model.embeddings.token_embedding.weight"].sum().item()
print(f"Expected token_embedding sum: {expected_sum:.4f}")

# Load via from_pretrained
te = CLIPTextModel.from_pretrained(
    model_path, subfolder="text_encoder"
)
actual_sum = te.state_dict()["embeddings.token_embedding.weight"].sum().item()
print(f"Actual token_embedding sum:   {actual_sum:.4f}")

assert abs(expected_sum - actual_sum) < 1e-5, (
    f"Weights were NOT loaded! expected={expected_sum:.4f}, got={actual_sum:.4f}"
)
```

**Output (failing):**
```
Checkpoint key example: text_model.embeddings.token_embedding.weight
Expected token_embedding sum: -4.9096
Actual token_embedding sum:   -0.0497    # ← random init, not checkpoint value

AssertionError: Weights were NOT loaded! expected=-4.9096, got=-0.0497
```

## Impact

This breaks any downstream code that loads `CLIPTextModel` or `CLIPVisionModel` from checkpoints saved with previous transformers versions — including all Stable Diffusion pipelines in diffusers.

--- Comment by Abineshabee at 2026-04-13T08:59:27Z ---
Hi @sayakpaul! I'd like to work on this issue and submit a fix.

I was able to successfully reproduce the problem on my end using the provided example. The checkpoint contains keys with the `text_model.*` prefix (e.g., `text_model.embeddings.token_embedding.weight`), while the current model expects flattened keys (e.g., `embeddings.token_embedding.weight`). As a result, all weights are marked as **UNEXPECTED/MISSING**, and the model falls back to random initialization.

To fix this, I’m planning to implement backward compatibility by overriding `_load_from_state_dict` in both `CLIPTextModel` and `CLIPVisionModel`. The idea is to remap old checkpoint keys by stripping the `text_model.` / `vision_model.` prefixes before delegating to the parent loader. This should allow older checkpoints to load correctly without affecting the current architecture.

I’ll also verify the fix using the provided reproducible script and run the existing CLIP test suite to ensure nothing else breaks.

Could you please assign this issue to me, or let me know if someone is already working on it?

Thanks!

--- Comment by github-actions[bot] at 2026-05-13T08:56:16Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #45381: transformers==5.3.0, qwen2.5-vl video input vision_position_ids seems to be wrong
State: closed  |  Labels: bug
Author: bicheng-xu  |  Created: 2026-04-11T22:43:30Z

### System Info

transformers == 5.3.0
[But the bug seems to be with any transformers >= 5.3.0, even in the current main branch]
qwen_vl_utils == 0.0.14
Python 3.12.4
Cuda 12.6

### Who can help?

@zucchini-nlp

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

I am using this script

```
from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info

# default: Load the model on the available device(s)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-3B-Instruct", torch_dtype="auto", device_map="auto"
)
# default processer
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")
# Messages containing a local video path and a text query
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": "file:///path/to/video1.mp4",
                "max_pixels": 360 * 420,
                "fps": 1.0,
            },
            {"type": "text", "text": "Describe this video."},
        ],
    }
]
#In Qwen 2.5 VL, frame rate information is also input into the model to align with absolute time.
# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs, video_kwargs = process_vision_info(messages, return_video_kwargs=True, return_video_metadata=True)

inputs = processor(
    text=[text],
    images=image_inputs,
    videos=[[video_inputs[0][0]]],
    video_metadata=[[video_inputs[0][1]]],
    padding=True,
    return_tensors="pt",
    **video_kwargs,
)
inputs = inputs.to("cuda")

# Inference
generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)
```

### Expected behavior

The vision_position_ids output at https://github.com/huggingface/transformers/blob/v5.3.0/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1177 is wrong. All video frames share the same position_temporal and the position_height is also wrong

--- Comment by bicheng-xu at 2026-04-11T23:42:42Z ---
I am currently using the get_rope_index() method from transformers v5.2.0: https://github.com/huggingface/transformers/blob/v5.2.0/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1019

--- Comment by bicheng-xu at 2026-04-11T23:47:25Z ---
Sorry, the issue was closed by mistake, it should be open...

--- Comment by bicheng-xu at 2026-04-11T23:52:36Z ---
Here is one video which can be used in the above test script.

https://github.com/user-attachments/assets/722bb355-abd1-4b6e-b594-3ee8ab600fd4

The video is from the ActivityNet dataset ( http://activity-net.org/ ), video_id: v_2PCZkpF1_wU

--- Comment by bicheng-xu at 2026-04-11T23:57:12Z ---
@zucchini-nlp

--- Comment by zucchini-nlp at 2026-04-13T09:36:18Z ---
Can you check with https://github.com/huggingface/transformers/pull/45330? I believe it should be fixed by now. Ah no, wait, video! Will check, very strange

--- Comment by bicheng-xu at 2026-04-13T21:39:53Z ---
@zucchini-nlp  No, this PR does not solve the video's position_ids issue.
The following is a detailed comparison with the get_rope_index() in the current main branch (code: commit tag a9f5b3a of the modeling_qwen2_5_vl.py file https://github.com/huggingface/transformers/blob/main/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1024 ) and the get_rope_index()  in the v5.2.0 (code: https://github.com/huggingface/transformers/blob/v5.2.0/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1019 )

The test script I'm using: The "v_2PCZkpF1_wU.mp4" video is this video: https://github.com/huggingface/transformers/issues/45381#issuecomment-4230392900
```
from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info

# default: Load the model on the available device(s)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-3B-Instruct", torch_dtype="auto", device_map="auto"
)
# default processer
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")
# Messages containing a local video path and a text query
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": "v_2PCZkpF1_wU.mp4",
                "max_pixels": 360 * 420,
                "fps": 1.0,
            },
            {"type": "text", "text": "Describe this video."},
        ],
    }
]
#In Qwen 2.5 VL, frame rate information is also input into the model to align with absolute time.
# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs, video_kwargs = process_vision_info(messages, return_video_kwargs=True, return_video_metadata=True)

inputs = processor(
    text=[text],
    images=image_inputs,
    videos=[[video_inputs[0][0]]],
    video_metadata=[[video_inputs[0][1]]],
    padding=True,
    return_tensors="pt",
    **video_kwargs,
)
inputs = inputs.to("cuda")

# Inference
generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)
```

RESULTS:
If I use the get_rope_index()  in the v5.2.0 (code: https://github.com/huggingface/transformers/blob/v5.2.0/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1019 ), the output llm_pos_ids_list[1] (the video's position_ids, at https://github.com/huggingface/transformers/blob/v5.2.0/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1161 ) should be correct.
(Pdb) llm_pos_ids_list
[tensor([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14],
        [ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14],
        [ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14]]), tensor([[ 15,  15,  15,  ..., 347, 347, 347],
        [ 15,  15,  15,  ...,  26,  26,  26],
        [ 15,  16,  17,  ...,  28,  29,  30]]), tensor([[348, 349, 350, 351, 352, 353, 354, 355, 356, 357],
        [348, 349, 350, 351, 352, 353, 354, 355, 356, 357],
        [348, 349, 350, 351, 352, 353, 354, 355, 356, 357]])]
(Pdb) llm_pos_ids_list[1].shape
torch.Size([3, 16128])
(Pdb) llm_pos_ids_list[1][0, :300]
tensor([15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 19, 19, 19, 19, 19, 19,
        19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
        19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
        19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
        19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
        19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
        19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19])
(Pdb) llm_pos_ids_list[1][1, :300]
tensor([15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16,
        16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17,
        17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18,
        18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19,
        19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20,
        20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
        21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22,
        22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23,
        24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25,
        25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26,
        26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16,
        16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17,
        17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18,
        18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
        19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20,
        21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21])
(Pdb) llm_pos_ids_list[1][2, :300]
tensor([15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16,
        17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18,
        19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20,
        21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22,
        23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
        25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
        27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
        29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
        15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16,
        17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18,
        19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20,
        21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22,
        23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
        25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
        27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
        29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
        15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26])

BUT if I use the get_rope_index() in the current main branch (code: commit tag a9f5b3a of the modeling_qwen2_5_vl.py file https://github.com/huggingface/transformers/blob/main/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1024 ), the output llm_pos_ids_list[1] (the video's position_ids, at https://github.com/huggingface/transformers/blob/main/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1131 ) is wrong.
(Pdb) llm_pos_ids_list
[tensor([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14],
        [ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14],
        [ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14]],
       device='cuda:0'), tensor([[60, 60, 60,  ..., 60, 60, 60],
        [15, 15, 15,  ..., 26, 26, 26],
        [15, 16, 17,  ..., 28, 29, 30]], device='cuda:0'), tensor([[31, 32, 33, 34, 35, 36, 37, 38, 39, 40],
        [31, 32, 33, 34, 35, 36, 37, 38, 39, 40],
        [31, 32, 33, 34, 35, 36, 37, 38, 39, 40]], device='cuda:0')]
(Pdb) llm_pos_ids_list[1].shape
torch.Size([3, 16128])
(Pdb) llm_pos_ids_list[1][0, :300]
tensor([60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60,
        60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60], device='cuda:0')
(Pdb) llm_pos_ids_list[1][1, :300]
tensor([15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
        15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15], device='cuda:0')
(Pdb) llm_pos_ids_list[1][2, :300]
tensor([15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16,
        17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18,
        19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20,
        21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22,
        23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
        25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
        27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
        29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
        15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16,
        17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18,
        19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20,
        21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22,
        23, 24, 25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
        25, 26, 27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
        27, 28, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
        29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
        15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26], device='cuda:0')
All video frames share the same position_temporal and the position_height is also wrong. Though the position_width is correct...

--- Comment by zucchini-nlp at 2026-04-14T08:59:35Z ---
You're right, that one is for image. I opened a PR for fixing the video inference in https://github.com/huggingface/transformers/pull/45400, will try to merge today


ISSUE #45376: [Bug] Loading gemma4 on transformers v4 raises misleading AttributeError: 'list' object has no attribute 'keys' instead of a clear version requirement error
State: closed  |  Labels: bug
Author: HARISH-CS-01  |  Created: 2026-04-11T12:57:06Z

## System info

```
- transformers version: 4.x (any release before v5.5.0)
- Platform: macOS (Apple Silicon) / Linux
- Python version: 3.12 / 3.13
- Model: google/gemma-4-E4B-it (requires transformers >= 5.5.0)
```

## Background

`google/gemma-4-E4B-it` uses the `gemma4` architecture, which was introduced in transformers **v5.5.0**. It cannot and should not load on v4. However, the error a user actually sees on v4 is deeply misleading and sends them down the wrong debugging path entirely.

## Steps to reproduce

```bash
pip install transformers        # installs v4.x by default
```

```python
from transformers import AutoTokenizer
AutoTokenizer.from_pretrained("google/gemma-4-E4B-it")
```

## Expected error (clear and actionable)

```
ValueError: google/gemma-4-E4B-it requires transformers >= 5.5.0.
Please upgrade: pip install "transformers>=5.5.0"
```

## Actual error (cryptic and misleading)

```
File .../transformers/tokenization_utils_base.py:1181
    self.SPECIAL_TOKENS_ATTRIBUTES = self.SPECIAL_TOKENS_ATTRIBUTES + list(special_tokens.keys())
AttributeError: 'list' object has no attribute 'keys'
```

## Why this is a problem

The actual error gives users no indication that the real problem is a version mismatch. Instead it points to internal tokenizer code, causing users to:

- Spend time patching `tokenizer_config.json` files manually
- Try reinstalling `tokenizers` and `transformers` in different combinations
- Assume it is a bug in the model's config rather than a version requirement
- File issues against the wrong repo (model page, mlx-lm, etc.)

This is a developer experience issue — the underlying incompatibility is expected, but the error message is not.

## Root cause

The `tokenizer_config.json` for gemma4 uses the v5-style `extra_special_tokens` list format:

```json
"extra_special_tokens": ["<token_a>", "<token_b>"]
```

When loaded by transformers v4, `_set_model_specific_special_tokens()` calls `.keys()` on this list and crashes — before the code ever gets to check whether the `gemma4` architecture is supported. The version check happens too late.

## Suggested fix

Two possible approaches:

**Option A** — Add a guard in `_set_model_specific_special_tokens()` so the list format fails gracefully with a helpful message:

```python
def _set_model_specific_special_tokens(self, special_tokens):
    if isinstance(special_tokens, list):
        raise ValueError(
            "This model's tokenizer config uses a format introduced in transformers v5. "
            "Please upgrade: pip install 'transformers>=5.0.0'"
        )
    ...
```

**Option B** — Move the architecture/version check earlier in the loading pipeline, before tokenizer config parsing, so the user sees a version error first.

--- Comment by hijingsong at 2026-04-11T16:23:48Z ---
Looking into this — `_set_model_specific_special_tokens` assumes dict but v5-format tokenizers can pass a list. Adding an `isinstance` check to handle both formats. Happy to submit a fix.

--- Comment by abderbejaoui at 2026-04-12T14:09:41Z ---
Hi 👋 I’m interested in working on this issue.
Could you clarify what changes are expected regarding the new agent version?
Is there an existing implementation or PR I can reference?

--- Comment by zucchini-nlp at 2026-04-13T12:25:21Z ---
How can Gemma4 be loaded in v4 if it was shipped in v5.5.0?

--- Comment by HARISH-CS-01 at 2026-04-14T05:49:57Z ---
Good catch — you're right that Gemma 4 requires transformers v5.5+. The real issue is that the error message on v4 is misleading. Instead of a clear "this model requires transformers v5.5+", users get a cryptic AttributeError about .keys(), which sends them down the wrong debugging path. The fix should be a better error message in v4, not a compatibility patch. I'll update the issue title and description accordingly.

--- Comment by github-actions[bot] at 2026-05-12T08:54:18Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #45375: Qwen3_5MoeVisionConfig missing deepstack_visual_indexes field — silently dropped by @strict
State: closed  |  Labels: bug
Author: sharonyu-115  |  Created: 2026-04-11T12:43:25Z

### System Info

Transformers v5.5.0

The @strict decorator on Qwen3_5MoeVisionConfig (added in #41250) silently drops the deepstack_visual_indexes field during config loading, because it's not declared as a class attribute. However, every Qwen3.5 MoE model on HuggingFace ships
  with this field in its config.json.

  For example, Qwen/Qwen3.5-35B-A3B-Base/config.json contains:

```
  {
    "vision_config": {
      "deepstack_visual_indexes": [],
      "depth": 27,
      "hidden_size": 1152,
      ...
    }
  }
```

  After loading with transformers 5.5.0+, `config.vision_config.deepstack_visual_indexes` raises `AttributeError` because `@strict` rejected the undeclared field.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

**Steps/Code to reproduce bug**

```
  from transformers import AutoConfig

  config = AutoConfig.from_pretrained("Qwen/Qwen3.5-35B-A3B-Base", trust_remote_code=True)
  print(config.vision_config.deepstack_visual_indexes)
  # AttributeError: 'Qwen3_5MoeVisionConfig' object has no attribute 'deepstack_visual_indexes'
```

**Suggested fix**

Add the missing field to Qwen3_5MoeVisionConfig in src/transformers/models/qwen3_5_moe/configuration_qwen3_5_moe.py:

```
  @strict
  class Qwen3_5MoeVisionConfig(PreTrainedConfig):
      ...
      num_position_embeddings: int = 2304
      initializer_range: float = 0.02
      deepstack_visual_indexes: list[int] = field(default_factory=list)  # <-- add this
```

### Expected behavior

**Expected behavior**

  `config.vision_config.deepstack_visual_indexes` should return the value from the model's config.json

--- Comment by zucchini-nlp at 2026-04-13T12:22:30Z ---
Strict never rejects undeclared fields, it just wouldn't type-check them against annotations. I tried to run the code you provided in `main` and couldn't reproduce the same error

Could you share you hf-hub package version, and try to install from `main`? Though I doubt that main has any diff from 5.5.0 🤔

--- Comment by github-actions[bot] at 2026-05-12T08:54:20Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #45373: Add Gemma4ForSequenceClassification (missing from gemma4 module — Gemma 2/3 have it)
State: open  |  Labels: Feature request
Author: LarsKlawitter  |  Created: 2026-04-11T06:41:23Z

### Feature request

<p style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, "system-ui", "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Please add Gemma4ForSequenceClassification (and ideally Gemma4TextForSequenceClassification) to transformers.models.gemma4. The current __all__ in modeling_gemma4.py contains only:
<div class="codeBlockWrapper_-a7MRw" style="position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, "system-ui", "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="color: rgb(204, 204, 204); font-family: -apple-system, "system-ui", "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 1px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button>
__all__ = [
    "Gemma4AudioModel",
    "Gemma4ForCausalLM",
    "Gemma4ForConditionalGeneration",
    "Gemma4Model",
    "Gemma4PreTrainedModel",
    "Gemma4TextModel",
    "Gemma4VisionModel",
]

<p style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, "system-ui", "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">No classification head variants are exported, which is inconsistent with every prior Gemma release:

Model family | ForSequenceClassification | TextForSequenceClassification
-- | -- | --
gemma | ✅ | N/A (text-only)
gemma2 | ✅ | N/A (text-only)
gemma3 | ✅ | ✅
gemma3n | ✅ | ✅
gemma4 | ❌ | ❌

<p style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, "system-ui", "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">As a result, AutoModelForSequenceClassification.from_pretrained("google/gemma-4-E4B", num_labels=3) raises:
<div class="codeBlockWrapper_-a7MRw" style="position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, "system-ui", "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="color: rgb(204, 204, 204); font-family: -apple-system, "system-ui", "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 1px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 1; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button>
ValueError: Unrecognized configuration class <class 'transformers.models.gemma4.configuration_gemma4.Gemma4Config'>
for this kind of AutoModel: AutoModelForSequenceClassification.

### Motivation

AutoModelForSequenceClassification is the standard entry point for fine-tuning causal LMs as classifiers — a common use case across research (domain classification, sentiment, reward modelling) and production (content moderation, intent detection, financial signal prediction). Excluding it from gemma4 forces users to either:

Roll a custom classifier head wrapper (breaks the AutoModel contract and duplicates logic that already exists in GenericForSequenceClassification),
Fine-tune generatively and parse argmax over label tokens (worse calibration, loses softmax-head properties), or
Skip Gemma 4 entirely and stay on Gemma 3 or a different family.
We are currently working around this in a financial-NLP fine-tuning pipeline by subclassing GenericForSequenceClassification and registering the result via AutoModelForSequenceClassification.register(). It works but is exactly the kind of boilerplate the AutoModel registry is supposed to eliminate, and it has to be duplicated by every project that wants to fine-tune Gemma 4 as a classifier.

Gemma 4 E4B is a Mixture-of-Experts model (Gemma4TextExperts + Gemma4TextRouter confirmed in modeling_gemma4.py), which makes it particularly inconvenient for users to hand-roll a classification head: the correct pooling behaviour and last_non_pad_token handling that GenericForSequenceClassification provides for free becomes something every user has to re-implement while also navigating MoE-specific LoRA pitfalls. Adding Gemma4(Text)ForSequenceClassification upstream would spare every downstream user from reinventing the same wheel.

### Your contribution

The implementation should be trivial — Gemma 3 provides the exact template. In transformers/models/gemma3/modeling_gemma3.py:

class Gemma3TextForSequenceClassification(GenericForSequenceClassification, Gemma3PreTrainedModel):
    config: Gemma3TextConfig
    input_modalities = ("text",)
And its multimodal sibling Gemma3ForSequenceClassification, which wires up a Gemma3Model + a nn.Linear(text_config.hidden_size, num_labels) head. The same pattern applied to Gemma 4 should require only a few lines of code plus:

Adding the class(es) to __all__ in modeling_gemma4.py
Registering them in auto/modeling_auto.py against Gemma4Config (and Gemma4TextConfig if it exists)
I'm happy to open a PR if that would help. The change is mechanical but I'd like to confirm first which of the two variants (multimodal-path + text-only, or just text-only) the maintainers prefer — Gemma 3 shipped both, so matching that seems sensible.

--- Comment by Grizzzlyy at 2026-04-13T12:53:20Z ---
Seems related to #45295

--- Comment by Charly21r at 2026-04-14T16:22:20Z ---
Hi! I’d like to take this and implement it. I’ll open a PR soon.

--- Comment by Charly21r at 2026-04-14T16:31:29Z ---
I’m planning to start with `Gemma4TextForSequenceClassification` first (following the Gemma 3 pattern) and open a PR for that initial support. We can extend to multimodal afterwards. If you’re also planning to work on this, happy to coordinate to avoid duplication.

--- Comment by LarsKlawitter at 2026-04-14T19:32:35Z ---
Happy to test this against a real downstream use case: QLoRA fine-tuning Gemma 4 E4B on a 3-class sequence classification head. I've got a working local wrapper following the Gemma 3 pattern that I can diff against your PR, and I can report back on whether it survives training + eval end-to-end.

--- Comment by Charly21r at 2026-04-14T19:57:00Z ---
That would be super helpful, thanks! Here's the PR: #45438 . Would love to hear how it goes with your QLoRA pipeline.

--- Comment by Rocketknight1 at 2026-04-15T10:45:54Z ---
We now have two PRs here, #45294 and #45438. They're both fairly similar, but reviewers will have to pick one!


ISSUE #45372: ImportError: cannot import name 'ReasoningEffort' from mistral_common breaks Gemma 4 processor loading in transformers 5.5.x / 5.6.0.dev0
State: open  |  Labels: bug
Author: Glademist  |  Created: 2026-04-11T06:34:13Z

### System Info

Environment
- transformers: 5.5.1 and 5.6.0.dev0 (git main as of ~2026-04-08)
- mistral-common: 1.9.1
- Python: 3.11.15
- Platform: macOS aarch64 (Apple Silicon)
- mlx-vlm: 0.4.x

Bug description
When loading any model via AutoProcessor (tested with google/gemma-4-31b-it and mlx-community/Qwen3.5-2B-4bit), transformers unconditionally imports ReasoningEffort from mistral_common.protocol.instruct.request inside tokenization_mistral_common.py, even when the target model has no relation to Mistral.

The import succeeds at the top of the file (line ~42) only if mistral-common exposes ReasoningEffort. In version 1.9.1 it does not, causing an ImportError. The import is guarded by is_mistral_common_available(), but the symbol ReasoningEffort itself is missing from mistral_common 1.9.1, making the guard useless.

A secondary failure occurs even when patching line 42 to catch the ImportError and stub ReasoningEffort = None: line 1042 uses ReasoningEffort in a type annotation (ReasoningEffort | None = None), which raises:

    TypeError: unsupported operand type(s) for |: 'NoneType' and 'NoneType'

The annotation is inside the MistralCommonBackend class body, which is evaluated at class definition time, not lazily.

Workaround
Patch line 42 of tokenization_mistral_common.py to stub the missing symbol as a valid type:

    try:
        from mistral_common.protocol.instruct.request import (
            ChatCompletionRequest, ReasoningEffort
        )
    except ImportError:
        from mistral_common.protocol.instruct.request import ChatCompletionRequest
        from typing import Any as ReasoningEffort

This allows the class body annotation to resolve without error and has no effect on Gemma 4 inference, which never exercises the Mistral reasoning code path.

Expected behavior
Loading a non-Mistral model (Gemma 4, Qwen3) should not fail due to a missing symbol in mistral_common. The import guard should either be tightened to only run inside Mistral-specific processor code, or ReasoningEffort should be conditionally imported with a safe fallback type.

Traceback (truncated)
  File ".../transformers/tokenization_mistral_common.py", line 42, in <module>
    from mistral_common.protocol.instruct.request import ChatCompletionRequest, ReasoningEffort
ImportError: cannot import name 'ReasoningEffort' from 'mistral_common.protocol.instruct.request'

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

fresh sourced enviroment
pip install mlx-lm mlx-vlm
mlx_vlm.generate --model mlx-community/gemma-4-31b-it-4bit --kv-bits 8 --kv-quant-scheme turboquant --enable-thinking --thinking-budget 800 --max-tokens 3000 --prompt "I have a dirty car. A car wash is 100 meters away. Do i drive there or walk if i want to wash my car?"

### Expected behavior

The model should run, however it crashes due to the ReasoningEffort failure

--- Comment by LfWhat at 2026-04-11T08:57:14Z ---
This is the problem of mistral-common, try pip install mistral-common==1.10.0

--- Comment by hijingsong at 2026-04-11T18:23:11Z ---
Looking into this — the unconditional import of ReasoningEffort from mistral_common breaks when users have an older version installed. Planning a try/except guard.

--- Comment by github-actions[bot] at 2026-05-11T09:02:03Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by Grople at 2026-05-15T10:31:54Z ---
我去掉model_type 成功启动了


ISSUE #45362: Qwen3.5-35B crashes with transformers chat
State: closed  |  Labels: bug
Author: Sector14  |  Created: 2026-04-10T15:39:40Z

### System Info

Using "transformers chat" with Qwen3.5-35B the moment a prompt is sent the server errors with an AttributeError

```
[snip]
File "/mnt/venvs/rocm6.4/lib64/python3.12/site-packages/transformers/cli/serving/chat_completion.py", line 174, in _streaming
    queue, streamer = gen_manager.generate_streaming(model, processor, inputs, gen_config, request_id=request_id)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/venvs/rocm6.4/lib64/python3.12/site-packages/transformers/cli/serving/utils.py", line 565, in generate_streaming
    streamer = DirectStreamer(processor._tokenizer, loop, queue, skip_special_tokens=True)
                              ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Qwen3VLProcessor' object has no attribute '_tokenizer'. Did you mean: 'tokenizer'?
```

I'm running transformers 5.5.0 from pip along with the rocm6.4 version of pytorch (2.8.0+rocm6.4.4.gitc1404424). If you want a full pip version list let me know.

 I've tried a couple of other random text models with transformers chat without issue, just Qwen so far that appears to be an issue.

Qwen's readme mentions transformers chat as a possible way to use it, so I'm assuming this used to work? Although they were using 4.57 dev version at the time the readme was written.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

transformers serve

transformers chat Qwen/Qwen3.5-35B-A3B

Once loaded, type any text at the prompt, press enter and server will error.

### Expected behavior

Model response streamed back.

--- Comment by sharziki at 2026-04-11T02:31:44Z ---
Hi! I can take this. The root cause is that `generate_streaming()` and `generate_cb_streaming()` in `serving/utils.py` access `processor._tokenizer` to get the Rust tokenizer backend, but this only works for `PreTrainedTokenizerFast` — `ProcessorMixin` subclasses like `Qwen3VLProcessor` expose the tokenizer at `.tokenizer` (public attribute) instead.

The fix is to resolve the fast tokenizer from the processor first, then access `._tokenizer`:
```python
rust_tok = getattr(processor, 'tokenizer', processor)._tokenizer
```

This handles both code paths (ProcessorMixin and PreTrainedTokenizerFast). Two locations to update: lines 565 and 664.

Happy to submit a PR. AI assistance (Claude Code) will be used — I'll review and validate every change.


ISSUE #45357: [Regression] Qwen3.5 `save_pretrained` still saves incorrect visual encoder keys in 5.5.3
State: closed  |  Labels: bug
Author: johnking0099  |  Created: 2026-04-10T10:01:58Z

### System Info

- `transformers` version: 5.5.0, 5.5.3
- Platform: Linux (NVIDIA A100 80GB × 8)
- Python version: 3.12
- PyTorch version: 2.9.1+cu128
- CUDA version: 12.8

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

## Description

`save_pretrained` on `Qwen3_5ForConditionalGeneration` produces incorrect weight keys for the visual encoder. This issue was partially addressed in v5.5.2 ([#45340](https://github.com/huggingface/transformers/pull/45340)), which fixed the text model key regression. However, the **visual encoder keys are still saved with a wrong prefix** in v5.5.3.

This makes it impossible to reload the saved checkpoint without manual key remapping.

## Reproduction

```python
import transformers
from safetensors.torch import load_file

model = transformers.Qwen3_5ForConditionalGeneration.from_pretrained("Qwen/Qwen3.5-0.8B")
model.save_pretrained("./qwen35-saved")

tensors = load_file("./qwen35-saved/model.safetensors")
keys = list(tensors.keys())

# Visual encoder keys should start with "model.visual.*"
bad_visual = [k for k in keys if k.startswith("model.language_model.visual.")]
assert len(bad_visual) == 0, f"Incorrect visual keys found:\n" + "\n".join(bad_visual[:3])
```

### Expected behavior

## Expected vs Actual Behavior

Comparing keys from the **original model weights** (reference) vs. checkpoint saved by `save_pretrained`:

| Module | Original model (reference) | Saved by 5.5.0 | Saved by 5.5.3 |
|---|---|---|---|
| Text layers | `model.language_model.layers.*` | `model.language_model.language_model.language_model.layers.*` ❌ | `model.language_model.layers.*` ✅ |
| Visual encoder | `model.visual.*` | `model.language_model.visual.*` ❌ | `model.language_model.visual.*` ❌ |

- **v5.5.0**: Both text and visual keys are broken.
- **v5.5.3**: Text keys are fixed (by #45340), but **visual encoder keys remain incorrect** — `model.language_model.visual.*` instead of `model.visual.*`.

## Impact

- Checkpoints saved after fine-tuning cannot be loaded by `from_pretrained` without manual key conversion.
- Inference frameworks (vLLM, lmdeploy, etc.) fail to load fine-tuned Qwen3.5 checkpoints.

## Related

- [#45216](https://github.com/huggingface/transformers/issues/45216) — Original report of the `save_pretrained` regression in 5.4.0
- [#45340](https://github.com/huggingface/transformers/pull/45340) — PR that fixed the text model key regression (but not the visual encoder)

--- Comment by Cyrilvallez at 2026-04-10T10:10:17Z ---
Oh damn 🥲 Having a look rn, thanks for the report @johnking0099 and verry sorry about that!

--- Comment by TtLuckyyy at 2026-04-26T13:29:15Z ---
Really help me!!

--- Comment by Cyrilvallez at 2026-05-12T07:45:12Z ---
Do you still experience the issue on latest main @TtLuckyyy?


ISSUE #45356: Regression in Kimi-K2.5 tokenizer from 5.3.0 to 5.4.0: incorrect codec handling and misleading fix_mistral_regex warning
State: closed  |  Labels: bug
Author: Lander-Hatsune  |  Created: 2026-04-10T09:42:26Z

### System Info

- OS: Linux
- Python: 3.10.12
- Model/tokenizer: `moonshotai/Kimi-K2.5`
- `trust_remote_code=True`

### Who can help?

@ArthurZucker and @itazap

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

There seems to be a regression for the Kimi-K2.5 tokenizer between `transformers==5.3.0` and `5.4.0+`.

In `5.3.0`, the tokenizer handles the `</think>` token correctly.

In `5.4.0` and later, this behavior breaks, and the warning shown by Transformers suggests using `fix_mistral_regex=True`, but following that suggestion triggers another error for this tokenizer.

### Reproduction

#### Works in 5.3.0

```python
from transformers import AutoTokenizer

t = AutoTokenizer.from_pretrained("moonshotai/Kimi-K2.5", trust_remote_code=True)
print(t.decode([163607]))
```

Output:

```python
'</think>'
```

#### Broken in 5.4.0+

```python
from transformers import AutoTokenizer

t = AutoTokenizer.from_pretrained("moonshotai/Kimi-K2.5", trust_remote_code=True)
print(t.decode([163607]))
```

Output:

```python
''
```

At the same time, Transformers emits this warning:

```text
The tokenizer you are loading from 'moonshotai/Kimi-K2.5' with an incorrect regex pattern...
You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
```

#### But following the warning also fails

```python
from transformers import AutoTokenizer

t = AutoTokenizer.from_pretrained(
    "moonshotai/Kimi-K2.5",
    trust_remote_code=True,
    fix_mistral_regex=True,
)
```

This raises:

```text
AttributeError: 'tokenizers.Tokenizer' object has no attribute 'backend_tokenizer'
```

### Expected behavior

### Expected behavior

- No regression in special-token handling between `5.3.0` and `5.4.0+`
- `t.decode([163607])` should still return `'</think>'`
- The warning should not recommend `fix_mistral_regex=True` for a tokenizer path that will immediately fail

### Actual behavior

- `</think>` handling is broken in `5.4.0+`
- The new warning points users to `fix_mistral_regex=True`, but that path crashes for this tokenizer

--- Comment by ArthurZucker at 2026-04-10T09:58:29Z ---
Having a look, its not expected but also its remote code soooooo yeah 😢

--- Comment by alan-cooney-dsit at 2026-04-13T15:24:37Z ---
Thanks for fixing @ArthurZucker  - please can we get a patch release for this one?

--- Comment by ArthurZucker at 2026-04-13T16:43:05Z ---
Yes we are pushing it!


A little bug in testing_utils.py

ISSUE #45341: A little bug in testing_utils.py
State: closed  |  Labels: bug
Author: MHRDYN7  |  Created: 2026-04-09T13:05:35Z

### System Info

.

### Who can help?

I was trying to run integration tests on cpu for a new model addition PR then got hit by an error due to line 3220

https://github.com/huggingface/transformers/blob/2fae57f5da9b0108d0c4da2692f5a702e6fb8c02/src/transformers/testing_utils.py#L3215-L3235

The reason is that I have CUDA installed on my system (lightning ai studio), but I did not have any GPU, so
```python
IS_CUDA_SYSTEM = torch.version.cuda is not None
```
was correctly set to the version and then the error occurred in line 3223
```python
major, minor = torch.cuda.get_device_capability()
```
as I did not have any gpu.

If this is not the intended behavior, the fix is simply changing line 3220 to `if (IS_CUDA_SYSTEM or IS_ROCM_SYSTEM) and torch.cuda.is_available():` . In this case we will also need to remove the `import torch` inside, which anyway seems redundant to me.

@remi-or I see you worked on this area most recently (10 months ago), otherwise could you please ping the right person

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

.

### Expected behavior

.

--- Comment by github-actions[bot] at 2026-05-10T08:33:59Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #45335: [t5gemma] resize_token_embeddings does not effect to decoder.embed_tokens
State: closed  |  Labels: bug
Author: KoichiYasuoka  |  Created: 2026-04-09T08:11:47Z

### System Info

- `transformers` version: 5.5.1
- Platform: Linux-6.6.113+-x86_64-with-glibc2.35
- Python version: 3.12.13
- Huggingface_hub version: 1.9.2
- Safetensors version: 0.7.0
- Accelerate version: 1.13.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0+cpu (NA)
- Using distributed or parallel set-up in script?: <fill in>

(Google Colaboratory)

### Who can help?

@zucchini-nlp

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Quick reproduce:

```
from transformers import T5GemmaForConditionalGeneration
mdl = T5GemmaForConditionalGeneration.from_pretrained("harshaljanjani/tiny-t5gemma-test")
e = mdl.get_input_embeddings()
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features
e = mdl.resize_token_embeddings(e.num_embeddings + 1)
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features
```

### Expected behavior

All `e.num_embeddings` `f.num_embeddings` and `g.out_features` should be increased to 256001.

--- Comment by KoichiYasuoka at 2026-04-09T08:15:06Z ---
Sorry @zucchini-nlp but I've just found another bug in `resize_token_embeddings` of T5Gemma. It seems similar to #45276, but slightly different.

--- Comment by zucchini-nlp at 2026-04-09T09:38:28Z ---
Oh... I'll check it today

--- Comment by KoichiYasuoka at 2026-04-10T03:27:34Z ---
Thank you @zucchini-nlp for including T5Gemma in #45324. Well, do you have a plan to include T5Gemma2 similar way?

```
from transformers import T5Gemma2ForConditionalGeneration
mdl = T5Gemma2ForConditionalGeneration.from_pretrained("kingabzpro/t5gemma2-latex-ocr-1k")
e = mdl.get_input_embeddings()
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features
e = mdl.resize_token_embeddings(e.num_embeddings + 1)
f = mdl.model.decoder.embed_tokens
g = mdl.get_output_embeddings()
print(e.num_embeddings, f.num_embeddings, g.out_features)
assert e.num_embeddings == f.num_embeddings == g.out_features
```

--- Comment by zucchini-nlp at 2026-04-14T16:15:21Z ---
The weights are resized since they are tied in T5Gemma2, so i see that it is only attributes. I will update the `out_features` so it is applied in all models, but I wish to just keep `decoder.embed_tokens` for now without dirty hacks

Better fixed at large scale for tied weights


ISSUE #45331: [Gemma4] Bug: audio token missing newline separators in chat_template.jinja causes multimodal failure when image precedes audio
State: closed  |  Labels:
Author: LfWhat  |  Created: 2026-04-09T03:35:46Z

## Bug Description

In `chat_template.jinja` for Gemma4, the image token has `\n\n` separators
but the audio token does not:

```jinja
{%- elif item['type'] == 'image' -%}
    {{- '\n\n<|image|>\n\n' -}}   ← has \n\n
{%- elif item['type'] == 'audio' -%}
    {{- '<|audio|>' -}}            ← missing \n\n
This causes the model to fail completely when image is placed before audio in the message content list.

Reproduction
python
from transformers import AutoProcessor, AutoModelForMultimodalLM
import torch

processor = AutoProcessor.from_pretrained("google/gemma-4-E2B-it")
model = AutoModelForMultimodalLM.from_pretrained(
    "google/gemma-4-E2B-it",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# ❌ image before audio → model fails
messages_image_first = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": IMAGE_PATH},
            {"type": "audio", "audio": AUDIO_PATH},
            {"type": "text", "text": "Describe the image and audio."},
        ]
    }
]

# ✅ audio before image → works correctly
messages_audio_first = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": AUDIO_PATH},
            {"type": "image", "url": IMAGE_PATH},
            {"type": "text", "text": "Describe the image and audio."},
        ]
    }
]
Root Cause
The jinja template inserts \n\n around image tokens but not around audio tokens.

Image-first token sequence (broken):

<|image>...<image|>\n\n<|audio>...<audio|>Describe the image and audio.
                                          ↑ audio token directly concatenated with text, no separator
Audio-first token sequence (correct):

<|audio>...<audio|>\n\n<|image>...<image|>\n\n Describe the image and audio.
                    ↑                      ↑ correct \n\n separators
Note also that before the fix, the two orderings produce different input_ids shapes:

audio-first shape: torch.Size([1, 738])
image-first shape: torch.Size([1, 737])  ← 1 token missing due to missing \n\n
Evidence
Top 10 next tokens with image-first (before fix):

'<turn|>': 0.7383   ← model immediately ends the turn
'<eos>':   0.1406
Top 10 next tokens with image-first (after fix):

'这张': 0.6797      ← model correctly starts generating
'好的': 0.2656
Full generation with audio-first (before fix):

这张图片展示了一个教室的场景。画面中有一位戴眼镜的女性老师站在讲台后面...
音频中有人在呼喊"Look look look at the girl"...
Full generation with image-first (before fix):

(empty, model outputs <turn|> immediately)
Full generation with image-first (after fix):

这张图片展示了一个教室的场景，有几位学生和一位老师...
音频内容似乎是孩子们在进行某种对话或游戏...
Fix
In chat_template.jinja, change:

jinja
{%- elif item['type'] == 'audio' -%}
    {{- '<|audio|>' -}}
to:

jinja
{%- elif item['type'] == 'audio' -%}
    {{- '\n\n<|audio|>\n\n' -}}
After the fix, both orderings produce identical input_ids shapes and correct outputs.

Environment
transformers version: 5.5.0
Model: google/gemma-4-E2B-it

--- Comment by Rocketknight1 at 2026-04-09T13:29:26Z ---
We're currently updating the Gemma4 template at https://github.com/huggingface/transformers/pull/45257 ! I'll flag this.

--- Comment by github-actions[bot] at 2026-05-09T08:29:29Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #45325: Qwen2.5-VL get_rope_index scales still-image temporal position_ids by tokens_per_second in transformers 5.3.0
State: closed  |  Labels: bug
Author: ayaan-fw  |  Created: 2026-04-08T18:20:48Z

### System Info

- `transformers` version: 5.3.0
- Platform: Linux-5.15.0-131-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 1.9.0
- Safetensors version: 0.6.2
- Accelerate version: 1.13.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0a0+145a3a7bda.nv25.10 (CUDA)
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: NVIDIA H200

### Who can help?

@zucchini-nlp @yonigozlan

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```import numpy as np
from PIL import Image

from transformers import Qwen2_5_VLConfig, Qwen2_5_VLProcessor
from transformers.models.qwen2_5_vl.modeling_qwen2_5_vl import Qwen2_5_VLModel

MODEL_ID = "Qwen/Qwen2.5-VL-3B-Instruct"

class DummyQwen2_5VLModel:
    # Reuse the public HF implementation without loading full model weights.
    get_rope_index = Qwen2_5_VLModel.get_rope_index
    get_vision_position_ids = Qwen2_5_VLModel.get_vision_position_ids

    def __init__(self, config):
        self.config = config

processor = Qwen2_5_VLProcessor.from_pretrained(MODEL_ID)
model = DummyQwen2_5VLModel(Qwen2_5_VLConfig.from_pretrained(MODEL_ID))

image = Image.fromarray(np.zeros((448, 448, 3), dtype=np.uint8))

prompt = (
    "<|im_start|>system\nYou are helpful.<|im_end|>\n"
    "<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|> describe<|im_end|>\n"
    "<|im_start|>assistant\n"
)

inputs = processor(
    text=prompt,
    images=[image],
    videos=None,
    return_tensors="pt",
    return_mm_token_type_ids=True,
)

position_ids, rope_deltas = model.get_rope_index(
    input_ids=inputs["input_ids"],
    mm_token_type_ids=inputs["mm_token_type_ids"],
    image_grid_thw=inputs["image_grid_thw"],
)

if position_ids.dim() == 3:
    position_ids = position_ids.squeeze(1)

image_start = (inputs["mm_token_type_ids"][0] == 1).nonzero(as_tuple=False)[0].item()

temporal = position_ids[0, image_start : image_start + 8].tolist()
height = position_ids[1, image_start : image_start + 8].tolist()
width = position_ids[2, image_start : image_start + 8].tolist()

print("tokens_per_second =", model.config.vision_config.tokens_per_second)
print("image_start =", image_start)
print("temporal =", temporal)
print("height =", height)
print("width =", width)

assert height[0] == image_start
assert width[0] == image_start

# This is the behavior I think is wrong for still images:
assert temporal[0] == image_start, (
    f"Expected temporal[0] == image_start for a still image, "
    f"but got temporal[0]={temporal[0]}, image_start={image_start}, "
    f"tokens_per_second={model.config.vision_config.tokens_per_second}"
)
```

Observed behavior on my side with `transformers==5.3.0`:
- `tokens_per_second = 4`
- the first still-image temporal position starts at `image_start * tokens_per_second`
- height and width still start at `image_start`

For example, if `image_start = 13`, the temporal IDs start at `52` while height/width start at `13`.

I think the relevant code path is:
- `get_rope_index()` uses the same branch for images and videos
- when `second_per_grid_ts` is absent, it defaults to `1`
- `time_interval = tokens_per_second * 1` is therefore applied to still images too
- `get_vision_position_ids()` then multiplies the still-image temporal origin by that interval

Relevant source in v5.3.0:
- `get_rope_index`: https://github.com/huggingface/transformers/blob/v5.3.0/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1144-L1175
- `get_vision_position_ids`: https://github.com/huggingface/transformers/blob/v5.3.0/src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py#L1021-L1073

### Expected behavior

For still images, I would expect the temporal RoPE origin to stay anchored at the image start position, just like height and width.

In other words, for image inputs I would expect:
- temporal IDs to start at `start_position`
- height IDs to start at `start_position`
- width IDs to start at `start_position`

and only videos should apply temporal scaling via `second_per_grid_ts * tokens_per_second`.

Right now in `transformers==5.3.0`, still images appear to inherit `tokens_per_second` scaling, which shifts the temporal origin even though a still image has no temporal spacing to encode.

--- Comment by Kash6 at 2026-04-08T23:50:47Z ---
Reproduced this on an RTX5080 with transformers 5.5.0.dev0. The root cause is in get_rope_index; time_interval = tokens_per_second * second_per_grid_ts is applied to both images and videos, but still images have no temporal dimension to scale. The fix is to only apply temporal scaling when modality_type == 2 (video) and use time_interval=1 for images.


ISSUE #45322: [Model Request] Add EUPE (Efficient Universal Perception Encoder) by Meta AI
State: open  |  Labels: New model
Author: forensicmike  |  Created: 2026-04-08T15:52:40Z

### Model description

I would like to request adding support for EUPE (Efficient Universal Perception Encoder), a recent vision backbone released by Meta AI.

Paper: https://arxiv.org/abs/2603.22387
Code: https://github.com/facebookresearch/eupe

EUPE is a multi-purpose vision encoder trained via distillation from multiple domain-specific foundation models into a unified representation. It is designed to be efficient (edge-friendly), general-purpose (strong across diverse downstream tasks), and competitive with specialized encoders of similar size.

The model family includes both Vision Transformers (ViT-T/S/B, patch size 16) and ConvNeXt backbones (T/S/B) (see model weights on huggingface at [facebook/eupe](https://huggingface.co/collections/facebook/eupe).

I would be willing to work on a PR for this model, including: model implementation (ViT + ConvNeXt variants), weight conversion script, tests and docs. Happy to align with maintainers on: scope (e.g., start with ViT only), naming conventions, integration approach.

Note that the non-commmercial research license for these models mirrors other Meta model releases (e.g. DINO, SAM) which already have implementations in the library.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

https://github.com/facebookresearch/eupe
https://huggingface.co/facebook/EUPE-ViT-B
@zcck3rn3l

--- Comment by Rocketknight1 at 2026-04-09T13:28:31Z ---
cc @nielsrogge @molbap

--- Comment by Aryan8912 at 2026-04-28T02:29:11Z ---
Could I work on this?


ISSUE #45313: Qwen3.5: DeepSpeed ZeRO-3 fails to load weights for language_model
State: closed  |  Labels: Should Fix
Author: debOliveira  |  Created: 2026-04-08T11:44:10Z

## System Info
- `transformers` version: 5.4.0
- Platform: Linux (H200 x4)
- Python version: 3.12.0
- DeepSpeed version: 0.18.5
- PyTorch version: 2.8.0+cu128 (CUDA)

## Problem
When loading Qwen/Qwen3.5-27B (also tested with 9B) with DeepSpeed ZeRO-3, `language_model` parameters are reported as MISSING in the load report.
```
Key                                                                  | Status  | Details
---------------------------------------------------------------------+---------+--------
model.language_model.layers.{0...63}.post_attention_layernorm.weight | MISSING |
model.language_model.layers.{0...62}.linear_attn.out_proj.weight     | MISSING |
model.language_model.norm.weight                                     | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_qkv.weight  | MISSING |
model.language_model.layers.{0...62}.linear_attn.conv1d.weight       | MISSING |
model.language_model.layers.{3...63}.self_attn.q_norm.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.dt_bias             | MISSING |
model.language_model.layers.{0...62}.linear_attn.A_log               | MISSING |
model.language_model.layers.{0...62}.linear_attn.norm.weight         | MISSING |
model.language_model.layers.{0...63}.mlp.gate_proj.weight            | MISSING |
model.language_model.layers.{0...63}.mlp.down_proj.weight            | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_a.weight    | MISSING |
model.language_model.layers.{3...63}.self_attn.q_proj.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.k_proj.weight         | MISSING |
model.language_model.layers.{0...63}.input_layernorm.weight          | MISSING |
model.language_model.layers.{0...63}.mlp.up_proj.weight              | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_b.weight    | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_z.weight    | MISSING |
model.language_model.layers.{3...63}.self_attn.k_norm.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.o_proj.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.v_proj.weight         | MISSING |
model.language_model.embed_tokens.weight                             | MISSING |
```

## Cause hypothesis

In `conversion_mapping.py`, the `language_model` weight keys are remapped to `model` only. This conversion is called in `_load_pretrained_model` when DeepSpeed ZeRO-3 is turned on.
https://github.com/huggingface/transformers/blob/d081c718b8825036a7662ec819313e5141dc34b5/src/transformers/conversion_mapping.py#L155-L157

The problem disappears when setting `target_patterns="model.language_model"` or using ZeRO-2.

## Reproduction
```python
from transformers import Qwen3_5ForConditionalGeneration
from transformers.integrations import HfDeepSpeedConfig

ds_cfg = {
    "bf16": {"enabled": True},
    "train_micro_batch_size_per_gpu": 1,
    "train_batch_size": 1,
    "zero_optimization": {
        "stage": 3,
    }
}
dschf = HfDeepSpeedConfig(ds_cfg)
model = Qwen3_5ForConditionalGeneration.from_pretrained("Qwen/Qwen3.5-27B", torch_dtype="auto")
```

## Expected Behavior

Model weights should load correctly with DeepSpeed ZeRO-3.

## Related issues
- #45310
- #45216

--- Comment by zucchini-nlp at 2026-04-08T13:37:17Z ---
Duplicate of https://github.com/huggingface/transformers/issues/45216 and will be fixed soon in https://github.com/huggingface/transformers/pull/45314

--- Comment by debOliveira at 2026-04-09T18:57:13Z ---
I tried the recent patch up to commit https://github.com/huggingface/transformers/commit/9d840ea5f31a14d0d66c176f42ed300d7f1d2f1c, but the issue still remains. Note that it works fine on stage 2 of deepspeed, it only raises on stage 3.

Using the same reproduction section above, I get:
```
Key                                                                  | Status  |
---------------------------------------------------------------------+---------+-
model.language_model.layers.{0...63}.input_layernorm.weight          | MISSING |
model.language_model.layers.{0...62}.linear_attn.out_proj.weight     | MISSING |
model.language_model.layers.{0...62}.linear_attn.dt_bias             | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_qkv.weight  | MISSING |
model.language_model.layers.{0...63}.mlp.down_proj.weight            | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_a.weight    | MISSING |
model.language_model.layers.{0...63}.post_attention_layernorm.weight | MISSING |
model.language_model.layers.{0...62}.linear_attn.norm.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.A_log               | MISSING |
model.language_model.layers.{0...63}.mlp.gate_proj.weight            | MISSING |
model.language_model.layers.{3...63}.self_attn.v_proj.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_z.weight    | MISSING |
model.language_model.layers.{3...63}.self_attn.o_proj.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_b.weight    | MISSING |
model.language_model.layers.{3...63}.self_attn.q_norm.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.q_proj.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.conv1d.weight       | MISSING |
model.language_model.embed_tokens.weight                             | MISSING |
model.language_model.layers.{0...63}.mlp.up_proj.weight              | MISSING |
model.language_model.layers.{3...63}.self_attn.k_proj.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.k_norm.weight         | MISSING |
model.language_model.norm.weight                                     | MISSING |
```

--- Comment by zucchini-nlp at 2026-04-10T09:18:20Z ---
@Cyrilvallez , then there is definitely smth going on internally in deepspeed

--- Comment by debOliveira at 2026-04-10T18:58:08Z ---
I tested the new patch at #45358, but still no success. The weights under `model.language_model.layers` are being remapped to only `model.layers`. From what I understand the target being `r"^model.(?!(?:language_model.|visual.))"` excludes `language_model` string.

A bad fix that I tested (and works) is
```python
WeightRenaming(
    source_patterns=r"^model.language_model.",
    target_patterns=r"^model.language_model."
)
```

--- Comment by BartekKruczek at 2026-04-14T09:27:44Z ---
Hello 👋

The same problem (PEFT + LoRA + Accelerate + Deepspeed3) on 4 x H200 with [Qwen3.5](https://huggingface.co/Qwen/Qwen3.5-27B) model.

```bash
Key                                                                  | Status  |
---------------------------------------------------------------------+---------+-
model.language_model.layers.{0...62}.linear_attn.in_proj_qkv.weight  | MISSING |
model.language_model.layers.{3...63}.self_attn.k_proj.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.dt_bias             | MISSING |
model.language_model.layers.{3...63}.self_attn.o_proj.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.A_log               | MISSING |
model.language_model.layers.{0...63}.mlp.up_proj.weight              | MISSING |
model.language_model.layers.{0...62}.linear_attn.out_proj.weight     | MISSING |
model.language_model.layers.{0...63}.post_attention_layernorm.weight | MISSING |
model.language_model.layers.{0...63}.mlp.gate_proj.weight            | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_b.weight    | MISSING |
model.language_model.layers.{0...62}.linear_attn.conv1d.weight       | MISSING |
model.language_model.layers.{3...63}.self_attn.k_norm.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.q_proj.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_a.weight    | MISSING |
model.language_model.layers.{0...63}.input_layernorm.weight          | MISSING |
model.language_model.layers.{0...63}.mlp.down_proj.weight            | MISSING |
model.language_model.layers.{0...62}.linear_attn.norm.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.q_norm.weight         | MISSING |
model.language_model.layers.{3...63}.self_attn.v_proj.weight         | MISSING |
model.language_model.layers.{0...62}.linear_attn.in_proj_z.weight    | MISSING |
model.language_model.embed_tokens.weight                             | MISSING |
model.language_model.norm.weight                                     | MISSING |
```

Edit: transformers 5.5.3, peft 0.18.1, deepspeed 0.18.9, accelerate 1.13.0

--- Comment by debOliveira at 2026-04-14T11:52:21Z ---
Also tried the patch at #45402, which seemed like a similar issue, but no success. I'm using my bad fix at the moment.

--- Comment by zucchini-nlp at 2026-04-14T12:24:39Z ---
Sorry for long delay, since Cyril is off until tomorrow I tried to reproduce the code and found the root cause. The prev fix is also not completely correct and works because we accidentally end up exploiting another code path

@Cyrilvallez , the regex in `qwen3_5_text` still matches and replaces sd keys in VLMs. Your PR fixed saving it back, but the loading is broken in DS. This line in loading fn reverts back any conversion, if the converted key is not found in sd (which i believe it not intended). DS load fn, in contrary, doesn't revert any conversion, so it fails to match keys 🙃

https://github.com/huggingface/transformers/blob/4396b1b5ce6900a600ab82423a12963bdd2fc1f2/src/transformers/core_model_loading.py#L1219-L1221

I will let you decide how to fix it when you're back. Personally I believe we shouldn't revert a conversion if not in state dict (I also found that this loophole affected prev CLIP issues, but it's another story)

@debOliveira we'll fix and add it in the next patch, thanks for pinging again :)

--- Comment by github-actions[bot] at 2026-05-09T08:29:31Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by Cyrilvallez at 2026-05-13T02:44:22Z ---
@zucchini-nlp The fix to revert conversion if not in state_dict was originally added to fix some remote code models, but I don't really like it either. It's kind of weird, and dangerous. Do you know if based on latest changes to all that machinery we still face the issue?

--- Comment by zucchini-nlp at 2026-05-18T01:33:50Z ---
Lemme check, tbh there were so many changes in conversion lately so msth might have been accidentally fixed

Agreed that it is dangerous. Do you remember which remote models those were? If it is a  model with small num of donwloads, we could deliberately break it and ask owner to pin `transformers` version in model doc

--- Comment by akshansh47 at 2026-05-19T17:56:35Z ---
We hit this exact load report on Qwen3.5-27B + DS-Z3 across 6 training runs and
shipped two complementary workarounds while waiting for the upstream fix.

### LoRA serving / adapter side

PEFT writes `base_model.model.model.language_model.layers.N.lora_*` keys; vLLM
< 0.19 looks up `base_model.model.model.layers.N.*` and silently falls back to
1-D placeholders, surfacing as `IndexError: too many indices for tensor of
dimension 1` at `vllm/lora/layers/column_parallel_linear.py:slice_lora_b`.

Strip the prefix in-place before upload:

```python
import os
import safetensors.torch as st

LANGUAGE_MODEL_PREFIX = ".language_model."

tensors = st.load_file(adapter_path)
if any(LANGUAGE_MODEL_PREFIX in k for k in tensors):
    renamed = {k.replace(LANGUAGE_MODEL_PREFIX, ".", 1): v for k, v in tensors.items()}
    st.save_file(renamed, adapter_path + ".tmp")
    os.replace(adapter_path + ".tmp", adapter_path)
```

vLLM 0.19+ accepts both naming conventions ([vllm#37375](https://github.com/vllm-project/vllm/pull/37375)).

### Training side under ZeRO-3

Direct attribute access to `lm_head.weight` / `linear_attn.conv1d.weight` returns
the rank-local 1-D shard. Wrap in `deepspeed.zero.GatheredParameters` for any
read that happens outside `model.forward()`:

```python
import deepspeed

with deepspeed.zero.GatheredParameters([weight], modifier_rank=None):
    full = weight.data.detach().clone()  # [V, H] survives context exit
```

Required for chunked LM heads, hybrid Mamba conv1d kernels, and any custom loss
that reads weights directly. Same architectural mismatch as the Qwen3-Next fix
in [#43926](https://github.com/huggingface/transformers/pull/43926); the 2-D
`language_model.*` weights hit a different branch but the same `zero.init()`
context-manager gap.

--- Comment by zucchini-nlp at 2026-05-20T01:09:03Z ---
The script provided above works now for me on `main` branch so I assume one of the many recent changes on conversion fixed it. Anyone who encounters the issue, please install from source and try again

I checked with

```python
from transformers import Qwen3_5ForConditionalGeneration
from transformers.integrations import HfDeepSpeedConfig

ds_cfg = {
    "bf16": {"enabled": True},
    "train_micro_batch_size_per_gpu": 1,
    "train_batch_size": 1,
    "zero_optimization": {
        "stage": 3,
    }
}
dschf = HfDeepSpeedConfig(ds_cfg)
model = Qwen3_5ForConditionalGeneration.from_pretrained("Qwen/Qwen3.5-27B", torch_dtype="auto")

```

--- Comment by zucchini-nlp at 2026-05-20T01:09:29Z ---
Also closing as resolved, but if you still see issues feel free to ping me and Cyril


ISSUE #45310: [BUG] transformers>=5.4.0, Qwen3.5 Moe from_pretrained error
State: closed  |  Labels: bug
Author: Jintao-Huang  |  Created: 2026-04-08T09:29:48Z

### System Info

```
import os
os.environ['CUDA_VISIBLE_DEVICS'] = '0'

from transformers import Qwen3_5ForConditionalGeneration, AutoTokenizer

model = Qwen3_5ForConditionalGeneration.from_pretrained('Qwen/Qwen3.5-35B-A3B')
model.save_pretrained('/root/Qwen3.5-35B-A3B', max_shard_size='10GB')
model = Qwen3_5ForConditionalGeneration.from_pretrained('/root/Qwen3.5-35B-A3B')
```

<img width="689" height="328" alt="Image" src="https://github.com/user-attachments/assets/32dc8552-d632-4057-a374-d2ad1556b64c" />

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

-

--- Comment by zucchini-nlp at 2026-04-08T11:31:12Z ---
Duplicate of https://github.com/huggingface/transformers/issues/45216, opening a pr today

--- Comment by Cyrilvallez at 2026-04-09T12:39:29Z ---
Hey! You are not using the auto mappings, and you're using the wrong model type. `'Qwen/Qwen3.5-35B-A3B'` is a `Qwen3_5MoeForConditionalGeneration`, not `Qwen3_5ForConditionalGeneration` (a warning shows up to tell you).
The rest is fixed in https://github.com/huggingface/transformers/pull/45340


ISSUE #45308: Feature request: Support evaluation every N epochs in TrainingArguments
State: open  |  Labels: Feature request
Author: varuna-km-18267  |  Created: 2026-04-08T06:22:20Z

### Feature request

Currently, Trainer supports evaluation strategies:
- "epoch": evaluate every epoch
- "steps": evaluate every N steps

However, there is no built-in way to evaluate every N epochs (e.g., every 5 epochs).

This is particularly useful when:
- evaluation is computationally expensive
- running large-scale training
- benchmarking periodically instead of every epoch

Suggested API:
- Add parameter like `eval_epochs` (int)
- Or extend `eval_strategy="epoch"` to support frequency

Example:
eval_strategy="epoch"
eval_epochs=5  # evaluate every 5 epochs

Currently, this requires custom Trainer overrides or callbacks.

### Motivation

-

### Your contribution

-

--- Comment by Rocketknight1 at 2026-04-08T14:56:05Z ---
cc @SunMarc

--- Comment by SunMarc at 2026-04-09T13:27:15Z ---
Thanks for the feature request. I'm not sure if it is worth adding a new arguments for that, so i'll wait to see if the community is interested in that. For now, you can also just specify steps even it is a bit for complicated.

--- Comment by pdjoshi-30 at 2026-05-01T16:29:18Z ---
Hi @SunMarc,

I’d like to work on this feature upon discussion and feedback

I agree that adding a new argument directly may increase API complexity. Instead of introducing a completely separate parameter, I was thinking of extending the existing `evaluation_strategy="epoch"` behavior with an optional frequency parameter like `eval_epochs`.

This way:
- backward compatibility is preserved (default = 1 epoch)
- no change for existing users
- allows flexibility for large-scale training where evaluation is expensive

Alternatively, we could also consider integrating this logic into the existing control flow without adding a completely new strategy.

Would love your thoughts before proceeding.


Bug, Generation

ISSUE #45307: Bug, Generation
State: closed  |  Labels:
Author: Regata3010  |  Created: 2026-04-08T04:07:37Z

## Title

`AssistantToTargetTranslator` crashes with `AttributeError: 'map_input_embeddings'` when using assisted generation with cross-vocab models

## Description

When using assisted generation (`model.generate(assistant_model=...)`) with models that have different vocabulary sizes but share the same tokenizer family, the `AssistantToTargetTranslator` crashes because `map_input_embeddings` is never initialized.

This affects model pairs like Qwen2.5-7B (vocab=152,064) + Qwen2.5-0.5B (vocab=151,936), which share the same Qwen2.5 tokenizer but have different vocab padding.

## Reproduction

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

target = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B", device_map="auto")
draft = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B")

input_ids = tokenizer.encode("Hello world", return_tensors="pt").to("cuda")

# This crashes:
output = target.generate(
    input_ids,
    assistant_model=draft,
    tokenizer=tokenizer,
    assistant_tokenizer=tokenizer,
    max_new_tokens=32,
)
```

Without `tokenizer` and `assistant_tokenizer`, it raises:
```
ValueError: The main and assistant models have different tokenizers.
```

With `tokenizer` and `assistant_tokenizer`, it raises:
```
AttributeError: 'AssistantToTargetTranslator' object has no attribute 'map_input_embeddings'
```

## Traceback

```
File "transformers/generation/utils.py", line 2521, in generate
    result = decoding_method(...)
File "transformers/generation/utils.py", line 3514, in _assisted_decoding
    candidate_input_ids, candidate_logits = candidate_generator.get_candidates(input_ids)
File "transformers/generation/candidate_generator.py", line 933, in get_candidates
    assistant_input_ids, num_added_tokens = self._prepare_assistant_input_ids(target_input_ids)
File "transformers/generation/candidate_generator.py", line 1009, in _prepare_assistant_input_ids
    self._atm_translator.unmap_input_ids()
File "transformers/generation/candidate_generator.py", line 754, in unmap_input_ids
    self.map_input_embeddings.map = False
AttributeError: 'AssistantToTargetTranslator' object has no attribute 'map_input_embeddings'
```

## Expected Behavior

Assisted generation should work with models from the same family that have slightly different vocab sizes, either by:
1. Properly initializing `map_input_embeddings` in `AssistantToTargetTranslator.__init__`
2. Or handling the case where the tokenizer is the same but vocab sizes differ (padding tokens)

## Environment

- `transformers` version: 5.4.0
- `torch` version: 2.11.0+cu128
- Python: 3.13.5
- GPU: NVIDIA H200
- OS: Linux (RHEL 9, HPC cluster)

## Context

Found while benchmarking a from-scratch speculative decoding implementation against HF's assisted generation across multiple model pairs. The bug only triggers with cross-vocab model pairs (e.g., Qwen2.5 family). Same-vocab pairs (e.g., Llama-3.1-8B + Llama-3.2-1B) work correctly.

## Workaround

Catching the error and falling back:
```python
try:
    output = target.generate(input_ids, assistant_model=draft, ...)
except (ValueError, AttributeError):
    # Fall back to standard generation
    output = target.generate(input_ids, max_new_tokens=32)
```

--- Comment by zucchini-nlp at 2026-04-08T13:50:19Z ---
Hm indeed, we need to check that `len(self._suppress_input_ids) > 0` as well imo when trying to unmap input ids here. Do you want to submit a PR?

https://github.com/huggingface/transformers/blob/c850500b54dfd0b4df22a6cf143a06955cc3a733/src/transformers/generation/candidate_generator.py#L746-L754

--- Comment by Regata3010 at 2026-04-08T15:56:48Z ---
Submitted a PR Request #45320

--- Comment by zucchini-nlp at 2026-04-10T09:30:25Z ---
Merged!


Remove import * usage

ISSUE #45306: Remove import * usage
State: closed  |  Labels:
Author: yueyinqiu  |  Created: 2026-04-08T00:22:53Z

Sorry but I want to revisit a closed issue #41669 here. That issue was automatically closed at the time without further responses. I hope we can reconsider the validity of that issue.

As I mentioned in that issue, I believe this is a serious enough problem. Actually, we're on the same page: we all want IDE autocomplete to work properly. But now it's making Pylance, a very popular type checker, almost unusable. This goes against our original intention.

I believe it's necessary for us to adjust this, even if it leads to some disruptive changes, but it is worthwhile. This issue will become increasingly serious as more content is embedded within transformers.

(Furthermore, it seems to cause some non-existent types to be incorrectly detected by the static checker, e.g. `transformers.BaseStreamer`. I'm not that familiar with this, so this behavior might be expected.)

--- Comment by Rocketknight1 at 2026-04-08T14:53:25Z ---
You can propose solutions if you want, but we're probably not going to break the repo too much for mypy/pylance compatibility!

--- Comment by github-actions[bot] at 2026-05-08T08:27:53Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.