Nonexistant import from image_utils

ISSUE #44933: Nonexistant import from image_utils
State: closed  |  Labels: bug
Author: josh-kean  |  Created: 2026-03-22T17:57:33Z

### System Info

I was getting the following error when running the latest version of main

`ImportError: cannot import name 'PILImageResampling' from 'transformers.image_utils' (/Users/josh/Documents/sandbox/.hugging/lib/python3.12/site-packages/transformers/image_utils.py)`

I found where it's being imported in src/transformers/video_processing_utils.py and it looks like this package isn't being used in video_processing_utils.py nor is it defined in image_utils.py

commenting out the import in video_processing_utils fixed the issue

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. install tranformers from source uv pip install '.[torch]'
2. run transformers serve
3. run chat cli transformers chat Qwen/Qwen3-4B

you'll get the error
ImportError: cannot import name 'PILImageResampling' from 'transformers.image_utils'

### Expected behavior

1. install tranformers from source uv pip install '.[torch]'
2. run transformers serve
3. run chat cli transformers chat Qwen/Qwen3-4B

the chat runs as expected

--- Comment by anshuS1310 at 2026-03-22T18:57:52Z ---
The Three location where the PILImageResampling is used in video_processing_utils.py

- Location 1 (Docstring): resample (PILImageResampling, *optional*...)
Impact: Zero. This is just a comment for humans. It won't cause an error.

- Location 2 (The Import): from .image_utils import (... PILImageResampling ...)
Impact: CRITICAL. This is the line that is crashing your program. Because image_utils fails to export that name, Python stops execution immediately here.

- Location 3 (Type Hint): resample: "PILImageResampling | ..."
Impact: Low. Because it is in quotes ("..."), Python treats it as a string. It doesn't look for the object until a type-checking tool (like MyPy) runs.

we can safely remove the PILImageResampling import from the importer line or for safety purpose we can use

<pre>
# Attempt to import the one that might be missing
try:
    from .image_utils import PILImageResampling
except ImportError:
    # If it's missing, define a placeholder so the type hints don't crash
    PILImageResampling = None

--- Comment by josh-kean at 2026-03-22T19:18:16Z ---
i can push that change up later, but if somebody else grabs it i won't be offended

--- Comment by anshuS1310 at 2026-03-22T19:27:02Z ---
Should I push it or you want to push ?

--- Comment by josh-kean at 2026-03-23T00:58:14Z ---
i had to step out, if you have an mr i'd say push it otherwise i'd be happy to!

--- Comment by zucchini-nlp at 2026-03-23T09:57:38Z ---
It is defined in `image_utils` as long as the env has Pillow installed. Ig your env didn't have Pillow, so we might need the placeholder when vision isn't found inside `image_utils.py` file

https://github.com/huggingface/transformers/blob/55cc1a7fb8e53a5e7e35ca9cf9759498f20abb93/src/transformers/image_utils.py#L46-L65

--- Comment by anshuS1310 at 2026-03-23T10:33:07Z ---
But there is no use of PILImageResampling in video_processing_utils.py still it is imported @zucchini-nlp

--- Comment by zucchini-nlp at 2026-03-23T13:25:43Z ---
it is imported for type checking. We can also import `if TYPE_CHECKING: import PILImageResampling`

--- Comment by josh-kean at 2026-03-23T13:47:57Z ---
im getting errors that may have been ignored in other mr's when running make check-repo. gonna look into those and see if I can wrap some more import fixes into this mr.

--- Comment by github-actions[bot] at 2026-04-22T08:30:39Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44929: First-class fine-tuning support for Mamba / Mamba-2 SSMs — architecture is production-ready, but the training path in Transformers isn't
State: open  |  Labels: Feature request
Author: lochanharishwar  |  Created: 2026-03-22T17:23:58Z

### Feature request

You can load Mamba models in Transformers — but the moment you try to actually fine-tune one, things fall apart fast. The standard Trainer was built around attention + KV cache assumptions that SSMs simply don't share. Gradient checkpointing breaks in weird ways, DataCollatorForLanguageModeling doesn't account for SSM inputs, and LoRA targeting on Mamba layers is a total wild west — everyone's doing it differently, nobody's sure if it's right. You're shipping hybrid SSM models like OLMo Hybrid in v5.3 but fine-tuning them reliably still feels like a DIY project.

**What I'm asking for**

A simple official Mamba fine-tuning example script — just one reference implementation people can trust

### Motivation

SSM and hybrid architectures aren't experimental anymore — they're in production. The people trying to fine-tune them on medical, legal, and scientific text are exactly who needs this to just work. Right now it doesn't.

### Your contribution

point me at the right files and I'll take a shot at the example script or docs.

--- Comment by Rocketknight1 at 2026-03-23T14:21:17Z ---
Seems like a good suggestion. The `examples` are often a bit outdated and unmaintained. A good place to put this would be towards the end of the model card for Mamba, and after we review it, it could be either referenced or copied to the other SSM/hybrid model cards.

--- Comment by hemantmm at 2026-03-23T17:11:24Z ---
@Rocketknight1, is this issue open to work on?

--- Comment by kf-rahman at 2026-03-23T21:24:03Z ---
@Rocketknight1 If its still open would love to help out on this

--- Comment by Rocketknight1 at 2026-03-24T12:43:27Z ---
The original author said he was willing to, so I'll let him decide if he wants to do it or let anyone else take it! cc @lochanharishwar

--- Comment by lochanharishwar at 2026-04-02T14:59:27Z ---
Thank You for notice ,if possible I also would like to work on it


ISSUE #44928: [Bug] Catastrophic gradient explosion (NaN) in RLHF with Qwen3.5 due to 3D position_ids forcing SDPA Math fallback and BF16 collapse
State: open  |  Labels: WIP, bug
Author: ouroborosscr  |  Created: 2026-03-22T16:46:05Z

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 5.3.0
- Platform: Linux-6.8.0-40-generic-x86_64-with-glibc2.35
- Python version: 3.11.15
- Huggingface_hub version: 1.7.1
- Safetensors version: 0.7.0
- Accelerate version: 1.13.0
- Accelerate config:    not found
- DeepSpeed version: 0.18.8
- PyTorch version (accelerator?): 2.10.0+cu128 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100-SXM4-80GB

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

We have fully isolated the issue and provided a reproducible repository here: 👉 https://github.com/ouroborosscr/Report-the-gradient-explosion-of-qwen3.5

1.Verify Dapo degradation
```python
CUDA_LAUNCH_BLOCKING=1 CUDA_HOME=/usr/local/cuda-12.8 LD_PRELOAD=/opt/anaconda3/envs/scr_train2/lib/libstdc++.so.6 CUDA_VISIBLE_DEVICES=2 torchrun --nproc_per_node=1 audit_qwen_hf.py
```
Output:
```
============================================================
🔍 [源码级审计] Qwen3.5 (SDPA 模式) 到底传了什么给底层？
============================================================
➡ 模拟 DAPO 训练输入，序列长度: 8192

🚨 [劫持成功] Qwen3.5 正在调用 PyTorch SDPA!
   Query 形状: torch.Size([1, 8, 8192, 128])
   ⚠ Qwen 传来的 Mask 形状: torch.Size([1, 1, 8192, 8192])
   ⚠ Mask 的数据类型: torch.bool
   💀 仅这个 Mask 矩阵就会占据显存: 0.125000 GB
   is_causal 参数: False
```

2.Verify gradient explosion

```python
CUDA_LAUNCH_BLOCKING=1 CUDA_HOME=/usr/local/cuda-12.8 LD_PRELOAD=/opt/anaconda3/envs/scr_train2/lib/libstdc++.so.6 CUDA_VISIBLE_DEVICES=2 torchrun --nproc_per_node=1 train_dapo_3_debug.py  --use_lora --use_4bit
```

Output:
```
……
💥 [爆点定位] 梯度在经过 【model.layers.30.linear_attn.norm】 的反向计算后瞬间爆炸！
   传出梯度最大值: 26240.0
⚠ [数值爆炸预警] 层: model.layers.27.self_attn.v_proj | 梯度 Max: 17280.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.27.self_attn.k_norm | 梯度 Max: 135168.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.27.self_attn.k_proj | 梯度 Max: 111616.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.27.self_attn.q_norm | 梯度 Max: 126976.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.27.self_attn.q_proj | 梯度 Max: 97280.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.23.self_attn | 梯度 Max: 67072.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.23.self_attn.o_proj | 梯度 Max: 67072.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.23.self_attn.v_proj | 梯度 Max: 13172736.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.23.self_attn.k_norm | 梯度 Max: 62390272.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.23.self_attn.k_proj | 梯度 Max: 45613056.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.23.self_attn.q_norm | 梯度 Max: 297795584.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.23.self_attn.q_proj | 梯度 Max: 242221056.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.19.self_attn | 梯度 Max: 132120576.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.19.self_attn.o_proj | 梯度 Max: 132120576.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.19.self_attn.v_proj | 梯度 Max: 48103633715200.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.19.self_attn.k_norm | 梯度 Max: 68444598829056.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.19.self_attn.k_proj | 梯度 Max: 54150947667968.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.19.self_attn.q_norm | 梯度 Max: 141836999983104.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.19.self_attn.q_proj | 梯度 Max: 204509162766336.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.15.self_attn | 梯度 Max: 131391639519232.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.15.self_attn.o_proj | 梯度 Max: 131391639519232.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.15.self_attn.v_proj | 梯度 Max: 1549526502191602335744.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.15.self_attn.k_norm | 梯度 Max: 1752440687002407403520.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.15.self_attn.k_proj | 梯度 Max: 2822351843277561397248.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.15.self_attn.q_norm | 梯度 Max: 2600990914393046777856.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.15.self_attn.q_proj | 梯度 Max: 5902958103587056517120.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.11.self_attn | 梯度 Max: 2951479051793528258560.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.11.self_attn.o_proj | 梯度 Max: 2951479051793528258560.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.11.self_attn.v_proj | 梯度 Max: 4584246707978673830485819392.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.11.self_attn.k_norm | 梯度 Max: 7969239002899635519663112192.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.11.self_attn.k_proj | 梯度 Max: 14932651723879899565970685952.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.11.self_attn.q_norm | 梯度 Max: 40852021296417549071671099392.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.11.self_attn.q_proj | 梯度 Max: 41470991316060239209120661504.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.7.self_attn | 梯度 Max: 52612451669628661683212779520.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.7.self_attn.o_proj | 梯度 Max: 52612451669628661683212779520.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.7.self_attn.v_proj | 梯度 Max: 7382797095729208034316799468109824.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.7.self_attn.k_norm | 梯度 Max: 29206669829258405410484041851863040.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.7.self_attn.k_proj | 梯度 Max: 50949412924372996104955495230472192.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.7.self_attn.q_norm | 梯度 Max: 55168154121932543553136523497963520.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.7.self_attn.q_proj | 梯度 Max: 113581493780449354374104607201689600.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.3.self_attn | 梯度 Max: 138893940965806639063190776806637568.00 | Dtype: torch.bfloat16
⚠ [数值爆炸预警] 层: model.layers.3.self_attn.o_proj | 梯度 Max: 138893940965806639063190776806637568.00 | Dtype: torch.bfloat16
……
```

3.Fix
```
CUDA_LAUNCH_BLOCKING=1 CUDA_HOME=/usr/local/cuda-12.8 LD_PRELOAD=/opt/anaconda3/envs/scr_train2/lib/libstdc++.so.6 CUDA_VISIBLE_DEVICES=2 torchrun --nproc_per_node=1 train_dapo_3_debug.py  --use_lora --use_4bit  --use_flash_attn

```

Output:
```
……
[🔍 22:47:55] ✅ Step 1 梯度全部正常 (最大 =0.003845 @ base_model.model.model.layers.3.self_attn.v_proj.lora_B.default.weight)
……
```

### Expected behavior

1. **Avoid Silent Fallbacks to the Math Backend:** When training Qwen3.5 models (or any Qwen2 architecture handling 3D `position_ids`/mRoPE), the `transformers` implementation explicitly materializes a massive 4D Dense Mask (`[Batch, 1, SeqLen, SeqLen]`) and sets `is_causal=False`. This design explicitly violates PyTorch SDPA’s fused kernel constraints (`if (attn_mask.has_value()) { return false; }`), silently forcing a downgrade to the `Math` backend. **Expected behavior:** The implementation should decouple mRoPE coordinate handling from the attention mask generation, preserving the ability to rely on the implicit `is_causal=True` mechanism, which keeps the highly optimized FlashAttention kernel engaged.

2. **Prevent BF16 Precision Collapse in Long-Context RLHF:** The PyTorch `Math` backend is fundamentally unsafe for accumulating Softmax denominators over thousands of tokens (e.g., 8K - 100K) in `bfloat16`. Without the FP32 SRAM accumulators used by fusion kernels, the `Math` backend suffers from severe truncation errors (swamping). Under RLHF losses (like DPO/GRPO/DAPO) which contain exponential amplifiers (`exp(beta * log_probs)`), these errors invariably snowball into catastrophic $10^{28}$ or `NaN` gradients. **Expected behavior:** `transformers` should provide a native `varlen` (variable-length) or `NestedTensors` implementation for `sdpa` that physically truncates padded tokens rather than masking them with `-3.4e38` in a dense tensor, thereby bypassing the mathematically unstable `bfloat16` accumulations.

3. **Explicit Warning or Fallback to FA2:** Until a native SDPA `varlen` solution is implemented, `transformers` should aggressively warn users when `sdpa` is initialized alongside padding masks on models requiring dense mask materialization. Currently, explicitly setting `attn_implementation="flash_attention_2"` is the _only_ mathematically safe approach, as `Qwen2FlashAttention2` uses `cu_seqlens` to physically drop padding and leverages FP32 registers internally, perfectly stabilizing the RLHF gradients.

--- Comment by ouroborosscr at 2026-03-22T17:03:54Z ---
Only flash attention2 has been found for the repair method. As for the repair of transformers, after all, the official is_causal=false, so I have not found a very suitable method.
Flash attention2 needs to use the version I reported（ https://github.com/QwenLM/Qwen3.5/issues/104 https://github.com/ouroborosscr/transformers/tree/fix/qwen35-flash-attn-3d-position-ids and https://github.com/huggingface/transformers/pull/44911 ）Otherwise, an out of bounds error will be reported.

--- Comment by zucchini-nlp at 2026-03-23T10:05:17Z ---
cc @vasqu , we talked about it recently when trying to get rid of that loop over each sequence in qwen's vision. IIRC you already have planned to add SDPA varlen path

https://github.com/huggingface/transformers/blob/55cc1a7fb8e53a5e7e35ca9cf9759498f20abb93/src/transformers/models/qwen2_vl/modeling_qwen2_vl.py#L414-L449

--- Comment by vasqu at 2026-03-25T13:51:10Z ---
Yup, planning to support the sdpa varlen path 🤗

The analysis here is all over the place tho and I'm a bit confused
1. There is no fallback to the math kernel, at worst SDPA would use the memory-efficient (xformers) backend
2. Where are we in the code? It can imo only be the text model because it creates a mask, not the vision model (it does not create a mask as also linked in the portion by @zucchini-nlp). If the position ids are not packed, then the mask will only be created when you compile the forward / mask fn - if it is packed, then we do create a mask as SDPA cannot handle it otherwise for the moment (sdpa varlen will be needed). This really doesn't fall on us but is a fundemantal limitation in the SDPA backend...

This is a 100% written with some coding agent 😅 we need a minimal reproducer, not a full-on custom training script.

TL;DR: SDPA is expected to create a mask to support packed sequences which is a fundamental limitation in that backend. Varlen support is planned but only for torch 2.10+ then


ISSUE #44918: Unpacking Qwen3.5 input embeddings fails with trl SFT trainer
State: closed  |  Labels: bug
Author: JakobJBauer  |  Created: 2026-03-21T23:43:15Z

### System Info

My environment:
```
datasets==4.6.1
faiss-cpu==1.13.2
numpy==2.4.2
pyserini==1.5.0
sentence-transformers==5.2.3
torch==2.10.0
torchvision==0.25.0
tqdm==4.67.3
trl==0.29.1
wandb==0.25.1
```

### Who can help?

@zucchini-nlp

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

This is the code I am using:
```py
config = SFTConfig(
        output_dir=checkpoints_path,
        max_steps=num_steps,
        dataset_text_field="text",
        max_length=2048,
        logging_strategy="steps",
        logging_steps=max(1, checkpoint_frequency // 5),
        disable_tqdm=False,
        save_strategy="steps",
        save_steps=checkpoint_frequency,
        eval_strategy="steps",
        eval_steps=checkpoint_frequency,
        load_best_model_at_end=True,
        metric_for_best_model="eval_loss",
        greater_is_better=False,
        report_to="wandb",
        run_name=run_name,
)

trainer = SFTTrainer(
        model=lm.model,
        args=config,
        train_dataset=train_ds,
        eval_dataset=eval_ds,
)
trainer.train()
trainer.save_model()
```

### Expected behavior

Successful SFT. Instead I get the following error  with a simple qwen3.5 SFT setup.  Running inference only worked fine for me

```
Traceback (most recent call last):
  File "/workspace/writeable/open-ended-csp/tmp.py", line 41, in <module>
    train_sft(
  File "/workspace/writeable/open-ended-csp/utils/sft_trainer.py", line 52, in train_sft
    trainer.train()
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/trainer.py", line 1412, in train
    return inner_training_loop(
           ^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/trainer.py", line 1742, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/trl/trainer/sft_trainer.py", line 1338, in training_step
    return super().training_step(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/trainer.py", line 1951, in training_step
    loss = self.compute_loss(model, inputs, num_items_in_batch=num_items_in_batch)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/trl/trainer/sft_trainer.py", line 1234, in compute_loss
    (loss, outputs) = super().compute_loss(
                      ^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/trainer.py", line 2022, in compute_loss
    outputs = model(**inputs)
              ^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/accelerate/utils/operations.py", line 823, in forward
    return model_forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/accelerate/utils/operations.py", line 811, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/torch/amp/autocast_mode.py", line 44, in decorate_autocast
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/utils/generic.py", line 841, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 1938, in forward
    outputs = self.model(
              ^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/utils/generic.py", line 841, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 1685, in forward
    position_ids = self.compute_3d_position_ids(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/writeable/open-ended-csp/.venv/lib/python3.12/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 1619, in compute_3d_position_ids
    batch_size, seq_length, _ = inputs_embeds.shape
    ^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 3)
wandb:
wandb: 🚀 View run test-qwen3.5 at: https://wandb.ai/jj-uoe/open-ended-csp/runs/1sercncq
wandb: Find logs at: wandb/run-20260321_232655-1sercncq/logs
```

--- Comment by zucchini-nlp at 2026-03-23T10:00:25Z ---
Should be reported in TRL in that case, though the issue might have been resolved in trl main branch. There were a few PRs that didn't make it to release, e.g.

https://github.com/huggingface/transformers/issues/44384#issuecomment-3997014468

--- Comment by JakobJBauer at 2026-03-23T15:08:05Z ---
Opened a ticket on trl that is currently being looked into
[https://github.com/huggingface/trl/issues/5334](https://github.com/huggingface/trl/issues/5334)

--- Comment by zucchini-nlp at 2026-03-23T15:20:11Z ---
Nice, closing this issue then


ISSUE #44913: In GPTNeoXConfig, rotary_pct silently reverts to default on reload
State: closed  |  Labels: bug
Author: ratishsp  |  Created: 2026-03-21T17:58:32Z

### System Info

- `transformers` version: 5.3.0
- Platform: Linux-6.17.0-19-generic-x86_64-with-glibc2.39
- Python version: 3.12.4
- Huggingface_hub version: 1.7.2
- Safetensors version: 0.4.5
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.5.1 (NA)
- Using distributed or parallel set-up in script?: No

When creating a `GPTNeoXConfig` with non-default `rotary_pct`, value is lost after `save_pretrained` / `from_pretrained`.

## Cause

https://github.com/huggingface/transformers/blob/3a3b59cb1a7c0238c8d1072e35d3879c5faff48e/src/transformers/models/gpt_neox/configuration_gpt_neox.py#L98

`save_pretrained` writes `partial_rotary_factor` inside `rope_parameters` but does not persist `rotary_pct` as a top-level key. On reload, `rotary_pct` is absent from kwargs, so this line unconditionally overwrites the correct value with `0.25`.

## Fix

```python
rotary_pct = kwargs.pop("rotary_pct", None)
if rotary_pct is not None:
    self.rope_parameters["partial_rotary_factor"] = rotary_pct
else:
    self.rope_parameters.setdefault("partial_rotary_factor", 0.25)
```

Verified locally after applying this, the value survives the round-trip.

Models using the default `rotary_pct=0.25` (gpt-neox-20b, Pythia, etc.) are unaffected since the overwrite produces the same value.

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
from transformers import GPTNeoXConfig

config = GPTNeoXConfig(rotary_pct=1.0)
print(config.rope_parameters["partial_rotary_factor"])  # 1.0
config.save_pretrained("/tmp/test")

config2 = GPTNeoXConfig.from_pretrained("/tmp/test")
print(config2.rope_parameters["partial_rotary_factor"])  # 0.25
```

### Expected behavior

`partial_rotary_factor` value should be retained

--- Comment by zucchini-nlp at 2026-03-23T10:09:34Z ---
Indeed, we forgot to update models where rope standardization is overriden prob. Can you open a PR to fix GPTNeoX and iirc there are a few more models with the same issue

This pattern can be copied around from general utils:

https://github.com/huggingface/transformers/blob/55cc1a7fb8e53a5e7e35ca9cf9759498f20abb93/src/transformers/modeling_rope_utils.py#L645-L648

--- Comment by Krishnachaitanyakc at 2026-03-25T02:14:38Z ---
Hi, I'd like to work on this. The fix applies the `setdefault` pattern (as referenced by @zucchini-nlp from `modeling_rope_utils.py`) to both `GPTNeoXConfig` and `GPTNeoXJapaneseConfig`, which have the same bug. Opening a PR shortly.

--- Comment by ratishsp at 2026-03-27T09:32:36Z ---
Sorry I got busy. Thanks @Krishnachaitanyakc and @zucchini-nlp!


ISSUE #44912: git-oss-20b will not properly load with MXFP4 quantization and falls back to bf16
State: closed  |  Labels: bug
Author: jottbecr  |  Created: 2026-03-21T17:38:02Z

### System Info

Hi, probably this is related to
https://github.com/huggingface/transformers/issues/42723

I get:
MXFP4 quantization requires Triton and kernels installed: CUDA requires Triton >= 3.4.0, XPU requires Triton >= 3.5.0, we will default to dequantizing the model to bf16

When executing the following code:
pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    token=HF_TOKEN,
    #model_kwargs={"dtype": torch.bfloat16},
    device_map="auto",
)

$ hf env

Copy-and-paste the text below in your GitHub issue.

- huggingface_hub version: 1.7.2
- Platform: Linux-6.8.0-106-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: /scratch/work/ml_ki/llama/huggingface/token
- Has saved token ?: True
- Who am I ?: jottbe
- Configured git credential helpers:
- Installation method: unknown
- httpx: 0.28.1
- hf_xet: 1.4.2
- gradio: N/A
- tensorboard: N/A
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /scratch/work/ml_ki/llama/huggingface/hub
- HF_ASSETS_CACHE: /scratch/work/ml_ki/llama/huggingface/assets
- HF_TOKEN_PATH: /scratch/work/ml_ki/llama/huggingface/token
- HF_STORED_TOKENS_PATH: /scratch/work/ml_ki/llama/huggingface/stored_tokens
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_DISABLE_XET: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10
- HF_XET_HIGH_PERFORMANCE: False

Package versions:
transformers              5.3.0
triton                    3.6.0
nvidia-cublas-cu12        12.8.4.1
nvidia-cuda-cupti-cu12    12.8.90
nvidia-cuda-nvrtc-cu12    12.8.93
nvidia-cuda-runtime-cu12  12.8.90
nvidia-cudnn-cu12         9.10.2.21
nvidia-cufft-cu12         11.3.3.83
nvidia-cufile-cu12        1.13.1.3
nvidia-curand-cu12        10.3.9.90
nvidia-cusolver-cu12      11.7.3.90
nvidia-cusparse-cu12      12.5.8.93
nvidia-cusparselt-cu12    0.7.1
nvidia-nccl-cu12          2.27.5
nvidia-nvjitlink-cu12     12.8.93
nvidia-nvshmem-cu12       3.4.5
nvidia-nvtx-cu12          12.8.90
torch                     2.10.0
torchaudio                2.10.0
torchvision               0.25.0

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    token=HF_TOKEN,
    #model_kwargs={"dtype": torch.bfloat16},
    device_map="auto",
)

### Expected behavior

Should load with 4bit quantization.

--- Comment by sasmita016 at 2026-03-21T18:18:01Z ---
I am willing to check on this, @maintainer could ypu please check and confirm

--- Comment by ArthurZucker at 2026-03-23T08:19:09Z ---
Will check the PR but yeah you need to install `kernels` library if you want to use the kernels for quantization

--- Comment by sasmita016 at 2026-03-23T09:15:39Z ---
@ArthurZucker @Cyrilvallez

 Hi! I had commented earlier that I was interested in working on this issue, so I was a bit surprised to see a PR opened already.

I wanted to understand the expected workflow here—should contributors start working immediately without waiting for assignment/approval? I was holding off assuming I should wait.

Just asking so I can follow the right process going forward. Thanks!

--- Comment by Cyrilvallez at 2026-03-23T14:31:09Z ---
Hey @sasmita016, we do not really engage in any kind of order here, given that you're not the raiser of the initial issue. People raising valid issues will always get priority if they're willing to work on the mentioned issue. Otherwise, we absolutely cannot control which users will or will not open PRs, people are free to do whatever they want. Then we decide based on PR quality.

Also, I want to emphasize that only code agents comment things such as "I am willing to check on this" on random issues that they did not open themselves, so the probability that you're an agent and I'm wasting my time writing this is unfortunately bigger than 95%... 🫠😑

--- Comment by sasmita016 at 2026-03-23T16:58:11Z ---
@Cyrilvallez Thanks for taking time and letting me know . Ok I learned now how it works here.
That said, I’m not a code agent. I commented because I was genuinely interested in contributing to the issue.
I will be looking forward to contribute more now that i know this repo rules. :)


ISSUE #44910: [Bug] Flash Attention crashes with illegal memory access on Qwen3.5 due to 3D position_ids being misinterpreted as packed sequence
State: closed  |  Labels: bug
Author: ouroborosscr  |  Created: 2026-03-21T15:38:54Z

### System Info

# [Bug] Flash Attention crashes with `illegal memory access` on Qwen3.5 due to 3D `position_ids` being misinterpreted as packed sequence

We fixed it in https://github.com/ouroborosscr/transformers/tree/fix/qwen35-flash-attn-3d-position-ids

## Description

When using `attn_implementation="flash_attention_2"` with Qwen3.5 models, all forward passes crash with `CUDA error: an illegal memory access was encountered`. This affects both training and inference.

**Root cause**: Qwen3.5 uses a hybrid architecture (GatedDeltaNet linear attention + standard attention) and passes **3D `position_ids`** with shape `[3, batch_size, seq_len]` (for multi-dimensional rotary embedding). The function `_is_packed_sequence()` in `modeling_flash_attention_utils.py` misinterprets this 3D tensor as a packed sequence indicator, causing `cu_seqlens` to be constructed with 3× the actual token count. Flash Attention then reads beyond the q/k/v tensor boundaries, resulting in an illegal memory access.

## Reproduction

```python
import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-9B",  # or any Qwen3.5 variant
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.bfloat16,
    ),
    torch_dtype=torch.bfloat16,
    device_map={"": 0},
    attn_implementation="flash_attention_2",
)

# This crashes immediately
input_ids = torch.randint(100, 5000, (1, 256), device="cuda")
with torch.no_grad():
    out = model(input_ids=input_ids, use_cache=False)
```

**Error**:
```
torch.AcceleratorError: CUDA error: an illegal memory access was encountered
```

**Traceback** (abbreviated):
```
File "transformers/modeling_flash_attention_utils.py", line 692, in _flash_attention_forward
    out = flash_varlen_fn(
File "flash_attn/flash_attn_interface.py", line 1443, in flash_attn_varlen_func
    return FlashAttnVarlenFunc.apply(
File "flash_attn/flash_attn_interface.py", line 165, in _flash_attn_varlen_forward
    out, softmax_lse, S_dmask, rng_state = flash_attn_gpu.varlen_fwd(
torch.AcceleratorError: CUDA error: an illegal memory access was encountered
```

## Root Cause Analysis

### Qwen3.5 hybrid architecture

Qwen3.5 uses a **mixed attention architecture**: 24 layers of `Qwen3_5GatedDeltaNet` (linear attention) and 8 layers of `Qwen3_5Attention` (standard attention, at layers 3, 7, 11, 15, 19, 23, 27, 31). Only the standard attention layers use flash attention.

Qwen3.5 passes **3D `position_ids`** with shape `[3, batch_size, seq_len]` for its multi-dimensional rotary embedding (3 sets of position indices).

### The bug

In `modeling_flash_attention_utils.py`, the function `_is_packed_sequence()` (line 444) does not handle tensors with more than 2 dimensions:

```python
def _is_packed_sequence(position_ids, batch_size):
    if position_ids is None:
        return False
    increasing_position_sequences = (
        torch.arange(position_ids.shape[1], device=position_ids.device) + position_ids.min()
    )
    return batch_size == 1 and (increasing_position_sequences - position_ids).abs().sum().bool()
```

When `position_ids` has shape `[3, 1, 256]`:
- `position_ids.shape[1]` returns `1` (not `256` as expected for a 2D `[batch, seq_len]` tensor)
- The function returns `True`, misidentifying this as a packed sequence

This triggers the packed-sequence code path at line 677:

```python
elif is_fa_with_varlen_kwargs or is_fa_with_position_ids:
    if cu_seq_lens_q is None or cu_seq_lens_k is None:
        q, k, v, (cu_seq_lens_q, cu_seq_lens_k), (max_length_q, max_length_k) = _prepare_from_posids(
            query_states, key_states, value_states, position_ids
        )
```

Inside `prepare_fa_kwargs_from_position_ids()` (line 362):

```python
position_ids = position_ids.reshape(-1)  # [3, 1, 256] → [768]
indices_q = (position_ids == 0).nonzero().view(-1)  # Finds 3 zero positions
```

This constructs `cu_seqlens = [0, 256, 512, 768]`, claiming 3 sequences with 768 total tokens. But the actual q/k/v tensors only contain 256 tokens. Flash Attention reads up to index 768, causing the illegal memory access.

### Intercepted parameters confirming the mismatch

```
🔍 varlen_fwd parameters:
  q: torch.Size([256, 16, 256])           ← 256 tokens
  cu_seqlens_q: tensor([0, 256, 512, 768]) ← claims 768 tokens
  q total=256 vs cu_seqlens_q[-1]=768      ← MISMATCH → crash
```

## Fix

Add a dimensionality check in `_is_packed_sequence()` to reject tensors with more than 2 dimensions, since packed sequences always use 2D position_ids `[batch, seq_len]`:

```python
def _is_packed_sequence(position_ids, batch_size):
    if position_ids is None:
        return False
    if position_ids.dim() > 2:
        return False
    increasing_position_sequences = (
        torch.arange(position_ids.shape[1], device=position_ids.device) + position_ids.min()
    )
    return batch_size == 1 and (increasing_position_sequences - position_ids).abs().sum().bool()
```

This fix has been validated: all 8 standard attention layers in Qwen3.5-9B pass flash attention forward successfully after applying the patch.

## Environment

- **Model**: Qwen3.5-9B (hybrid GatedDeltaNet + standard attention)
- **GPU**: NVIDIA A100-SXM4-80GB
- **PyTorch**: 2.9.0 / 2.10.0 (both affected)
- **Transformers**: 5.3.0
- **flash-attn**: 2.8.3
- **CUDA**: 12.8

## Impact

- Affects **all Qwen3.5 variants** (and potentially any future model using >2D position_ids)
- Blocks both training and inference when using `flash_attention_2`
- No workaround other than falling back to `sdpa` or `eager` attention implementations

### Who can help?

@vasqu @ArthurZucker (attention)

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

GRPO reinforcement learning training with Qwen3.5-9B using TRL GRPOTrainer
When using `attn_implementation="flash_attention_2"` with Qwen3.5, all forward passes crash with CUDA illegal memory access. Minimal reproduction:
```python
import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-9B",
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.bfloat16,
    ),
    torch_dtype=torch.bfloat16,
    device_map={"": 0},
    attn_implementation="flash_attention_2",
)

input_ids = torch.randint(100, 5000, (1, 256), device="cuda")
with torch.no_grad():
    out = model(input_ids=input_ids, use_cache=False)  # crashes here
```

**Error:**
```
torch.AcceleratorError: CUDA error: an illegal memory access was encountered
```

**Root cause:**

Qwen3.5 is a hybrid architecture (24 GatedDeltaNet layers + 8 standard attention layers). It uses 3D `position_ids` with shape `[3, batch_size, seq_len]` for multi-dimensional rotary embedding.

`_is_packed_sequence()` in `modeling_flash_attention_utils.py` (line 444) does not handle >2D tensors:
```python
def _is_packed_sequence(position_ids, batch_size):
    if position_ids is None:
        return False
    increasing_position_sequences = (
        torch.arange(position_ids.shape[1], device=position_ids.device) + position_ids.min()
    )
    return batch_size == 1 and (increasing_position_sequences - position_ids).abs().sum().bool()
```

When `position_ids` has shape `[3, 1, 256]`, `position_ids.shape[1]` returns `1` instead of the sequence length, and the function returns `True`, misidentifying this as a packed sequence.

This triggers `prepare_fa_kwargs_from_position_ids()` which does:
```python
position_ids = position_ids.reshape(-1)  # [3, 1, 256] → [768]
indices_q = (position_ids == 0).nonzero()  # finds 3 zero positions
# constructs cu_seqlens = [0, 256, 512, 768] — claims 768 tokens
```

But q/k/v only contain 256 tokens. Flash attention reads up to index 768, causing the illegal memory access.

**Intercepted evidence:**
```
q: torch.Size([256, 16, 256])           ← 256 tokens
cu_seqlens_q: tensor([0, 256, 512, 768]) ← claims 768 tokens
q total=256 vs cu_seqlens_q[-1]=768      ← MISMATCH → crash
```

**Environment:**
- Model: Qwen3.5-9B
- GPU: NVIDIA A100-SXM4-80GB
- PyTorch: 2.9.0 and 2.10.0 (both affected)
- transformers: 5.3.0
- flash-attn: 2.8.3
- CUDA: 12.8

**Fix:** Add `if position_ids.dim() > 2: return False` in `_is_packed_sequence()`. Packed sequences always use 2D `[batch, seq_len]` position_ids.

### Expected behavior

Model forward pass with `attn_implementation="flash_attention_2"` should complete successfully without CUDA errors. After the fix (adding dimensionality check for >2D position_ids), all 8 standard attention layers in Qwen3.5-9B pass flash attention forward correctly.

--- Comment by zucchini-nlp at 2026-03-23T10:12:29Z ---
We recently fixed qwen3.5 to pass over text-position-ids inside an LM module, which didn't yet make it to a release. Prob you have to install from source for now

https://github.com/huggingface/transformers/blob/a8683756653094e3fc3016df8abcdeaaec758f9a/src/transformers/models/qwen3_5/modeling_qwen3_5.py#L1362-L1370

--- Comment by lucasjinreal at 2026-03-24T05:57:18Z ---
@zucchini-nlp  Same error,

```
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/qwen35/lib/python3.10/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 592, in forward
    core_attn_out, last_recurrent_state = self.chunk_gated_delta_rule(
  File "/root/miniconda3/envs/qwen35/lib/python3.10/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 375, in torch_chunk_gated_delta_rule
    torch.zeros(batch_size, num_heads, k_head_dim, v_head_dim).to(value)
torch.AcceleratorError: CUDA error: an illegal memory access was encountered
Search for `cudaErrorIllegalAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
```

Is there any solution? My situation is training is OK, but inference error.....

--- Comment by ouroborosscr at 2026-03-24T07:53:49Z ---
@lucasjinreal
Using the latest version of transformers in GitHub should solve this problem. It seems that the author has not pushed this version to pipy.
Reference:https://github.com/huggingface/transformers/pull/44911
Or you can use the temporary patch I wrote.

--- Comment by ouroborosscr at 2026-03-24T07:57:07Z ---
About the reasoning error, you can see whether it is related to the gradient explosion problem I reported: https://github.com/ouroborosscr/Report-the-gradient-explosion-of-qwen3.5

--- Comment by zucchini-nlp at 2026-03-24T14:36:36Z ---
Same here @ouroborosscr , I will close it. Anyone who has the same issue just need to install from source until the next release :)

--- Comment by lucasjinreal at 2026-03-25T02:03:24Z ---
@zucchini-nlp it works.


ISSUE #44908: inverse_sqrt scheduler ignores lr_scheduler_kwargs (timescale not passed)
State: closed  |  Labels: bug
Author: magarwal0205  |  Created: 2026-03-21T06:53:50Z

### System Info

Incomplete arguments passed for schedulers where name is explicitly checked.

https://github.com/huggingface/transformers/blob/v5.3.0/src/transformers/optimization.py#L664

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Trigger a learning job with "inverse_sqrt" and set timesteps = 'random', it still works because the arguments are never passed.

### Expected behavior

timesteps should take effect.

--- Comment by r266-tech at 2026-03-22T17:31:03Z ---
I've submitted a fix for this in #44932. The issue was that get_scheduler was not forwarding scheduler_specific_kwargs to get_inverse_sqrt_schedule, so the timescale parameter was silently ignored.


ISSUE #44906: Remove unnecessary `expand_as` in `get_placeholder_mask` across VLMs
State: open  |  Labels: Feature request
Author: syncdoth  |  Created: 2026-03-21T06:05:36Z

### Feature request

## Problem

The `get_placeholder_mask` function (and equivalent inline patterns) across ~70 multimodal model files expands a boolean placeholder mask from shape `(B, S, 1)` to `(B, S, H)` via `.expand_as(inputs_embeds)` before passing it to `masked_scatter`. This expansion is unnecessary because `masked_scatter` natively supports broadcasting.

For example, in `modeling_llava.py`:
```python
special_image_mask = special_image_mask.unsqueeze(-1).expand_as(inputs_embeds).to(inputs_embeds.device)
```

The validation check also uses data-dependent boolean indexing:
```python
inputs_embeds[special_image_mask].numel() == image_features.numel()
```

### Motivation

## Motivation

Mainly, Memory. While `expand_as` itself returns a stride-0 view (no copy), the subsequent `.to(device)` call materializes the full `(B, S, H)` boolean tensor when a device transfer is needed. Practically, I assume that the need for actual device transfer is quite rare; however, I believe this is still a safer implementation.

### Your contribution

## Proposed fix

1. Remove `.expand_as(inputs_embeds)`, keeping the mask as `(B, S, 1)`:
```python
special_image_mask = special_image_mask.unsqueeze(-1).to(inputs_embeds.device)
```

2. Replace the validation with equivalent arithmetic:
```python
n_image_tokens * inputs_embeds.shape[-1] == image_features.numel()
```

## Correctness

`masked_scatter`, `torch.where`, and element-wise `*` all support broadcasting a `(B, S, 1)` mask to `(B, S, H)`. Verified with:

```python
import torch
B, S, H = 2, 10, 16
inputs_embeds = torch.randn(B, S, H)
features = torch.randn(3, H)
mask_2d = torch.zeros(B, S, dtype=torch.bool)
mask_2d[0, 2], mask_2d[0, 5], mask_2d[1, 3] = True, True, True

mask_old = mask_2d.unsqueeze(-1).expand_as(inputs_embeds)
mask_new = mask_2d.unsqueeze(-1)

result_old = inputs_embeds.clone().masked_scatter(mask_old, features)
result_new = inputs_embeds.clone().masked_scatter(mask_new, features)
assert torch.equal(result_old, result_new)  # ✓ identical
```

Affects `get_placeholder_mask` and equivalent inline patterns in ~70 files across all VLM models (llava, qwen2_vl, paligemma, gemma3n, chameleon, video_llava, etc.), plus `tensor_parallel.py` and `ovis2`.

I have a PR ready: https://github.com/syncdoth/transformers/tree/remove-expand-as-placeholder-mask


ISSUE #44898: [BUG] Perceiver image classification (non-default res) fails even with interpolate_pos_encoding=True
State: closed  |  Labels: bug
Author: harshaljanjani  |  Created: 2026-03-20T19:58:09Z

### System Info

* `transformers` version: `5.0.0.dev0`
* Platform: `Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39`
* Python version: `3.12.3`
* `huggingface_hub` version: `1.3.2`
* `safetensors` version: `0.7.0`
* `accelerate` version: `1.12.0`
* Accelerate config: `not installed`
* DeepSpeed version: `not installed`
* PyTorch version (accelerator?): `2.9.1+cu128 (CUDA)`
* GPU type: `NVIDIA L4`
* NVIDIA driver version: `550.90.07`
* CUDA version: `12.4`

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import torch
from transformers import PerceiverForImageClassificationLearned, PerceiverImageProcessorPil
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
image_processor = PerceiverImageProcessorPil(size={"height": 384, "width": 384})
model = PerceiverForImageClassificationLearned.from_pretrained("deepmind/vision-perceiver-learned")
model.eval()
inputs = image_processor(image, return_tensors="pt").pixel_values
try:
    with torch.no_grad():
        outputs = model(inputs=inputs, interpolate_pos_encoding=True)
    print("Logits shape:", outputs.logits.shape)
    predicted_class = outputs.logits.argmax(-1).item()
    print("Predicted class:", predicted_class)
except Exception as e:
    print(e)
```

→ Trying to run image classification on a 384×384 image (pretrained default is 224×224) and even after setting `interpolate_pos_encoding=True` expecting the model to handle the resolution difference, the model crashes with a `RuntimeError`.
→ From the screenshot, 384×384 = 147456 and 224×224 = 50176 so it was never actually resized (see the reproduction output).

**Current Repro Output:**

<img alt="Image" src="https://github.com/user-attachments/assets/3f1ac00d-5f36-4d3b-be2a-21f46accd0bb" />

### Expected behavior

→ Inference should complete successfully (torch.Size([1, 1000])) when `interpolate_pos_encoding=True` is passed with non-native input res.


ISSUE #44877: Strict config prevents loading `granite_speech` config
State: closed  |  Labels: bug
Author: tomaarsen  |  Created: 2026-03-20T09:57:02Z

### System Info

- `transformers` version: 5.3.0.dev0
- Platform: Windows-10-10.0.26200-SP0
- Python version: 3.11.13
- Huggingface_hub version: 1.7.1
- Safetensors version: 0.6.2
- Accelerate version: 1.11.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0+cu126 (CUDA)
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: NVIDIA GeForce RTX 3090

### Who can help?

@zucchini-nlp

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoConfig

model_name = "ibm-granite/granite-4.0-1b-speech"
config = AutoConfig.from_pretrained(model_name)
```
```
Traceback (most recent call last):
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 148, in __strict_setattr__
    validator(value)
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 607, in validator
    type_validator(field.name, value, field.type)
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 474, in type_validator
    _validate_simple_type(name, value, expected_type)
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 597, in _validate_simple_type
    raise TypeError(
TypeError: Field 'embedding_multiplier' expected float, got int (value: 12)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[sic]/demo_granite_4_1b_speech.py", line 4, in <module>
    config = AutoConfig.from_pretrained(model_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/src/transformers/models/auto/configuration_auto.py", line 1486, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/src/transformers/configuration_utils.py", line 757, in from_dict
    config = cls(**config_dict)
             ^^^^^^^^^^^^^^^^^^
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 279, in init_with_validate
    initial_init(self, *args, **kwargs)  # type: ignore [call-arg]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 194, in __init__
    self.__post_init__(**additional_kwargs)
  File "[sic]/src/transformers/models/granite_speech/configuration_granite_speech.py", line 122, in __post_init__
    self.text_config = CONFIG_MAPPING[self.text_config["model_type"]](**self.text_config)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 279, in init_with_validate
    initial_init(self, *args, **kwargs)  # type: ignore [call-arg]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 179, in __init__
    setattr(self, f.name, standard_kwargs[f.name])
  File "[sic]/Lib/site-packages/huggingface_hub/dataclasses.py", line 150, in __strict_setattr__
    raise StrictDataclassFieldValidationError(field=name, cause=e) from e
huggingface_hub.errors.StrictDataclassFieldValidationError: Validation error for field 'embedding_multiplier':
    TypeError: Field 'embedding_multiplier' expected float, got int (value: 12)
```

### Expected behavior

For the config to load as normal.

- Tom Aarsen

--- Comment by tomaarsen at 2026-03-20T10:01:14Z ---
This is the parameter causing the issue: https://github.com/huggingface/transformers/blob/2cd52c267ce4d0212eaa40c0ec7192a11654336f/src/transformers/models/granite/configuration_granite.py#L87

- Tom Aarsen

--- Comment by zucchini-nlp at 2026-03-20T10:12:16Z ---
do you have a PR by chance 😅

or we can add it in https://github.com/huggingface/transformers/pull/44874, we're filing a general type hint fix based on vLLM CI jobs

--- Comment by tomaarsen at 2026-03-20T10:19:17Z ---
I'm not sure how you want this to be fixed. I'm personally not the biggest fan of strict dataclasses, I feel like they result in unnecessary errors more often than they help with finding misconfigurations.

- Tom Aarsen

--- Comment by zucchini-nlp at 2026-03-20T10:47:03Z ---
I had a lot of issues with these types of args as well (dropout and multipliers) which could be saved as `int` or `float` in the hub. I think we would need to allow `int` and update the typing hint

> I feel like they result in unnecessary errors more often than they help with finding misconfigurations

It definitely results in a lot of errors since we didn't constrict config params to certain types. I think it's a good opportunity for us to consolidate configs and to keep correct type hints in documentation.


ISSUE #44871: [Gemma-3] Inconsistent eos_token_id configuration: tokenizer has single value (1) but model.config has list [1, 106]
State: closed  |  Labels: bug
Author: IvanFan-Van  |  Created: 2026-03-20T04:29:57Z

### System Info

- `transformers` version: 5.3.0
- Platform: Windows-11-10.0.26100-SP0
- Python version: 3.12.11
- Huggingface_hub version: 1.7.1
- Safetensors version: 0.7.0
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.7.1+cu118 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 4060 Laptop GPU

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL = "google/gemma-3-1b-it"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL)
print(tokenizer.eos_token_id) # 1
print(model.config.eos_token_id) # [1, 106]
```

### Expected behavior

## Expected Behavior

Tokenizer and Model Config should be consistent on eos_token_id so that the following generation loop can work correctly:
```python
for _ in range(max_new_tokens):
    ...
    if new_token_id.item() == tokenizer.eos_token_id:
        break  # should break the loop
```

--- Comment by ashwin3005 at 2026-03-20T09:56:38Z ---
This is expected behavior, not a bug.

### We have two EOS tokens for instruction-tuned models

- `token 1` is `<eos>`  :  the standard end-of-sequence token.
- `token 106` is `<end_of_turn>` :  a special token added by the chat template for instruction-tuned models

### Why we have two EOS tokens?

  During model conversion ([convert_gemma3_weights.py](https://github.com/huggingface/transformers/blob/aa1c36f1a9f454e69c4eac83071ced235942c7ed/src/transformers/models/gemma3/convert_gemma3_weights.py):608-611),

when a chat template is included (for instruction-tuned (ends with '-it' variants), the config is explicitly set to:
  ```python
# Chat template is included for instruction tuned models, which treat
# both "<eos>" and "<end_of_turn>" as generation stoppers.
config.eos_token_id = [1, 106]
  ```
  So google/gemma-3-1b-it's config gets [1, 106] because the model should stop on either token. The tokenizer only
  exposes a single eos_token_id because it doesn't know about chat-template-level stop tokens.

so, when you do,
```python
if new_token_id.item() == tokenizer.eos_token_id:  # only checks token 1
    break
 ```
you miss the <end_of_turn> (106).

The correct check when doing manual generation loops is check against the model's config (handles all stop tokens)

```python
eos_ids = model.config.eos_token_id
if isinstance(eos_ids, int):
    eos_ids = [eos_ids]
if new_token_id.item() in eos_ids:
    break
```

  Or you can simply use model.generate() which handles this automatically.

  TL;DR

| | Value | reason |
|---|---|---|
|tokenizer.eos_token_id | 1 |Only knows about <eos> |
| model.config.eos_token_id | [1, 106] |Includes <end_of_turn> for chat models |


ISSUE #44869: Whisper word timestamp decode crashes on trailing replacement character at end of decoded token stream
State: closed  |  Labels: bug
Author: chromatic-descension  |  Created: 2026-03-20T01:25:49Z

### System Info

### System Info

- OS: macOS
- `transformers`: `5.3.0.dev0`
- Model: `openai/whisper-medium.en`

### Reproduction

I hit an `IndexError: string index out of range` in Whisper word-timestamp decoding and traced it to `src/transformers/models/whisper/tokenization_whisper.py`.

The failing code path is in `_split_tokens_on_unicode()`:

```py
decoded_full[unicode_offset + decoded.index(replacement_char)]
```

The bug happens when the decoded token stream ends with a dangling Unicode replacement character (`�`, `U+FFFD`). In that case, the computed index can equal `len(decoded_full)`, so the code reads one past the end of the string and crashes.

For the failing case I traced locally, the values were:
- `unicode_offset = 298`
- `decoded.index(replacement_char) = 0`
- `target_index = 298`
- `len(decoded_full) = 298`

So the effective access becomes:

```py
decoded_full[298]
```

but the last valid index is `297`.

The underlying ASR output for the bad chunk decoded to a long run of musical note symbols followed by a dangling final replacement character (...🎵 🎵 🎵 🎵 🎵 �). Segment-level decoding succeeded, but word-level timestamp collation crashed in `_split_tokens_on_unicode()`.

### Error

```text
IndexError: string index out of range
```

### Expected behavior

- trailing incomplete Unicode fragments at EOF should be ignored or handled safely
- Whisper word timestamp decoding should not crash with `IndexError`

### Additional context

I have a local fix prepared for this EOF bounds case and can open a PR if this approach looks reasonable.

### Who can help?

@ArthurZucker @itazap

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Full end to end reproduction would involve the original audio file (2m 14s of music with some vocals), but the underlying problem is simpler and can be reproduced by calling the `_split_tokens_on_unicode` method with data that could reasonably be outputted.

```python
from collections import defaultdict
from transformers.models.whisper.tokenization_whisper import _split_tokens_on_unicode

class DummyTokenizer:
    def __init__(self):
        self.responses = defaultdict(list)

    def decode(self, tokens, decode_with_timestamps=False):
        key = tuple(tokens)
        if self.responses[key]:
            return self.responses[key].pop(0)

tokenizer = DummyTokenizer()
tokenizer.responses[(1, 2)] = ["ab"]   # decoded_full
tokenizer.responses[(1,)] = ["ab"]     # first token decodes cleanly
tokenizer.responses[(2,)] = ["�"]      # trailing replacement char at EOF

print(_split_tokens_on_unicode(tokenizer, [1, 2]))
```

Before the fix, this raises:

```text
IndexError: string index out of range
```

Because it tries to read `decoded_full[2]` when `len(decoded_full) == 2`.

### Expected behavior

Whisper word-timestamp decoding should safely ignore or stop on a trailing incomplete Unicode fragment at end-of-string, instead of crashing with `IndexError: string index out of range`.

--- Comment by howardpen9 at 2026-03-20T13:04:49Z ---
Hi, I'd like to work on this. The fix is a bounds check in `_split_tokens_on_unicode()` — pre-computing `target_index` and guarding against out-of-bounds access when a trailing replacement character lands at the end of `decoded_full`. I'll submit a PR shortly. (AI-assisted drafting, human-reviewed.)

--- Comment by Jesteinbe at 2026-04-13T16:22:50Z ---
Is there any news on this fix? I'm also hitting this issue but it looks like the PR was closed without merging.

--- Comment by itazap at 2026-04-15T13:47:57Z ---
opened a PR, thanks for catching!


ISSUE #44868: [rag-end2end-retriever] Broken Google Drive link for SQuAD dataset and hyperparameters
State: closed  |  Labels:
Author: lmmanriquem  |  Created: 2026-03-19T22:42:17Z

## Description

The README for the `rag-end2end-retriever` research project contains a broken
Google Drive link to the SQuAD training dataset, knowledge base, and
hyperparameters used in the experiments.

**Location:**
https://github.com/huggingface/transformers-research-projects/tree/main/rag-end2end-retriever

**Broken link text:**
> Training dataset, the knowledge-base, and hyperparameters used in experiments
> can be accessed from [here](https://drive.google.com/drive/folders/1qyzV-PaEARWvaU_jjpnU_NUS3U_dSjtG?usp=sharing).

**Issue:** The Google Drive folder returns "Sorry, the file you have requested
does not exist" (404).

This link is essential for anyone trying to reproduce the results reported in:
> Siriwardhana et al., *Improving the Domain Adaptation of Retrieval Augmented
> Generation (RAG) Models for Open Domain Question Answering*, TACL 2023.
> https://aclanthology.org/2023.tacl-1.1/

Could @shamanez please restore or update the link?

Thank you.

--- Comment by github-actions[bot] at 2026-04-19T08:16:50Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by Rocketknight1 at 2026-04-20T14:37:43Z ---
Closing with https://github.com/huggingface/transformers-research-projects/pull/5


ISSUE #44863: NemotronH implementation can't load NemotronH checkpoints!
State: closed  |  Labels: bug
Author: elfprince13  |  Created: 2026-03-19T15:15:06Z

### System Info

```console
- `transformers` version: 5.3.0
- Platform: macOS-15.7.3-arm64-arm-64bit
- Python version: 3.12.11
- Huggingface_hub version: 1.7.1
- Safetensors version: 0.7.0
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0 (NA)
- Using distributed or parallel set-up in script?: NO
```

### Who can help?

@ArthurZucker @Cyrilvallez @liding-nv

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

either of:

```python
tokenizer  = AutoTokenizer.from_pretrained("nvidia/Nemotron-H-8B-Reasoning-128K")
model = NemotronHForCausalLM.from_pretrained("nvidia/Nemotron-H-8B-Reasoning-128K", torch_dtype=torch.bfloat16)
```

or

```python
tokenizer  = AutoTokenizer.from_pretrained("nvidia/Nemotron-H-4B-Instruct-128K")
model = NemotronHForCausalLM.from_pretrained("nvidia/Nemotron-H-4B-Instruct-128K", torch_dtype=torch.bfloat16)
```

fails. The first error appears to be that these functions:
https://github.com/huggingface/transformers/blob/884333368ff329090c73bd00e57996727f301de3/src/transformers/models/nemotron_h/configuration_nemotron_h.py#L259-L269

do not correctly handle the `-` character in layer type patterns, but the saved models on the hub use them.

If I add a check to ignore the `-` character I instead see something like this:
```console
num_hidden_layers (52) is deprecated and doesn't match layers_block_type length (28). Using layers_block_type length.
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
num_hidden_layers (52) is deprecated and doesn't match layers_block_type length (28). Using layers_block_type length.
Fetching 4 files: 100%|██████████| 4/4 [05:01<00:00, 75.32s/it]
The fast path is not available because one of `(selective_state_update, causal_conv1d_fn, causal_conv1d_update)` is None. Falling back to the naive implementation. To install follow https://github.com/state-spaces/mamba/#installation and https://github.com/Dao-AILab/causal-conv1d
Loading weights: 100%|██████████| 119/119 [00:00<00:00, 26544.82it/s]
NemotronHForCausalLM LOAD REPORT from: nvidia/Nemotron-H-8B-Reasoning-128K
Key                                              | Status     |
-------------------------------------------------+------------+-
model.layers.{4...50}.mixer.A_log                | UNEXPECTED |
model.layers.{1...51}.mixer.down_proj.weight     | UNEXPECTED |
model.layers.{4...50}.mixer.norm.weight          | UNEXPECTED |
model.layers.{7, 18, 29, 40}.mixer.v_proj.weight | UNEXPECTED |
model.layers.{4...50}.mixer.out_proj.weight      | UNEXPECTED |
model.layers.{4...50}.mixer.in_proj.weight       | UNEXPECTED |
model.layers.{28...51}.norm.weight               | UNEXPECTED |
model.layers.{7, 18, 29, 40}.mixer.o_proj.weight | UNEXPECTED |
model.layers.{4...50}.mixer.dt_bias              | UNEXPECTED |
model.layers.{1...51}.mixer.up_proj.weight       | UNEXPECTED |
model.layers.{7, 18, 29, 40}.mixer.k_proj.weight | UNEXPECTED |
model.layers.{4...50}.mixer.D                    | UNEXPECTED |
model.layers.{4...50}.mixer.conv1d.bias          | UNEXPECTED |
model.layers.{7, 18, 29, 40}.mixer.q_proj.weight | UNEXPECTED |
model.layers.{4...50}.mixer.conv1d.weight        | UNEXPECTED |
model.layers.{1...27}.mixer.out_proj.weight      | MISSING    |
model.layers.{1...27}.mixer.norm.weight          | MISSING    |
model.layers.{1...27}.mixer.A_log                | MISSING    |
model.layers.{1...27}.mixer.dt_bias              | MISSING    |
model.layers.{4, 10, 16, 22}.mixer.o_proj.weight | MISSING    |
model.layers.{1...27}.mixer.conv1d.bias          | MISSING    |
model.layers.{4, 10, 16, 22}.mixer.k_proj.weight | MISSING    |
model.layers.{1...27}.mixer.in_proj.weight       | MISSING    |
model.layers.{1...27}.mixer.D                    | MISSING    |
model.layers.{1...27}.mixer.conv1d.weight        | MISSING    |
model.layers.{4, 10, 16, 22}.mixer.q_proj.weight | MISSING    |
model.layers.{4, 10, 16, 22}.mixer.v_proj.weight | MISSING    |

Notes:
- UNEXPECTED    :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
- MISSING    :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.
```

and the model then crashes when calling `model.generate`:

```console
Traceback (most recent call last):
  File "/Users/thomas/Documents/GitHub/local-lm/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 275, in __getattr__
    return self.data[item]
           ~~~~~~~~~^^^^^^
KeyError: 'shape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/main.py", line 17, in <module>
    outputs = model.generate(tokenized_chat)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/path/.venv/lib/python3.12/site-packages/transformers/generation/utils.py", line 2390, in generate
    batch_size = inputs_tensor.shape[0]
                 ^^^^^^^^^^^^^^^^^^^
  File "/path/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 277, in __getattr__
    raise AttributeError
AttributeError
```

### Expected behavior

The model should output a prediction instead of crashing.

--- Comment by Rocketknight1 at 2026-03-20T13:02:32Z ---
cc @zucchini-nlp this bug first appears in #41250. Before that PR, though, the tokenizer loads with an error:

```
You are using a model of type `nemotron_h` to instantiate a model of type ``. This may be expected if you are loading a checkpoint that shares a subset of the architecture (e.g., loading a `sam2_video` checkpoint into `Sam2Model`), but is otherwise not supported and can yield errors. Please verify that the checkpoint is compatible with the model you are instantiating.
```

Maybe the checkpoint is misconfigured?

--- Comment by zucchini-nlp at 2026-03-20T13:09:56Z ---
~~Huh, another one related to new python I guess. I have had python 3.10 for a long time, but started seeing errors today with users who have python 3.14 related to config/model attributes I ran this script and it works with 3.10 and fails with 3.14. I want to check 3.12 for completeness haha. Could be smth from 3.12 on, if we judge based on a few similar issues I am seeing~~

Oke, seems like it is just failing even before the config refactor, so I believe hub config isn't correct. And I am not sure if the 'nemotron-H' is supported because the PR adding model was tested with 'nemotron-v3'  apparently

--- Comment by elfprince13 at 2026-03-20T17:18:19Z ---
I think there are a few layers of issues here, but yeah I was digging in more and it seems like the "nemotron h" implementation doesn't actually match the nemotron-h architecture. 🤔

--- Comment by ArthurZucker at 2026-03-23T08:25:41Z ---
Mmmm  cc @liding-nv if you want to have a look! I think you wanted this one to be compatible no?

--- Comment by liding-nv at 2026-03-27T23:48:04Z ---
@ArthurZucker yes I think we want to make it compatible to model that has dense MLP layer (specified by `-` pattern).
we initially added this in the modeling code, but removed it during the review process because we thought it only support the latest MoE model (aka nemotron-3-super).
I will create a PR to support that as well...

--- Comment by janimo at 2026-03-28T11:20:31Z ---
#44763 is related

--- Comment by ArthurZucker at 2026-03-29T18:49:56Z ---
Answered on the PR ! 🤗

--- Comment by github-actions[bot] at 2026-04-23T08:31:54Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by ArthurZucker at 2026-04-24T05:48:37Z ---
This is fixed

--- Comment by elfprince13 at 2026-04-24T13:16:57Z ---
Awesome, thank you!--
~Thomas


ISSUE #44861: _get_tied_weight_keys crashes with AttributeError when _tied_weights_keys is a list
State: closed  |  Labels:
Author: gh-wf  |  Created: 2026-03-19T15:13:12Z

 ### System Info

  - `transformers` version: 5.3.0
  - Platform: Linux
  - Python version: 3.13

  ### Who can help?

  @Cyrilvallez

  ### Information

  - [ ] The official example scripts
  - [x] My own modified scripts

  ### Reproduction

  Full finetune of `NVIDIA-Nemotron-3-Nano-4B` crashes at checkpoint save with:

  File "transformers/modeling_utils.py", line 341, in _get_tied_weight_keys
      tied_weight_keys.extend([f"{name}.{k}" if name else k for k in tied.keys()])
  AttributeError: 'list' object has no attribute 'keys'

  The Nemotron-H model defines `_tied_weights_keys` as a list, but `_get_tied_weight_keys` assumes it is always a dict
  and calls `.keys()` on it.

  ### Expected behavior

  `save_pretrained` should handle both list and dict types for `_tied_weights_keys` without crashing.


ISSUE #44857: LwDetrImageLoss crashes when using float16 AMP and Cuda
State: closed  |  Labels: bug
Author: m-matthias  |  Created: 2026-03-19T12:56:22Z

### System Info

- `transformers` version: 5.3.0.dev0
- Platform: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 1.7.1
- Safetensors version: 0.7.0
- Accelerate version: 1.13.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0+cu128 (CUDA)
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA RTX 500 Ada Generation Laptop GPU

### Who can help?

@yonigozlan @molbap @sbucaille

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```
import torch

from transformers import LwDetrForObjectDetection

model = LwDetrForObjectDetection.from_pretrained("AnnaZhang/lwdetr_small_60e_coco")
model.train()
model.cuda()

inputs = {
    "pixel_values": torch.rand((1, 3, 640, 640), device="cuda"),
    "pixel_mask": torch.ones((1, 640, 640), device="cuda"),
    "labels": [
        {
            "class_labels": torch.tensor([1], device="cuda"),
            "boxes": torch.tensor([[0.5, 0.5, 0.1, 0.1]], device="cuda"),
        }
    ],
}

with torch.autocast(device_type="cuda", dtype=torch.float16):
    outputs = model(**inputs)
```

This leads to the following error:
```
".../transformers/loss/loss_lw_detr.py", line 139, in loss_labels
    pos_weights[pos_ind] = t
    ~~~~~~~~~~~^^^^^^^^^
RuntimeError: Index put requires the source and destination dtypes match, got Half for the destination and Float for the source.
```

It seems the reason for this is that the torch.pow operations inside loss_labels() cause an upcast to float32 on the GPU, leading to a type mismatch.

### Expected behavior

Correct handling of float16 AMP on Cuda GPU, without crashing.

--- Comment by sbucaille at 2026-03-19T13:58:39Z ---
Hey @m-matthias , thanks for bringing up this bug, I'll have a check today

--- Comment by m-matthias at 2026-03-19T19:42:16Z ---
If it's helpful, I could also try to fix it and submit a PR? Should be just a minor change I believe.

--- Comment by sbucaille at 2026-03-19T20:03:58Z ---
Sure you can definitely submit a PR, ping @yonigozlan once it is done


ISSUE #44855: IndentationError when importing DebertaV2Model on Python 3.13 - @torch.jit.script fails to parse function with comment between decorator and def
State: closed  |  Labels: bug
Author: MNIKIEMA  |  Created: 2026-03-19T11:07:31Z

## Description

Importing `DebertaV2Model` from `transformers` (or any library that depends on it, such as `gliner`) raises an `IndentationError` on Python 3.13. The error originates in `torch.jit.script` when it attempts to re-parse the source of a JIT-scripted function that has a comment placed between the `@torch.jit.script` decorator and the `def` statement.

## Root Cause

In `modeling_deberta_v2.py`, several functions are decorated with `@torch.jit.script` with a comment in between:

```python
@torch.jit.script
# Copied from transformers.models.deberta.modeling_deberta.c2p_dynamic_expand
def c2p_dynamic_expand(c2p_pos, query_layer, relative_pos):
    return c2p_pos.expand([...])
```

When `@torch.jit.script` is applied, PyTorch's JIT frontend (`torch/_sources.py`) calls `inspect.getsource()` on the function, dedents the source, and passes it to `ast.parse()`. On Python 3.13, the comment between the decorator and `def` is included in the extracted snippet. After dedenting, the snippet becomes:

```
@torch.jit.script
# Copied from ...
def c2p_dynamic_expand(...):
    return ...
```

Python 3.13's `ast.parse` (backed by a stricter parser) then fails with `IndentationError: expected an indented block after function definition on line 3`, seemingly not associating the return statement as the function body when a comment precedes the `def` in this context.

## Affected Functions

All in `transformers/models/deberta_v2/modeling_deberta_v2.py`:

- `c2p_dynamic_expand` (line 104)
- `p2c_dynamic_expand` (line 110)
- `pos_dynamic_expand` (line 116)

## Workaround

Remove the comments between `@torch.jit.script` and each `def`:

```python
# Copied from transformers.models.deberta.modeling_deberta.c2p_dynamic_expand
@torch.jit.script
def c2p_dynamic_expand(c2p_pos, query_layer, relative_pos):
    return c2p_pos.expand([...])
```

## Related Issues

- [pytorch/pytorch #165238](https://github.com/pytorch/pytorch/issues/165238)
- [pytorch/pytorch #65452](https://github.com/pytorch/pytorch/issues/65452)
- [pytorch/pytorch #25043](https://github.com/pytorch/pytorch/issues/25043)
- [huggingface/transformers #35443](https://github.com/huggingface/transformers/issues/35443)
- [huggingface/transformers #34190](https://github.com/huggingface/transformers/issues/34190)
- [huggingface/trl #4239](https://github.com/huggingface/trl/issues/4239)

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

## Environment

| | |
|---|---|
| Python | 3.13.8 |
| torch | 2.10.0+cu128 |
| transformers | 5.3.0 |
| gliner | 0.2.24 |
| OS | Ubuntu 24.04.4 LTS (Noble Numbat), kernel 6.17.0-19-generic, x86_64|

## To Reproduce

```bash
python -c "from gliner import GLiNER"
```

### Expected behavior

The import should pass.

--- Comment by MNIKIEMA at 2026-03-19T11:21:56Z ---
Is passing the comment before the decorator OK?
cc @albertvillanova

--- Comment by Rocketknight1 at 2026-03-19T14:05:17Z ---
Yeah, interesting bug! I'm not sure if moving the comment before the decorator will work because `check_copies.py` uses `start_index = line_index + 1` - in other words, it expects the line after the `# Copied from` to be the function `def`.

I think the simple solution here is just to remove `# Copied from` entirely here, but if this becomes a more general problem then we'll need to rewrite `check_copies.py`.

(This is not an invitation for code agents to submit a slop PR to `check_copies.py`)

--- Comment by MNIKIEMA at 2026-03-19T15:16:45Z ---
> Yeah, interesting bug! I'm not sure if moving the comment before the decorator will work because `check_copies.py` uses `start_index = line_index + 1` - in other words, it expects the line after the `# Copied from` to be the function `def`.
>
> I think the simple solution here is just to remove `# Copied from` entirely here, but if this becomes a more general problem then we'll need to rewrite `check_copies.py`.
>
> (This is not an invitation for code agents to submit a slop PR to `check_copies.py`)

In my case, moving the comment before the decorator works. However, I have no idea about the side effects.

--- Comment by Rocketknight1 at 2026-03-20T13:34:32Z ---
@MNIKIEMA try running `make fix-repo` after that change. Although the Python code might work, it might break out repo linter tools.

--- Comment by Krishnachaitanyakc at 2026-03-25T02:37:51Z ---
Hi, I'd like to work on this. As @Rocketknight1 suggested, the fix is to remove the `# Copied from` comments between the `@torch.jit.script` decorators and the `def` statements for `c2p_dynamic_expand`, `p2c_dynamic_expand`, and `pos_dynamic_expand` in `modeling_deberta_v2.py`. The previous PRs tried moving the comments instead of removing them, which isn't what was requested. I'll open a PR shortly.

--- Comment by albertvillanova at 2026-03-26T07:48:41Z ---
Yes, this was a regression introduced just in Python 3.13.8, and fixed in Python 3.13.9, as I explained in my issue:
- https://github.com/huggingface/transformers/issues/41472


ISSUE #44849: Transformers Qwen3.5 had a bug when set output_hidden_states=True
State: closed  |  Labels: bug
Author: lucasjinreal  |  Created: 2026-03-19T08:27:35Z

### System Info

Version: 5.2.0

in qwen3.5

outputs = model_wrapper.generate(**inputs, output_hidden_states=True)

outpus something like this:

```
><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|image_pad|><|vision_end|>请详细描述这张图片的内容。<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n这张!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!']
```

ignore the `output_hideen_states` params, normal

```
on_end|>请详细描述这张图片的内容。<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n这张图片是一张包含表格的截图，内容是一份关于不同模型在多个任务上表现的数据对比表。\n\n**整体布局：**\n图片的主体是一个表格，表格的标题为“Model”，列出了多个模型名称，以及它们在不同任务上的平均长度（Avg. Length）、任务1、任务2、任务3、任务4和任务5的得分百分比。表格下方还有一行说明文字。\n\n**表格内容详情：**\n表格共有8行数据，对应8个不同的模型。\n\n- **第一行：**\n  - **Model:** `PI0*`\n  - **Avg. Length:** 2.954\n  - **Task 1:** 84.8%\n  - **Task 2:** 70.4%\n  - **Task 3:** 55.9%\n  - **Task 4:** 46.6%\n  - **Task 5:** 37.7%\n\n- **第二行：**\n  - **Model:** `PI0.5*`\n  - **Avg. Length:** 3.885\n  - **Task 1:** 92.5%\n  - **Task 2:** 84.0%\n  - **Task 3:** 76.6%\n  - **Task 4:** 71.0%\n  - **Task 5:** 64.4%\n\n- **第三行：**\n  - **Model:** `qwenpi (qwen2.5-vl-3B-instruct-action)`\n  - **Avg. Length:** 3.5']
```

### Who can help?

w

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

f

### Expected behavior

rg

--- Comment by Rocketknight1 at 2026-03-19T13:06:57Z ---
Hey, can you give us a complete reproducer script for this? It's possible that `output_hidden_states` switches the attention mechanism to `eager`, resulting in slight numerical differences

--- Comment by lucasjinreal at 2026-03-19T14:20:24Z ---
No, the output is totally mess.

In Qwen3 and Qwen2.5, i can set it freely without any weired output.

--- Comment by sdharani91 at 2026-03-19T18:10:38Z ---
I was not able to reproduce this - pls share a repro script.
Do you know whether this repro was on the Qwen3.5 fast path (flash-linear-attention / causal-conv1d) or on the torch fallback path?

Tested with below script and outputs look identical.

```
import sys
sys.path.insert(0, "src")
import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "Qwen/Qwen3.5-0.8B"

processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto" if torch.cuda.is_available() else None,
)
if not torch.cuda.is_available():
    model.to("cpu")

message = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg",
            },
            {"type": "text", "text": "What kind of animal is this?"},
        ],
    }
]

inputs = processor.apply_chat_template(
    message,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

gen_kwargs = dict(max_new_tokens=30, do_sample=False)

with torch.no_grad():
    out_normal = model.generate(**inputs, **gen_kwargs)
    out_hidden = model.generate(**inputs, output_hidden_states=True, **gen_kwargs)

text_normal = processor.decode(out_normal[0], skip_special_tokens=True)
text_hidden = processor.decode(out_hidden[0], skip_special_tokens=True)

print("NORMAL:")
print(text_normal)
print()
print("WITH output_hidden_states=True:")
print(text_hidden)
print()
print("Same ids?", torch.equal(out_normal, out_hidden))

```

--- Comment by lucasjinreal at 2026-03-20T01:27:23Z ---
@sdharani91 I debugged a little, seems caused by this:

```
text = self.processor.apply_chat_template(
            messages,
            # tokenize=True,
            # padding=True,
            add_generation_prompt=True,
            # return_dict=True,
            # return_tensors="pt",
        )

        # batch_inputs = self.processor.apply_chat_template(
        #     messages, tokenize=True, padding=True, add_generation_prompt=True, return_dict=True, return_tensors="pt"
        # )

        batch_inputs = self.processor(
            text=text,
            max_pixels=196 * 196,
            min_pixels=28 * 28,
            images=images,
            return_tensors="pt",
            do_resize=True,
            padding=True,
        )

```

this usage got error，

but this normal:

```
batch_inputs = self.processor.apply_chat_template(
            messages, tokenize=True, padding=True, add_generation_prompt=True, return_dict=True, return_tensors="pt"
        )
```

Do u know why? how to constraint the image size in processor manually

--- Comment by github-actions[bot] at 2026-04-19T08:16:52Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44843: AutoTokenizer.from_pretrained calls model_info() unconditionally in _patch_mistral_regex, breaks HF_HUB_OFFLINE mode
State: closed  |  Labels:
Author: nv-yna  |  Created: 2026-03-19T05:36:56Z

### System Info

- `transformers` version: 4.57.3
- `huggingface_hub` version: 0.36.2
- Python: 3.12
- OS: Linux (Ubuntu 24.04, inside NVIDIA container)

### Who can help?

@ArthurZucker @itazap

### Regression introduced in

PR #42389 (`[Mistral Tokenizers] Fix tokenizer detection`), included in v4.57.2 → v4.57.3.

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import os
from huggingface_hub import snapshot_download

# Step 1: Pre-download a non-Mistral model
snapshot_download("Qwen/Qwen3-0.6B")

# Step 2: Enable offline mode
os.environ["HF_HUB_OFFLINE"] = "1"

# Step 3: Load tokenizer — crashes even though model is fully cached
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
```

This raises:

```
huggingface_hub.errors.OfflineModeIsEnabled: Cannot reach
https://huggingface.co/api/models/Qwen/Qwen3-0.6B: offline mode is enabled.
```

Full traceback:

```
transformers/models/auto/tokenization_auto.py:1156, in from_pretrained
transformers/tokenization_utils_base.py:2113, in from_pretrained
transformers/tokenization_utils_base.py:2395, in _from_pretrained
transformers/tokenization_utils_base.py:2438, in _patch_mistral_regex
transformers/tokenization_utils_base.py:2432, in is_base_mistral
huggingface_hub/hf_api.py:2660, in model_info
→ OfflineModeIsEnabled
```

### Expected behavior

`AutoTokenizer.from_pretrained()` should work in offline mode (`HF_HUB_OFFLINE=1`) when the model is already cached locally. This worked in transformers 4.57.1.

### Root cause

In `tokenization_utils_base.py`, `_patch_mistral_regex` defines `is_base_mistral()` which calls `huggingface_hub.model_info()` — an API call — for **every** model, not just Mistral models. The call happens before any cache check or `local_files_only` guard:

```python
# tokenization_utils_base.py, _patch_mistral_regex
def is_base_mistral(model_id: str) -> bool:
    model = model_info(model_id)  # <-- unconditional API call
    ...

if _is_local or is_base_mistral(pretrained_model_name_or_path):  # <-- called for ALL non-local models
```

The `local_files_only` parameter is passed to `_patch_mistral_regex` but is never used to guard the `is_base_mistral()` call.

### Suggested fix

The `is_base_mistral()` call should either:

1. Be wrapped in a try/except that catches `OfflineModeIsEnabled` and returns `False` (safe default — if we can't reach the API, assume it's not a Mistral model), or
2. Be skipped when `local_files_only=True` or `HF_HUB_OFFLINE=1` is set

### Impact

This breaks any CI/CD pipeline or air-gapped environment that:
1. Pre-downloads models with `snapshot_download()`
2. Sets `HF_HUB_OFFLINE=1` to prevent network access during test execution
3. Loads tokenizers via `AutoTokenizer.from_pretrained()`

This pattern is common in ML CI pipelines (we hit this in NVIDIA Dynamo's CI with TensorRT-LLM).

--- Comment by Rocketknight1 at 2026-03-19T13:10:03Z ---
cc @vasqu since it was your PR!

--- Comment by vasqu at 2026-03-19T15:07:10Z ---
We made a few small patch releases after that, can you check the latest v4 version first? `4.57.6`

--- Comment by nfinnie at 2026-03-23T11:15:07Z ---
> We made a few small patch releases after that, can you check the latest v4 version first? `4.57.6`

I still hit this issue with 4.57.6

--- Comment by vasqu at 2026-03-27T10:10:13Z ---
Taking a look today! We will release a patch release for this Edit: no release, intended behavior after investigating

--- Comment by Wauplin at 2026-03-27T10:58:01Z ---
Hey, maintainer of the underlying `huggingface_hub` library here. We do not support and do not plan to support setting environment variables at runtime with `os.environ["HF_HUB_OFFLINE"] = "1"`. Environment is read only once at import time and should not be modified afterward. The correct way of doing this is to run a separate script when using the offline tokenizer (e.g. `HF_HUB_OFFLINE=1 python my_offline_script.py`).

--- Comment by github-actions[bot] at 2026-04-21T08:31:55Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.