Error running Qwen-3-8B-NVFP4

ISSUE #44292: Error running Qwen-3-8B-NVFP4
State: closed  |  Labels: bug
Author: IKACE  |  Created: 2026-02-26T07:21:27Z

### System Info

Hi,

I encountered the following error while running [Qwen-3-8B-NVFP4 model](https://huggingface.co/RedHatAI/Qwen3-8B-NVFP4).

Is NVFP4 not supported on transformers? My package info is also shared at the bottom.

```
Traceback (most recent call last):
  File "/raid/yilegu/diagnosis_agent_demo/drivers/driver_hf_v2.py", line 372, in <module>
    main()
  File "/raid/yilegu/diagnosis_agent_demo/drivers/driver_hf_v2.py", line 356, in main
    _ = generate_text(
        ^^^^^^^^^^^^^^
  File "/raid/yilegu/diagnosis_agent_demo/drivers/driver_hf_v2.py", line 138, in generate_text
    output_ids = model.generate(**inputs, **gen_kwargs, return_dict_in_generate=True, output_scores=False)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/generation/utils.py", line 2566, in generate
    result = decoding_method(
             ^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/generation/utils.py", line 2786, in _sample
    outputs = self(**model_inputs, return_dict=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/utils/generic.py", line 918, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 480, in forward
    outputs: BaseModelOutputWithPast = self.model(
                                       ^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/utils/generic.py", line 1072, in wrapper
    outputs = func(self, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 410, in forward
    hidden_states = decoder_layer(
                    ^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/modeling_layers.py", line 94, in __call__
    return super().__call__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 260, in forward
    hidden_states, _ = self.self_attn(
                       ^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 200, in forward
    query_states = self.q_norm(self.q_proj(hidden_states).view(hidden_shape)).transpose(1, 2)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/compressed_tensors/quantization/lifecycle/forward.py", line 387, in wrapped_forward
    output = forward_func_orig.__get__(module, module.__class__)(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/compressed_tensors/linear/compressed_linear.py", line 103, in forward
    weight_data = self.compressor.decompress_module(self)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/compressed_tensors/compressors/base.py", line 188, in decompress_module
    return self.decompress_weight(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/raid/yilegu/uv_venvs/transformers-v4576/lib/python3.12/site-packages/compressed_tensors/compressors/quantized_compressors/fp4_quantized.py", line 126, in decompress_weight
    scale = compressed_data["weight_scale"]
            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'weight_scale'
```

Package Info
```
Using Python 3.12.3 environment at: uv_venvs/transformers-v4576
Package                  Version
------------------------ ---------
accelerate               1.12.0
annotated-types          0.7.0
certifi                  2026.1.4
charset-normalizer       3.4.4
compressed-tensors       0.13.0
cuda-bindings            12.9.4
cuda-pathfinder          1.3.4
filelock                 3.24.3
fsspec                   2026.2.0
hf-xet                   1.2.0
huggingface-hub          0.36.2
idna                     3.11
jinja2                   3.1.6
loguru                   0.7.3
markupsafe               3.0.3
mpmath                   1.3.0
networkx                 3.6.1
numpy                    2.4.2
nvidia-cublas-cu12       12.8.4.1
nvidia-cuda-cupti-cu12   12.8.90
nvidia-cuda-nvrtc-cu12   12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12        9.10.2.21
nvidia-cufft-cu12        11.3.3.83
nvidia-cufile-cu12       1.13.1.3
nvidia-curand-cu12       10.3.9.90
nvidia-cusolver-cu12     11.7.3.90
nvidia-cusparse-cu12     12.5.8.93
nvidia-cusparselt-cu12   0.7.1
nvidia-nccl-cu12         2.27.5
nvidia-nvjitlink-cu12    12.8.93
nvidia-nvshmem-cu12      3.4.5
nvidia-nvtx-cu12         12.8.90
packaging                26.0
psutil                   7.2.2
pydantic                 2.12.5
pydantic-core            2.41.5
pyyaml                   6.0.3
regex                    2026.2.19
requests                 2.32.5
safetensors              0.7.0
setuptools               82.0.0
sympy                    1.14.0
tokenizers               0.22.2
torch                    2.10.0
tqdm                     4.67.3
transformers             4.57.6
triton                   3.6.0
typing-extensions        4.15.0
typing-inspection        0.4.2
urllib3                  2.6.3
```

### Who can help?

@ArthurZucker  @SunMarc @MekkCyber

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_ID = "RedHatAI/Qwen3-8B-NVFP4"
DEVICE = "cuda:0" if torch.cuda.is_available() else "cpu"

print("Torch:", torch.__version__)
print("CUDA available:", torch.cuda.is_available())
if torch.cuda.is_available():
    print("GPU:", torch.cuda.get_device_name(0))

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID)
model.eval()
model.to(DEVICE)

# Single request
prompt = "Say hello in one sentence."
inputs = tokenizer(prompt, return_tensors="pt").to(DEVICE)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=32,
        do_sample=False
    )

generated = outputs[:, inputs["input_ids"].shape[1]:]
text = tokenizer.decode(generated[0], skip_special_tokens=False)

print("\n=== OUTPUT ===")
print(text)
```

### Expected behavior

No error

--- Comment by SunMarc at 2026-02-27T11:42:57Z ---
Hmmm this should be supported as this is using `compressed-tensors` library integration in transformers. Maybe you can open an issue there so that they can make sure that the support is up-to-date !

--- Comment by github-actions[bot] at 2026-03-28T08:07:55Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44291: Bug: TypeError when loading model with `init_empty_weights` in transformers >= 5.0.0rc0 due to unexpected `_is_hf_initialized` argument
State: closed  |  Labels: bug
Author: chenyushuo  |  Created: 2026-02-26T02:36:06Z

### System Info

- `transformers` version: 5.2.0
- Platform: Linux-5.10.134-013.5.kangaroo.al8.x86_64-x86_64-with-glibc2.39
- Python version: 3.12.12
- Huggingface_hub version: 1.4.1
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1+cu128 (CUDA)
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: NVIDIA H20

### Who can help?

@Cyrilvallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. Run the following scripts:
```python
import transformers
from accelerate import init_empty_weights

model_path = 'Qwen/Qwen3-0.6B'
with init_empty_weights():
    model = transformers.AutoModelForCausalLM.from_pretrained(model_path)
```
2. I have also tried loading other models, but it always encounter the following error:
```
Traceback (most recent call last):
  File "/mnt/data/chenyushuo.cys/Trinity/tmpfiles/test20260225.py", line 11, in <module>
    model = transformers.AutoModelForCausalLM.from_pretrained(model_path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 374, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4072, in from_pretrained
    loading_info, disk_offload_index = cls._load_pretrained_model(model, state_dict, checkpoint_files, load_config)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4191, in _load_pretrained_model
    loading_info, disk_offload_index = convert_and_load_state_dict_in_model(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 1225, in convert_and_load_state_dict_in_model
    set_param_for_module(
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 922, in set_param_for_module
    setattr(module_obj, param_name, param_value)
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1989, in __setattr__
    self.register_parameter(name, value)
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/accelerate/big_modeling.py", line 135, in register_empty_parameter
    module._parameters[name] = param_cls(module._parameters[name].to(device), **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Parameter.__new__() got an unexpected keyword argument '_is_hf_initialized'
```
3. **Investigation Findings:**
The error occurs because `transformers.core_model_loading.set_param_for_module` attaches a custom `_is_hf_initialized` attribute to the parameter object. When `accelerate.big_modeling.register_empty_parameter` later tries to reconstruct this parameter, it indiscriminately passes all existing attributes (including `_is_hf_initialized`) to `torch.nn.Parameter.__new__()`. As `torch.nn.Parameter` does not recognize this keyword argument, the process fails with a `TypeError`.

### Expected behavior

I also run the above code on `transformers<=4.57.6`, and the model loaded normally.

--- Comment by cynricfu at 2026-02-26T08:39:42Z ---
I cranked [a simple fix](https://github.com/huggingface/accelerate/pull/3943) for this by adding a line that pops the `_is_hf_initialized` attribute before initializing the empty parameter object.

--- Comment by Rocketknight1 at 2026-02-26T14:16:56Z ---
Happening in accelerate so cc @sunmarc maybe?

--- Comment by SunMarc at 2026-02-27T11:32:40Z ---
You should load the model using `model = transformers.AutoModelForCausalLM.from_config(model_path)` instead or use `torch.device("meta")`. cc @Cyrilvallez I think there is no need to use with `init_empty_weights` anymore ?

--- Comment by eval-dev at 2026-03-06T23:59:53Z ---
> You should load the model using `model = transformers.AutoModelForCausalLM.from_config(model_path)` instead or use `torch.device("meta")`

hi @SunMarc, apart from this approach, can you help with a fix in accelerate library? Because `init_empty_weights` is used by many training frameworks, like verl. Fixing in init_empty_weights will be the ultimate solution.

--- Comment by Cyrilvallez at 2026-03-09T11:56:35Z ---
Hey @chenyushuo! As @SunMarc explained, it does not make any sense in general to try to call `from_pretrained`, while also using a `meta` or `empty_weight` context manager. `from_pretrained`, as its names indicates, wants to load existing pretrained weights. We no longer allow it with meta device, see [here](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/accelerate.py#L97-L104). `init_empty_weights` is the same, and while we cannot detect it and prevent it, it should not be used as it's a clear anti-pattern.

Use `from_config` or `Model(config)` directly instead!


ISSUE #44280: Add species bias to model card templates and bias documentation
State: closed  |  Labels:
Author: samtuckerdavis  |  Created: 2026-02-25T14:03:32Z

## Problem

The auto-generated model card template in src/transformers/modelcard.py includes an "Intended uses & limitations" section but provides no guidance for documenting species bias — the systematic devaluation of non-human animals' interests relative to humans.

When bias is discussed in the Transformers ecosystem, it covers racial, gender, religious biases. Species bias is absent, despite LLMs trained on human text inheriting measurable speciesist patterns.

## Why this matters

- **Empirical evidence**: Hagendorff et al. (2023) demonstrated LLMs exhibit significant speciesist bias ([DOI: 10.1007/s43681-023-00380-w](https://doi.org/10.1007/s43681-023-00380-w))
- **Benchmarking**: SpeciesismBench (2025) provides standardized evaluation ([arXiv:2508.11534](https://arxiv.org/abs/2508.11534))
- **Domain relevance**: Models on HuggingFace are used in food systems, agriculture, environmental science — domains where speciesist bias affects real-world decisions

## Proposed changes

### 1. Update auto-generated model card template
In src/transformers/modelcard.py, expand the template to include species alongside other bias categories in a "Bias, Risks, and Limitations" section.

### 2. Model documentation pages
For commonly used models (BERT, GPT-2, LLaMA), note that models trained on internet text reproduce anthropocentric value hierarchies, especially relevant for food/agriculture/environmental domains.

### 3. add_new_model documentation
The guide at docs/source/en/add_new_model.md could enumerate bias categories to consider, including species.

## Related work

We have an open PR on huggingface/hub-docs adding speciesist bias to Hub model card guidelines: [huggingface/hub-docs#2249](https://github.com/huggingface/hub-docs/pull/2249)

Happy to submit a PR implementing any of these changes.

— [Open Paws](https://openpaws.ai)

--- Comment by github-actions[bot] at 2026-03-28T08:07:57Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


Dependency issue with transformers

ISSUE #44279: Dependency issue with transformers
State: closed  |  Labels: bug
Author: Chilliwiddit  |  Created: 2026-02-25T13:09:17Z

### System Info

```
%%capture
!pip uninstall -y bitsandbytes bitsandbytes-cuda* torch torchvision torchaudio xformers transformers datasets pyarrow huggingface-hub numpy spacy thinc

!pip install "numpy<2.0.0"

!pip install --index-url https://download.pytorch.org/whl/cu124 torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0

!pip install bitsandbytes==0.49.2

!pip install --upgrade transformers accelerate peft huggingface_hub datasets pyarrow
!pip install evaluate rouge_score bert-score faiss-gpu sentence-transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
from huggingface_hub import login
from datasets import load_dataset
from sentence_transformers import SentenceTransformer
from sklearn.neighbors import NearestNeighbors
import numpy as np
import os
import gc
from tqdm import tqdm
import evaluate
import time

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_55/852286074.py in <cell line: 0>()
      1 import torch
----> 2 from transformers import AutoModelForCausalLM, AutoTokenizer
      3 from peft import PeftModel
      4 from huggingface_hub import login
      5 from datasets import load_dataset

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
```

What's happened here? How do I fix this?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Copy and run the code in a kaggle notebook with a T4 GPU

### Expected behavior

Normally it would install just fine and I would be able to proceed with the rest of the code. A copy of the notebook can be found [here](https://www.kaggle.com/code/nivedhithjeyashanker/copy-of-evaluate-icl-alpaca)

--- Comment by taruasnigdha at 2026-02-25T15:51:54Z ---
I think the issue is that Kaggle already comes with:

Preinstalled PyTorch
Preinstalled NumPy
Precompiled C extensions
Python 3.12

When you force-reinstall numpy, you break Kaggle’s precompiled ABI stack → exactly the dtype size changed error you're seeing.

First restart the notebook and run :
```
%%capture

# Remove only high-level HF libraries (NOT torch, NOT numpy)
!pip uninstall -y transformers datasets accelerate peft huggingface_hub bitsandbytes sentence-transformers faiss-gpu

# Install torch-compatible stack (DO NOT touch numpy)
!pip install transformers accelerate peft huggingface_hub datasets
!pip install bitsandbytes==0.49.0
!pip install sentence-transformers
!pip install faiss-gpu
!pip install evaluate rouge_score bert-score
```

Kaggle’s PyTorch is compiled against its system NumPy.
If you replace NumPy → binary mismatch → crash.

--- Comment by Chilliwiddit at 2026-02-26T07:16:37Z ---
thank you, much appreciated @taruasnigdha


Loading kimik2 is taking forever

ISSUE #44276: Loading kimik2 is taking forever
State: closed  |  Labels:
Author: savitha-suresh  |  Created: 2026-02-25T08:33:05Z

```import os
import json
import yaml
import argparse
from dotenv import load_dotenv
import torch
from pathlib import Path
from transformers import (
    AutoConfig,
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)

model_name = os.path.expanduser("~/model/kimik2")
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
```

The above is slow at - match_named_modules (compressed_tensors/utils/match.py:69)
Even if i bypass this. there is another forloop due to which the execution appears stuck
```
Thread 0x7E57077D8740 (active+gil): "MainThread"
    <dictcomp> (transformers/modeling_utils.py:5353)
    _load_pretrained_model (transformers/modeling_utils.py:5352)
 ```
The version i am using is 4.57.6.

1. Can someone please help to let me know how to enable fast loading of large models like kimik2?
2. How to load a model that is natively in int4? https://huggingface.co/moonshotai/Kimi-K2-Thinking/tree/main. Will AutoModelForCausalLM.from_pretrained only work with bf16 and then use QLoRa to convert again to int4?

--- Comment by Saad-Mallebhari at 2026-02-28T10:14:40Z ---
@savitha-suresh
**1. Slow loading**
The bottleneck in `match_named_modules` is a known issue with `compressed_tensors` on large MoE models , it pattern-matches every module name against quantization configs, which scales poorly with model size. Not much you can do to skip it, but you can speed up overall loading with:
```python
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
    low_cpu_mem_usage=True,
)
```
`low_cpu_mem_usage=True` avoids materializing the full model in CPU RAM before moving to GPU, which significantly reduces load time for large models.

**2. Loading native int4**
Kimi-K2 is quantized in `compressed_tensors` format , not bitsandbytes. You do NOT need to re-quantize with QLoRA. `AutoModelForCausalLM.from_pretrained` will handle it automatically as long as `compressed_tensors` is installed:
```bash
pip install compressed-tensors
```
Then load normally , the quantization config in the model's `config.json` will be picked up automatically. No `BitsAndBytesConfig` needed.

--- Comment by liaol at 2026-03-04T13:07:37Z ---
> [@savitha-suresh](https://github.com/savitha-suresh) **1. Slow loading** The bottleneck in `match_named_modules` is a known issue with `compressed_tensors` on large MoE models , it pattern-matches every module name against quantization configs, which scales poorly with model size. Not much you can do to skip it, but you can speed up overall loading with:
>
> model = AutoModelForCausalLM.from_pretrained(
>     model_name,
>     torch_dtype=torch.bfloat16,
>     device_map="auto",
>     trust_remote_code=True,
>     low_cpu_mem_usage=True,
> )
> `low_cpu_mem_usage=True` avoids materializing the full model in CPU RAM before moving to GPU, which significantly reduces load time for large models.
>
> **2. Loading native int4** Kimi-K2 is quantized in `compressed_tensors` format , not bitsandbytes. You do NOT need to re-quantize with QLoRA. `AutoModelForCausalLM.from_pretrained` will handle it automatically as long as `compressed_tensors` is installed:
>
> pip install compressed-tensors
> Then load normally , the quantization config in the model's `config.json` will be picked up automatically. No `BitsAndBytesConfig` needed.

@Saad-Mallebhari

After installing compressed-tensors, the performance is still slow.
The stack is:

```
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 597, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/transformers/modeling_utils.py", line 277, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4998, in from_pretrained
    hf_quantizer.preprocess_model(
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/transformers/quantizers/base.py", line 225, in preprocess_model
    return self._process_model_before_weight_loading(model, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/transformers/quantizers/quantizer_compressed_tensors.py", line 84, in _process_model_before_weight_loading
    self.compressor.compress_model(model=model)
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/compressed_tensors/compressors/model_compressors/model_compressor.py", line 456, in compress_model
    for prefix, module in tqdm(
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/compressed_tensors/utils/match.py", line 57, in match_named_modules
    if is_match(name, module, target, fused=fused):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/compressed_tensors/utils/match.py", line 371, in is_match
    any(
  File "/sgl-workspace/bf16/lib/python3.12/site-packages/compressed_tensors/utils/match.py", line 372, in <genexpr>
    _match_name(name, target, fused) or _match_class(module, target)
```

--- Comment by Saad-Mallebhari at 2026-03-05T08:42:38Z ---
@liaol
Thanks for sharing the full stack , this suggests the bottleneck is inside `compressed_tensors` itself at `match_named_modules`, not in the transformers loading path. The `low_cpu_mem_usage` flag won't help here because the slow step happens during `preprocess_model` before weights are even loaded.

The `match_named_modules` loop pattern-matches every module in the model against quantization configs , on a large MoE model like Kimi-K2 with hundreds of expert layers, this scales very poorly.

A few things worth trying:

1. **Try a different compressed-tensors version** - it's worth checking whether this slowness is version-specific by testing an older release:
```bash
pip show compressed-tensors  # note your current version first
pip install compressed-tensors==<older_version>
```

2. **File an issue on the compressed-tensors repo** , since the bottleneck appears to originate from their `match_named_modules` implementation, reporting it directly there is likely the most effective path:
   https://github.com/neuralmagic/compressed-tensors

3. **If your goal is inference rather than debugging this load path**, you might also consider trying vLLM , it has its own compressed-tensors integration that handles large MoE quantized models and may avoid this path entirely.

There's no clean workaround on the transformers side for this specific bottleneck , the slowdown appears to originate from `compressed_tensors` and would likely need to be addressed there.

--- Comment by github-actions[bot] at 2026-03-30T08:27:35Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by kylesayrs at 2026-05-07T19:33:55Z ---
I'm not sure why people think that the bottleneck is the matching logic, rather than [all of the work that's being done within the iterable](https://github.com/vllm-project/compressed-tensors/blob/0.12.2/src/compressed_tensors/compressors/model_compressors/model_compressor.py#L477-L531).

On a `INTEL(R) XEON(R) PLATINUM 8570` for `moonshotai/Kimi-K2-Thinking`,`match_named_modules` achieved a speed of `244232it/s` for `116423` modules.

This cost is highly negligible w.r.t. the cost of compressing the modules on the meta device
```
Iterating with match_named_modues: 69120it [00:00, 145067.86it/s]  # ie 0.47s for 116423 modules
Applying Quantization Config: 69120it [00:03, 19410.65it/s]
Compressing model: 100%|█| 116423/116423 [01:18<00:00, 1480.83it/s]
```

I think it's likely that there was a slowdown in older releases of compressed tensors before the ModelCompressor refactor which resulted in slowdowns without tqdms. The newer version loads the model in under 5 minutes.


Lazy loading is not working properly

ISSUE #44273: Lazy loading is not working properly
State: open  |  Labels: bug
Author: albertvillanova  |  Created: 2026-02-25T06:23:02Z

### Problem

Lazy loading is not working properly:
- importing transformers takes ~3.5s

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```bash
time python -c "import transformers"
```
```
real	0m3.669s
user	0m4.789s
sys	0m0.311s
```

### Expected behavior

It should take less <0.5s

--- Comment by github-actions[bot] at 2026-03-27T08:12:01Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by albertvillanova at 2026-03-27T15:04:35Z ---
As of today, the lazy import continues broken:
```bash
time python -c "import transformers"

real	0m3.639s
user	0m4.863s
sys	0m0.320s
```

CC: @Rocketknight1, @LysandreJik

--- Comment by tommy-fcy at 2026-04-17T15:16:54Z ---
**TL;DR:** Disable unused backend probing before importing transformers:

```bash
export TRANSFORMERS_NO_TF=1
export TRANSFORMERS_NO_FLAX=1
```

Or in Python:
```python
import os
os.environ["TRANSFORMERS_NO_TF"] = "1"
os.environ["TRANSFORMERS_NO_FLAX"] = "1"
import transformers
```

**Root cause:** `transformers` runs an expensive directory scan (`_create_import_structure_from_path`) at import time across 300+ model directories. Having multiple ML backends (PyTorch + TensorFlow/JAX) installed causes eager probing of all backends, which is the single largest contributor — typically inflating import time from ~1s to ~3.5s. Additionally, use selective imports (`from transformers import AutoModel`) instead of `import transformers` when possible.

If you don't need TF/JAX at all: `pip uninstall tensorflow jax flax` provides the largest speedup.

Related: #44246, #16863

---
*Generated by [OctoScout](https://github.com/tommy-fcy/OctoScout) — AI-assisted ML compatibility diagnosis.*

--- Comment by tarekziade at 2026-04-20T09:27:06Z ---
@albertvillanova I worked on this lately, I pushed a fix ~3weeks ago. see https://github.com/huggingface/transformers/pull/45013

Would you mind trying again with a fresh version of `main`?

This is my timing with torch installed:

```
(venv) m5 ➜transformers git:(tarekziade-otel) ✗ time python -c "import transformers"
python -c "import transformers"  0.30s user 0.09s system 98% cpu 0.393 total
```

I also added a QA test for this to make sure it would not slow down again:

```
✗ python utils/check_import_complexity.py
Import complexity OK: 585 modules (max 1000)
```

and you can use `--display` to see the whole tree

--- Comment by albertvillanova at 2026-04-23T05:25:38Z ---
Thanks for addressing this issue, @tarekziade.

Unfortunately, I am still seeing a non-negligible latency:
```bash
time python -c "import transformers"

real	0m2.738s
user	0m1.389s
sys	0m0.144s
```

--- Comment by tarekziade at 2026-04-23T05:50:44Z ---
Could you run `python utils/check_import_complexity.py` and see how many imports you see?
If it's a much bigger number than mine, something is odd, we'll need to run with `--display` so we can investigate what gets pulled where.
If it's like mine, this means your hardware is slower than mine and we would need to see if there are more places we could lazy load.

--- Comment by albertvillanova at 2026-04-23T06:19:29Z ---
```bash
python utils/check_import_complexity.py
Import complexity OK: 595 modules (max 1000)
```
<details>
<summary>--display</summary>

```bash
python utils/check_import_complexity.py --display
└── transformers
    ├── transformers.dependency_versions_check
    │   ├── transformers.dependency_versions_table
    │   ├── transformers.utils
    │   │   ├── packaging
    │   │   ├── packaging.version
    │   │   │   └── packaging._structures
    │   │   ├── transformers.utils.auto_docstring
    │   │   │   ├── regex
    │   │   │   │   └── regex._main
    │   │   │   │       └── regex._regex_core
    │   │   │   │           ├── string
    │   │   │   │           │   └── _string
    │   │   │   │           ├── unicodedata
    │   │   │   │           └── regex._regex
    │   │   │   ├── typing_extensions
    │   │   │   │   └── _socket
    │   │   │   ├── transformers.utils.doc
    │   │   │   │   └── textwrap
    │   │   │   └── transformers.utils.generic
    │   │   │       ├── json
    │   │   │       │   ├── json.decoder
    │   │   │       │   │   └── json.scanner
    │   │   │       │   │       └── _json
    │   │   │       │   └── json.encoder
    │   │   │       ├── random
    │   │   │       │   ├── math
    │   │   │       │   ├── bisect
    │   │   │       │   │   └── _bisect
    │   │   │       │   ├── _random
    │   │   │       │   └── _sha512
    │   │   │       ├── numpy
    │   │   │       │   ├── numpy._globals
    │   │   │       │   │   └── numpy._utils
    │   │   │       │   │       └── numpy._utils._convertions
    │   │   │       │   ├── numpy._expired_attrs_2_0
    │   │   │       │   ├── numpy.version
    │   │   │       │   ├── numpy._distributor_init
    │   │   │       │   ├── numpy.__config__
    │   │   │       │   │   └── numpy._core
    │   │   │       │   │       ├── numpy._core.multiarray
    │   │   │       │   │       │   └── numpy._core.overrides
    │   │   │       │   │       │       ├── numpy._utils._inspect
    │   │   │       │   │       │       ├── numpy._core._multiarray_umath
    │   │   │       │   │       │       ├── datetime
    │   │   │       │   │       │       │   └── _datetime
    │   │   │       │   │       │       ├── numpy.exceptions
    │   │   │       │   │       │       ├── numpy._core._exceptions
    │   │   │       │   │       │       ├── numpy._core.printoptions
    │   │   │       │   │       │       │   └── contextvars
    │   │   │       │   │       │       │       └── _contextvars
    │   │   │       │   │       │       └── numpy.dtypes
    │   │   │       │   │       ├── numpy._core.umath
    │   │   │       │   │       ├── numpy._core.numerictypes
    │   │   │       │   │       │   ├── numbers
    │   │   │       │   │       │   ├── numpy._core._string_helpers
    │   │   │       │   │       │   ├── numpy._core._type_aliases
    │   │   │       │   │       │   └── numpy._core._dtype
    │   │   │       │   │       ├── numpy._core.numeric
    │   │   │       │   │       │   ├── numpy._core.shape_base
    │   │   │       │   │       │   │   └── numpy._core.fromnumeric
    │   │   │       │   │       │   │       └── numpy._core._methods
    │   │   │       │   │       │   │           └── pickle
    │   │   │       │   │       │   │               ├── struct
    │   │   │       │   │       │   │               │   └── _struct
    │   │   │       │   │       │   │               ├── _compat_pickle
    │   │   │       │   │       │   │               └── _pickle
    │   │   │       │   │       │   ├── numpy._core._ufunc_config
    │   │   │       │   │       │   ├── numpy._core.arrayprint
    │   │   │       │   │       │   └── numpy._core._asarray
    │   │   │       │   │       ├── numpy._core.records
    │   │   │       │   │       ├── numpy._core.memmap
    │   │   │       │   │       ├── numpy._core.function_base
    │   │   │       │   │       ├── numpy._core._machar
    │   │   │       │   │       ├── numpy._core.getlimits
    │   │   │       │   │       ├── numpy._core.einsumfunc
    │   │   │       │   │       ├── numpy._core._add_newdocs
    │   │   │       │   │       ├── numpy._core._add_newdocs_scalars
    │   │   │       │   │       ├── numpy._core._dtype_ctypes
    │   │   │       │   │       ├── numpy._core._internal
    │   │   │       │   │       │   └── ctypes
    │   │   │       │   │       │       ├── _ctypes
    │   │   │       │   │       │       └── ctypes._endian
    │   │   │       │   │       └── numpy._pytesttester
    │   │   │       │   ├── numpy.lib
    │   │   │       │   │   ├── numpy.lib.array_utils
    │   │   │       │   │   │   └── numpy.lib._array_utils_impl
    │   │   │       │   │   ├── numpy.lib.introspect
    │   │   │       │   │   ├── numpy.lib.mixins
    │   │   │       │   │   ├── numpy.lib.npyio
    │   │   │       │   │   │   └── numpy.lib._npyio_impl
    │   │   │       │   │   │       ├── numpy.lib.format
    │   │   │       │   │   │       │   └── numpy.lib._utils_impl
    │   │   │       │   │   │       │       └── platform
    │   │   │       │   │   │       │           └── subprocess
    │   │   │       │   │   │       │               ├── signal
    │   │   │       │   │   │       │               ├── fcntl
    │   │   │       │   │   │       │               ├── _posixsubprocess
    │   │   │       │   │   │       │               ├── select
    │   │   │       │   │   │       │               └── selectors
    │   │   │       │   │   │       ├── numpy.lib._datasource
    │   │   │       │   │   │       └── numpy.lib._iotools
    │   │   │       │   │   ├── numpy.lib.scimath
    │   │   │       │   │   │   └── numpy.lib._scimath_impl
    │   │   │       │   │   │       └── numpy.lib._type_check_impl
    │   │   │       │   │   │           └── numpy.lib._ufunclike_impl
    │   │   │       │   │   ├── numpy.lib.stride_tricks
    │   │   │       │   │   │   └── numpy.lib._stride_tricks_impl
    │   │   │       │   │   ├── numpy.lib._index_tricks_impl
    │   │   │       │   │   │   ├── numpy.matrixlib
    │   │   │       │   │   │   │   └── numpy.matrixlib.defmatrix
    │   │   │       │   │   │   │       └── numpy.linalg
    │   │   │       │   │   │   │           ├── numpy.linalg.linalg
    │   │   │       │   │   │   │           └── numpy.linalg._linalg
    │   │   │       │   │   │   │               ├── numpy.lib._twodim_base_impl
    │   │   │       │   │   │   │               ├── numpy.linalg._umath_linalg
    │   │   │       │   │   │   │               └── numpy._typing
    │   │   │       │   │   │   │                   ├── numpy._typing._nested_sequence
    │   │   │       │   │   │   │                   ├── numpy._typing._nbit_base
    │   │   │       │   │   │   │                   ├── numpy._typing._nbit
    │   │   │       │   │   │   │                   ├── numpy._typing._char_codes
    │   │   │       │   │   │   │                   ├── numpy._typing._scalars
    │   │   │       │   │   │   │                   ├── numpy._typing._shape
    │   │   │       │   │   │   │                   ├── numpy._typing._dtype_like
    │   │   │       │   │   │   │                   ├── numpy._typing._array_like
    │   │   │       │   │   │   │                   └── numpy._typing._ufunc
    │   │   │       │   │   │   └── numpy.lib._function_base_impl
    │   │   │       │   │   │       └── numpy.lib._histograms_impl
    │   │   │       │   │   ├── numpy.lib._nanfunctions_impl
    │   │   │       │   │   ├── numpy.lib._shape_base_impl
    │   │   │       │   │   ├── numpy.lib._arraysetops_impl
    │   │   │       │   │   ├── numpy.lib._polynomial_impl
    │   │   │       │   │   ├── numpy.lib._arrayterator_impl
    │   │   │       │   │   ├── numpy.lib._arraypad_impl
    │   │   │       │   │   └── numpy.lib._version
    │   │   │       │   └── numpy._array_api_info
    │   │   │       ├── transformers.utils.logging
    │   │   │       │   ├── logging
    │   │   │       │   │   ├── traceback
    │   │   │       │   │   └── atexit
    │   │   │       │   ├── huggingface_hub
    │   │   │       │   ├── huggingface_hub.utils
    │   │   │       │   │   ├── huggingface_hub.errors
    │   │   │       │   │   │   └── httpx
    │   │   │       │   │   │       ├── httpx.__version__
    │   │   │       │   │   │       ├── httpx._api
    │   │   │       │   │   │       │   └── httpx._client
    │   │   │       │   │   │       │       ├── httpx._auth
    │   │   │       │   │   │       │       │   ├── hashlib
    │   │   │       │   │   │       │       │   │   ├── _hashlib
    │   │   │       │   │   │       │       │   │   └── _blake2
    │   │   │       │   │   │       │       │   ├── base64
    │   │   │       │   │   │       │       │   │   └── binascii
    │   │   │       │   │   │       │       │   ├── urllib.request
    │   │   │       │   │   │       │       │   │   ├── email
    │   │   │       │   │   │       │       │   │   ├── http
    │   │   │       │   │   │       │       │   │   ├── http.client
    │   │   │       │   │   │       │       │   │   │   ├── email.parser
    │   │   │       │   │   │       │       │   │   │   │   └── email.feedparser
    │   │   │       │   │   │       │       │   │   │   │       ├── email.errors
    │   │   │       │   │   │       │       │   │   │   │       └── email._policybase
    │   │   │       │   │   │       │       │   │   │   │           ├── email.header
    │   │   │       │   │   │       │       │   │   │   │           │   ├── email.quoprimime
    │   │   │       │   │   │       │       │   │   │   │           │   ├── email.base64mime
    │   │   │       │   │   │       │       │   │   │   │           │   └── email.charset
    │   │   │       │   │   │       │       │   │   │   │           │       └── email.encoders
    │   │   │       │   │   │       │       │   │   │   │           │           └── quopri
    │   │   │       │   │   │       │       │   │   │   │           └── email.utils
    │   │   │       │   │   │       │       │   │   │   │               ├── socket
    │   │   │       │   │   │       │       │   │   │   │               │   └── array
    │   │   │       │   │   │       │       │   │   │   │               └── email._parseaddr
    │   │   │       │   │   │       │       │   │   │   │                   └── calendar
    │   │   │       │   │   │       │       │   │   │   ├── email.message
    │   │   │       │   │   │       │       │   │   │   │   ├── uu
    │   │   │       │   │   │       │       │   │   │   │   ├── email._encoded_words
    │   │   │       │   │   │       │       │   │   │   │   └── email.iterators
    │   │   │       │   │   │       │       │   │   │   └── ssl
    │   │   │       │   │   │       │       │   │   │       └── _ssl
    │   │   │       │   │   │       │       │   │   ├── tempfile
    │   │   │       │   │   │       │       │   │   └── urllib.error
    │   │   │       │   │   │       │       │   │       └── urllib.response
    │   │   │       │   │   │       │       │   ├── httpx._exceptions
    │   │   │       │   │   │       │       │   └── httpx._models
    │   │   │       │   │   │       │       │       ├── http.cookiejar
    │   │   │       │   │   │       │       │       ├── httpx._content
    │   │   │       │   │   │       │       │       │   └── httpx._multipart
    │   │   │       │   │   │       │       │       │       ├── mimetypes
    │   │   │       │   │   │       │       │       │       ├── httpx._types
    │   │   │       │   │   │       │       │       │       └── httpx._utils
    │   │   │       │   │   │       │       │       ├── httpx._decoders
    │   │   │       │   │   │       │       │       ├── httpx._status_codes
    │   │   │       │   │   │       │       │       └── httpx._urls
    │   │   │       │   │   │       │       │           ├── idna
    │   │   │       │   │   │       │       │           │   ├── idna.core
    │   │   │       │   │   │       │       │           │   │   ├── idna.idnadata
    │   │   │       │   │   │       │       │           │   │   └── idna.intranges
    │   │   │       │   │   │       │       │           │   └── idna.package_data
    │   │   │       │   │   │       │       │           └── httpx._urlparse
    │   │   │       │   │   │       │       ├── httpx._config
    │   │   │       │   │   │       │       └── httpx._transports
    │   │   │       │   │   │       │           ├── httpx._transports.asgi
    │   │   │       │   │   │       │           │   └── httpx._transports.base
    │   │   │       │   │   │       │           ├── httpx._transports.default
    │   │   │       │   │   │       │           ├── httpx._transports.mock
    │   │   │       │   │   │       │           └── httpx._transports.wsgi
    │   │   │       │   │   │       └── httpx._main
    │   │   │       │   │   │           ├── click
    │   │   │       │   │   │           │   ├── click.core
    │   │   │       │   │   │           │   │   ├── click.types
    │   │   │       │   │   │           │   │   │   ├── click._compat
    │   │   │       │   │   │           │   │   │   └── click.exceptions
    │   │   │       │   │   │           │   │   │       ├── click.globals
    │   │   │       │   │   │           │   │   │       └── click.utils
    │   │   │       │   │   │           │   │   ├── click._utils
    │   │   │       │   │   │           │   │   ├── click.formatting
    │   │   │       │   │   │           │   │   │   └── click.parser
    │   │   │       │   │   │           │   │   └── click.termui
    │   │   │       │   │   │           │   └── click.decorators
    │   │   │       │   │   │           ├── pygments
    │   │   │       │   │   │           ├── pygments.lexers
    │   │   │       │   │   │           │   ├── pygments.lexers._mapping
    │   │   │       │   │   │           │   ├── pygments.modeline
    │   │   │       │   │   │           │   ├── pygments.plugin
    │   │   │       │   │   │           │   │   └── importlib.metadata
    │   │   │       │   │   │           │   │       ├── csv
    │   │   │       │   │   │           │   │       │   └── _csv
    │   │   │       │   │   │           │   │       ├── zipfile
    │   │   │       │   │   │           │   │       ├── importlib.metadata._adapters
    │   │   │       │   │   │           │   │       │   └── importlib.metadata._text
    │   │   │       │   │   │           │   │       │       └── importlib.metadata._functools
    │   │   │       │   │   │           │   │       ├── importlib.metadata._meta
    │   │   │       │   │   │           │   │       ├── importlib.metadata._collections
    │   │   │       │   │   │           │   │       └── importlib.metadata._itertools
    │   │   │       │   │   │           │   └── pygments.util
    │   │   │       │   │   │           ├── rich
    │   │   │       │   │   │           │   └── rich._extension
    │   │   │       │   │   │           ├── rich.console
    │   │   │       │   │   │           │   ├── getpass
    │   │   │       │   │   │           │   │   └── termios
    │   │   │       │   │   │           │   ├── html
    │   │   │       │   │   │           │   │   └── html.entities
    │   │   │       │   │   │           │   ├── rich._null_file
    │   │   │       │   │   │           │   ├── rich.errors
    │   │   │       │   │   │           │   ├── rich.themes
    │   │   │       │   │   │           │   │   ├── rich.default_styles
    │   │   │       │   │   │           │   │   │   └── rich.style
    │   │   │       │   │   │           │   │   │       └── rich.color
    │   │   │       │   │   │           │   │   │           ├── colorsys
    │   │   │       │   │   │           │   │   │           ├── rich._palettes
    │   │   │       │   │   │           │   │   │           │   └── rich.palette
    │   │   │       │   │   │           │   │   │           │       └── rich.color_triplet
    │   │   │       │   │   │           │   │   │           ├── rich.repr
    │   │   │       │   │   │           │   │   │           └── rich.terminal_theme
    │   │   │       │   │   │           │   │   └── rich.theme
    │   │   │       │   │   │           │   │       └── configparser
    │   │   │       │   │   │           │   ├── rich._emoji_replace
    │   │   │       │   │   │           │   │   └── rich._emoji_codes
    │   │   │       │   │   │           │   ├── rich._export_format
    │   │   │       │   │   │           │   ├── rich._fileno
    │   │   │       │   │   │           │   ├── rich._log_render
    │   │   │       │   │   │           │   │   └── rich.text
    │   │   │       │   │   │           │   │       ├── rich._loop
    │   │   │       │   │   │           │   │       ├── rich._pick
    │   │   │       │   │   │           │   │       ├── rich._wrap
    │   │   │       │   │   │           │   │       │   └── rich.cells
    │   │   │       │   │   │           │   │       │       └── rich._unicode_data
    │   │   │       │   │   │           │   │       │           └── rich._unicode_data._versions
    │   │   │       │   │   │           │   │       ├── rich.align
    │   │   │       │   │   │           │   │       │   └── rich.constrain
    │   │   │       │   │   │           │   │       │       ├── rich.jupyter
    │   │   │       │   │   │           │   │       │       │   └── rich.segment
    │   │   │       │   │   │           │   │       │       └── rich.measure
    │   │   │       │   │   │           │   │       │           └── rich.protocol
    │   │   │       │   │   │           │   │       ├── rich.containers
    │   │   │       │   │   │           │   │       ├── rich.control
    │   │   │       │   │   │           │   │       └── rich.emoji
    │   │   │       │   │   │           │   ├── rich.highlighter
    │   │   │       │   │   │           │   ├── rich.markup
    │   │   │       │   │   │           │   ├── rich.pager
    │   │   │       │   │   │           │   ├── rich.pretty
    │   │   │       │   │   │           │   │   ├── attr
    │   │   │       │   │   │           │   │   │   ├── attr.converters
    │   │   │       │   │   │           │   │   │   │   ├── attr._compat
    │   │   │       │   │   │           │   │   │   │   └── attr._make
    │   │   │       │   │   │           │   │   │   │       ├── attr._config
    │   │   │       │   │   │           │   │   │   │       └── attr.setters
    │   │   │       │   │   │           │   │   │   │           └── attr.exceptions
    │   │   │       │   │   │           │   │   │   ├── attr.filters
    │   │   │       │   │   │           │   │   │   ├── attr.validators
    │   │   │       │   │   │           │   │   │   ├── attr._cmp
    │   │   │       │   │   │           │   │   │   ├── attr._funcs
    │   │   │       │   │   │           │   │   │   ├── attr._next_gen
    │   │   │       │   │   │           │   │   │   └── attr._version_info
    │   │   │       │   │   │           │   │   └── rich.abc
    │   │   │       │   │   │           │   ├── rich.region
    │   │   │       │   │   │           │   ├── rich.scope
    │   │   │       │   │   │           │   │   ├── rich.panel
    │   │   │       │   │   │           │   │   │   ├── rich.box
    │   │   │       │   │   │           │   │   │   └── rich.padding
    │   │   │       │   │   │           │   │   └── rich.table
    │   │   │       │   │   │           │   │       └── rich._ratio
    │   │   │       │   │   │           │   │           └── fractions
    │   │   │       │   │   │           │   │               └── decimal
    │   │   │       │   │   │           │   │                   └── _decimal
    │   │   │       │   │   │           │   ├── rich.screen
    │   │   │       │   │   │           │   └── rich.styled
    │   │   │       │   │   │           ├── rich.progress
    │   │   │       │   │   │           │   ├── mmap
    │   │   │       │   │   │           │   ├── rich.filesize
    │   │   │       │   │   │           │   ├── rich.live
    │   │   │       │   │   │           │   │   ├── rich.file_proxy
    │   │   │       │   │   │           │   │   │   └── rich.ansi
    │   │   │       │   │   │           │   │   └── rich.live_render
    │   │   │       │   │   │           │   ├── rich.progress_bar
    │   │   │       │   │   │           │   └── rich.spinner
    │   │   │       │   │   │           │       └── rich._spinners
    │   │   │       │   │   │           └── rich.syntax
    │   │   │       │   │   │               ├── pygments.lexer
    │   │   │       │   │   │               │   ├── pygments.filter
    │   │   │       │   │   │               │   ├── pygments.filters
    │   │   │       │   │   │               │   │   └── pygments.token
    │   │   │       │   │   │               │   └── pygments.regexopt
    │   │   │       │   │   │               ├── pygments.style
    │   │   │       │   │   │               └── pygments.styles
    │   │   │       │   │   │                   └── pygments.styles._mapping
    │   │   │       │   │   ├── huggingface_hub.utils.tqdm
    │   │   │       │   │   │   ├── tqdm
    │   │   │       │   │   │   │   ├── tqdm._monitor
    │   │   │       │   │   │   │   ├── tqdm._tqdm_pandas
    │   │   │       │   │   │   │   ├── tqdm.cli
    │   │   │       │   │   │   │   │   ├── tqdm.std
    │   │   │       │   │   │   │   │   │   └── tqdm.utils
    │   │   │       │   │   │   │   │   └── tqdm.version
    │   │   │       │   │   │   │   └── tqdm.gui
    │   │   │       │   │   │   ├── tqdm.auto
    │   │   │       │   │   │   │   ├── tqdm.autonotebook
    │   │   │       │   │   │   │   └── tqdm.asyncio
    │   │   │       │   │   │   │       └── asyncio
    │   │   │       │   │   │   │           ├── asyncio.base_events
    │   │   │       │   │   │   │           │   ├── concurrent
    │   │   │       │   │   │   │           │   ├── concurrent.futures
    │   │   │       │   │   │   │           │   │   └── concurrent.futures._base
    │   │   │       │   │   │   │           │   ├── heapq
    │   │   │       │   │   │   │           │   │   └── _heapq
    │   │   │       │   │   │   │           │   ├── asyncio.constants
    │   │   │       │   │   │   │           │   ├── asyncio.coroutines
    │   │   │       │   │   │   │           │   │   ├── asyncio.base_futures
    │   │   │       │   │   │   │           │   │   │   └── asyncio.format_helpers
    │   │   │       │   │   │   │           │   │   └── asyncio.log
    │   │   │       │   │   │   │           │   ├── asyncio.events
    │   │   │       │   │   │   │           │   │   ├── _asyncio
    │   │   │       │   │   │   │           │   │   ├── asyncio.exceptions
    │   │   │       │   │   │   │           │   │   └── asyncio.base_tasks
    │   │   │       │   │   │   │           │   ├── asyncio.futures
    │   │   │       │   │   │   │           │   ├── asyncio.protocols
    │   │   │       │   │   │   │           │   ├── asyncio.sslproto
    │   │   │       │   │   │   │           │   │   └── asyncio.transports
    │   │   │       │   │   │   │           │   ├── asyncio.staggered
    │   │   │       │   │   │   │           │   │   └── asyncio.locks
    │   │   │       │   │   │   │           │   │       ├── asyncio.mixins
    │   │   │       │   │   │   │           │   │       └── asyncio.tasks
    │   │   │       │   │   │   │           │   └── asyncio.trsock
    │   │   │       │   │   │   │           ├── asyncio.runners
    │   │   │       │   │   │   │           ├── asyncio.queues
    │   │   │       │   │   │   │           ├── asyncio.streams
    │   │   │       │   │   │   │           ├── asyncio.subprocess
    │   │   │       │   │   │   │           ├── asyncio.threads
    │   │   │       │   │   │   │           └── asyncio.unix_events
    │   │   │       │   │   │   │               ├── asyncio.base_subprocess
    │   │   │       │   │   │   │               └── asyncio.selector_events
    │   │   │       │   │   │   └── huggingface_hub.constants
    │   │   │       │   │   ├── huggingface_hub.utils._auth
    │   │   │       │   │   │   └── huggingface_hub.utils._runtime
    │   │   │       │   │   ├── huggingface_hub.utils._cache_assets
    │   │   │       │   │   ├── huggingface_hub.utils._cache_manager
    │   │   │       │   │   │   ├── huggingface_hub.utils.logging
    │   │   │       │   │   │   ├── huggingface_hub.utils._parsing
    │   │   │       │   │   │   └── huggingface_hub.utils._terminal
    │   │   │       │   │   ├── huggingface_hub.utils._chunk_utils
    │   │   │       │   │   ├── huggingface_hub.utils._datetime
    │   │   │       │   │   ├── huggingface_hub.utils._detect_agent
    │   │   │       │   │   ├── huggingface_hub.utils._experimental
    │   │   │       │   │   ├── huggingface_hub.utils._fixes
    │   │   │       │   │   │   ├── yaml
    │   │   │       │   │   │   │   ├── yaml.error
    │   │   │       │   │   │   │   ├── yaml.tokens
    │   │   │       │   │   │   │   ├── yaml.events
    │   │   │       │   │   │   │   ├── yaml.nodes
    │   │   │       │   │   │   │   ├── yaml.loader
    │   │   │       │   │   │   │   │   ├── yaml.reader
    │   │   │       │   │   │   │   │   ├── yaml.scanner
    │   │   │       │   │   │   │   │   ├── yaml.parser
    │   │   │       │   │   │   │   │   ├── yaml.composer
    │   │   │       │   │   │   │   │   ├── yaml.constructor
    │   │   │       │   │   │   │   │   └── yaml.resolver
    │   │   │       │   │   │   │   ├── yaml.dumper
    │   │   │       │   │   │   │   │   ├── yaml.emitter
    │   │   │       │   │   │   │   │   ├── yaml.serializer
    │   │   │       │   │   │   │   │   └── yaml.representer
    │   │   │       │   │   │   │   └── yaml.cyaml
    │   │   │       │   │   │   │       └── yaml._yaml
    │   │   │       │   │   │   └── filelock
    │   │   │       │   │   │       ├── filelock._api
    │   │   │       │   │   │       │   └── filelock._error
    │   │   │       │   │   │       ├── filelock._async_read_write
    │   │   │       │   │   │       │   └── filelock._read_write
    │   │   │       │   │   │       │       └── sqlite3
    │   │   │       │   │   │       │           └── sqlite3.dbapi2
    │   │   │       │   │   │       │               └── _sqlite3
    │   │   │       │   │   │       ├── filelock._soft
    │   │   │       │   │   │       │   └── filelock._util
    │   │   │       │   │   │       ├── filelock._unix
    │   │   │       │   │   │       ├── filelock._windows
    │   │   │       │   │   │       ├── filelock.asyncio
    │   │   │       │   │   │       └── filelock.version
    │   │   │       │   │   ├── huggingface_hub.utils._git_credential
    │   │   │       │   │   │   └── huggingface_hub.utils._subprocess
    │   │   │       │   │   ├── huggingface_hub.utils._headers
    │   │   │       │   │   │   └── huggingface_hub.utils._validators
    │   │   │       │   │   │       └── huggingface_hub.utils._typing
    │   │   │       │   │   ├── huggingface_hub.utils._http
    │   │   │       │   │   │   ├── uuid
    │   │   │       │   │   │   │   └── _uuid
    │   │   │       │   │   │   ├── shlex
    │   │   │       │   │   │   └── huggingface_hub.utils._lfs
    │   │   │       │   │   ├── huggingface_hub.utils._pagination
    │   │   │       │   │   ├── huggingface_hub.utils._paths
    │   │   │       │   │   ├── huggingface_hub.utils._safetensors
    │   │   │       │   │   ├── huggingface_hub.utils._telemetry
    │   │   │       │   │   │   └── queue
    │   │   │       │   │   │       └── _queue
    │   │   │       │   │   └── huggingface_hub.utils._xet
    │   │   │       │   └── transformers._typing
    │   │   │       ├── transformers.utils.import_utils
    │   │   │       └── torch
    │   │   ├── transformers.utils.chat_template_utils
    │   │   │   ├── jinja2
    │   │   │   ├── jinja2.bccache
    │   │   │   ├── jinja2.environment
    │   │   │   │   ├── markupsafe
    │   │   │   │   │   └── markupsafe._speedups
    │   │   │   │   ├── jinja2.nodes
    │   │   │   │   │   └── jinja2.utils
    │   │   │   │   ├── jinja2.compiler
    │   │   │   │   │   ├── jinja2.exceptions
    │   │   │   │   │   ├── jinja2.idtracking
    │   │   │   │   │   │   └── jinja2.visitor
    │   │   │   │   │   └── jinja2.optimizer
    │   │   │   │   ├── jinja2.defaults
    │   │   │   │   │   ├── jinja2.filters
    │   │   │   │   │   │   ├── jinja2.async_utils
    │   │   │   │   │   │   └── jinja2.runtime
    │   │   │   │   │   └── jinja2.tests
    │   │   │   │   ├── jinja2.lexer
    │   │   │   │   │   └── jinja2._identifier
    │   │   │   │   └── jinja2.parser
    │   │   │   ├── jinja2.loaders
    │   │   │   ├── jinja2.ext
    │   │   │   │   └── pprint
    │   │   │   ├── jinja2.meta
    │   │   │   ├── jinja2.sandbox
    │   │   │   ├── PIL
    │   │   │   │   └── PIL._version
    │   │   │   └── PIL.Image
    │   │   │       ├── PIL.ExifTags
    │   │   │       ├── PIL.ImageMode
    │   │   │       ├── PIL.TiffTags
    │   │   │       ├── PIL._binary
    │   │   │       ├── PIL._deprecate
    │   │   │       ├── PIL._util
    │   │   │       └── PIL._imaging
    │   │   ├── transformers.utils.constants
    │   │   ├── transformers.utils.hub
    │   │   │   ├── huggingface_hub.file_download
    │   │   │   │   ├── huggingface_hub._local_folder
    │   │   │   │   └── huggingface_hub.utils.sha
    │   │   │   │       └── huggingface_hub.utils.insecure_hashlib
    │   │   │   ├── huggingface_hub.hf_api
    │   │   │   │   ├── concurrent.futures.thread
    │   │   │   │   ├── httpcore
    │   │   │   │   │   ├── httpcore._api
    │   │   │   │   │   │   ├── httpcore._models
    │   │   │   │   │   │   └── httpcore._sync
    │   │   │   │   │   │       ├── httpcore._sync.connection
    │   │   │   │   │   │       │   ├── httpcore._backends
    │   │   │   │   │   │       │   ├── httpcore._backends.sync
    │   │   │   │   │   │       │   │   ├── httpcore._exceptions
    │   │   │   │   │   │       │   │   ├── httpcore._utils
    │   │   │   │   │   │       │   │   └── httpcore._backends.base
    │   │   │   │   │   │       │   ├── httpcore._ssl
    │   │   │   │   │   │       │   │   └── certifi
    │   │   │   │   │   │       │   │       └── certifi.core
    │   │   │   │   │   │       │   │           └── importlib.resources
    │   │   │   │   │   │       │   │               └── importlib._common
    │   │   │   │   │   │       │   │                   └── importlib._adapters
    │   │   │   │   │   │       │   ├── httpcore._synchronization
    │   │   │   │   │   │       │   │   └── anyio
    │   │   │   │   │   │       │   │       ├── anyio._core
    │   │   │   │   │   │       │   │       ├── anyio._core._contextmanagers
    │   │   │   │   │   │       │   │       ├── anyio._core._eventloop
    │   │   │   │   │   │       │   │       │   ├── anyio._core._exceptions
    │   │   │   │   │   │       │   │       │   │   └── exceptiongroup
    │   │   │   │   │   │       │   │       │   │       ├── exceptiongroup._catch
    │   │   │   │   │   │       │   │       │   │       │   └── exceptiongroup._exceptions
    │   │   │   │   │   │       │   │       │   │       ├── exceptiongroup._version
    │   │   │   │   │   │       │   │       │   │       ├── exceptiongroup._formatting
    │   │   │   │   │   │       │   │       │   │       └── exceptiongroup._suppress
    │   │   │   │   │   │       │   │       │   └── sniffio
    │   │   │   │   │   │       │   │       │       ├── sniffio._version
    │   │   │   │   │   │       │   │       │       └── sniffio._impl
    │   │   │   │   │   │       │   │       ├── anyio._core._fileio
    │   │   │   │   │   │       │   │       │   └── anyio.to_thread
    │   │   │   │   │   │       │   │       │       └── anyio.abc
    │   │   │   │   │   │       │   │       │           ├── anyio.abc._eventloop
    │   │   │   │   │   │       │   │       │           ├── anyio.abc._resources
    │   │   │   │   │   │       │   │       │           ├── anyio.abc._sockets
    │   │   │   │   │   │       │   │       │           │   ├── anyio._core._typedattr
    │   │   │   │   │   │       │   │       │           │   └── anyio.abc._streams
    │   │   │   │   │   │       │   │       │           │       └── anyio.abc._tasks
    │   │   │   │   │   │       │   │       │           ├── anyio.abc._subprocesses
    │   │   │   │   │   │       │   │       │           ├── anyio.abc._testing
    │   │   │   │   │   │       │   │       │           ├── anyio._core._synchronization
    │   │   │   │   │   │       │   │       │           │   ├── anyio.lowlevel
    │   │   │   │   │   │       │   │       │           │   ├── anyio._core._tasks
    │   │   │   │   │   │       │   │       │           │   └── anyio._core._testing
    │   │   │   │   │   │       │   │       │           └── anyio.from_thread
    │   │   │   │   │   │       │   │       ├── anyio._core._resources
    │   │   │   │   │   │       │   │       ├── anyio._core._signals
    │   │   │   │   │   │       │   │       ├── anyio._core._sockets
    │   │   │   │   │   │       │   │       │   ├── anyio.streams
    │   │   │   │   │   │       │   │       │   ├── anyio.streams.stapled
    │   │   │   │   │   │       │   │       │   └── anyio.streams.tls
    │   │   │   │   │   │       │   │       ├── anyio._core._streams
    │   │   │   │   │   │       │   │       │   └── anyio.streams.memory
    │   │   │   │   │   │       │   │       ├── anyio._core._subprocesses
    │   │   │   │   │   │       │   │       └── anyio._core._tempfile
    │   │   │   │   │   │       │   ├── httpcore._trace
    │   │   │   │   │   │       │   └── httpcore._sync.http11
    │   │   │   │   │   │       │       ├── h11
    │   │   │   │   │   │       │       │   ├── h11._connection
    │   │   │   │   │   │       │       │   │   ├── h11._events
    │   │   │   │   │   │       │       │   │   │   ├── h11._abnf
    │   │   │   │   │   │       │       │   │   │   └── h11._headers
    │   │   │   │   │   │       │       │   │   │       └── h11._util
    │   │   │   │   │   │       │       │   │   ├── h11._readers
    │   │   │   │   │   │       │       │   │   │   ├── h11._receivebuffer
    │   │   │   │   │   │       │       │   │   │   └── h11._state
    │   │   │   │   │   │       │       │   │   └── h11._writers
    │   │   │   │   │   │       │       │   └── h11._version
    │   │   │   │   │   │       │       └── httpcore._sync.interfaces
    │   │   │   │   │   │       ├── httpcore._sync.connection_pool
    │   │   │   │   │   │       ├── httpcore._sync.http_proxy
    │   │   │   │   │   │       ├── httpcore._sync.http2
    │   │   │   │   │   │       └── httpcore._sync.socks_proxy
    │   │   │   │   │   ├── httpcore._async
    │   │   │   │   │   │   ├── httpcore._async.connection
    │   │   │   │   │   │   │   ├── httpcore._backends.auto
    │   │   │   │   │   │   │   └── httpcore._async.http11
    │   │   │   │   │   │   │       └── httpcore._async.interfaces
    │   │   │   │   │   │   ├── httpcore._async.connection_pool
    │   │   │   │   │   │   ├── httpcore._async.http_proxy
    │   │   │   │   │   │   ├── httpcore._async.http2
    │   │   │   │   │   │   └── httpcore._async.socks_proxy
    │   │   │   │   │   ├── httpcore._backends.mock
    │   │   │   │   │   ├── httpcore._backends.anyio
    │   │   │   │   │   └── httpcore._backends.trio
    │   │   │   │   ├── tqdm.contrib
    │   │   │   │   ├── tqdm.contrib.concurrent
    │   │   │   │   ├── huggingface_hub._buckets
    │   │   │   │   ├── huggingface_hub._commit_api
    │   │   │   │   │   └── huggingface_hub.lfs
    │   │   │   │   ├── huggingface_hub._dataset_viewer
    │   │   │   │   ├── huggingface_hub._eval_results
    │   │   │   │   ├── huggingface_hub._inference_endpoints
    │   │   │   │   ├── huggingface_hub._jobs_api
    │   │   │   │   │   └── huggingface_hub._space_api
    │   │   │   │   ├── huggingface_hub._upload_large_folder
    │   │   │   │   ├── huggingface_hub.community
    │   │   │   │   ├── huggingface_hub.repocard_data
    │   │   │   │   ├── huggingface_hub.utils._deprecation
    │   │   │   │   ├── huggingface_hub.utils._verification
    │   │   │   │   └── huggingface_hub.utils.endpoint_helpers
    │   │   │   ├── huggingface_hub.repocard
    │   │   │   └── huggingface_hub._snapshot_download
    │   │   ├── transformers.utils.kernel_config
    │   │   └── transformers.utils.peft_utils
    │   ├── transformers.utils.versions
    │   ├── tokenizers
    │   └── accelerate
    ├── sentencepiece
    ├── mistral_common
    ├── torchvision
    ├── scipy
    ├── torchaudio
    └── timm

Total modules imported: 595
```
</details>

--- Comment by tarekziade at 2026-04-23T06:54:18Z ---
Thanks a lot! you have the "fastest" version as well at this point. I will see if we can cut more of it

--- Comment by github-actions[bot] at 2026-05-17T08:40:18Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by albertvillanova at 2026-05-18T12:30:25Z ---
I hope this issue does not get closed.


ISSUE #44265: [BUG] torch.export.export fails for models using torch_compilable_check (Mask2Former, DeformableDetr, etc.)
State: closed  |  Labels: bug
Author: harshaljanjani  |  Created: 2026-02-24T19:58:12Z

### System Info

* `transformers` version: `5.0.0.dev0`
* Platform: `Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39`
* Python version: `3.12.3`
* `huggingface_hub` version: `1.3.2`
* `safetensors` version: `0.7.0`
* `accelerate` version: `1.12.0`
* Accelerate config: `not installed`
* DeepSpeed version: `not installed`
* PyTorch version (accelerator?): `2.9.1+cu128 (CUDA)`
* GPU type: `NVIDIA L4`
* NVIDIA driver version: `550.90.07`
* CUDA version: `12.4`

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

**Deformable DETR and Mask2Former use case:**

```python
import torch
import numpy as np
from transformers import Mask2FormerForUniversalSegmentation, DeformableDetrForObjectDetection, AutoImageProcessor
from PIL import Image
import httpx
from io import BytesIO

url = "https://huggingface.co/microsoft/kosmos-2-patch14-224/resolve/main/snowman.png"
response = httpx.get(url, follow_redirects=True)
image = Image.open(BytesIO(response.content))
# mask2former
try:
    model = Mask2FormerForUniversalSegmentation.from_pretrained("facebook/mask2former-swin-tiny-ade-semantic").eval()
    image_processor = AutoImageProcessor.from_pretrained("facebook/mask2former-swin-tiny-ade-semantic")
    inputs = image_processor(image, return_tensors="pt")
    exported = torch.export.export(model, args=(inputs["pixel_values"], inputs["pixel_mask"]), strict=True)
    print(f"{len(exported.graph.nodes)} nodes")
    outputs = exported.module()(inputs["pixel_values"], inputs["pixel_mask"])
    print(f"class_queries_logits: {outputs.class_queries_logits.shape}")
    print(f"masks_queries_logits: {outputs.masks_queries_logits.shape}")
except Exception as e:
    print(f"Mask2Former FAILED: {chr(10).join(str(e).split(chr(10))[:2])}")
# deformabledetr
try:
    model = DeformableDetrForObjectDetection.from_pretrained("SenseTime/deformable-detr").eval()
    image_processor = AutoImageProcessor.from_pretrained("SenseTime/deformable-detr")
    inputs = image_processor(image, return_tensors="pt")
    exported = torch.export.export(model, args=(inputs["pixel_values"], inputs["pixel_mask"]), strict=True)
    print(f"{len(exported.graph.nodes)} nodes")
    outputs = exported.module()(inputs["pixel_values"], inputs["pixel_mask"])
    print(f"logits: {outputs.logits.shape}")
    print(f"pred_boxes: {outputs.pred_boxes.shape}")
except Exception as e:
    print(f"DeformableDetr FAILED: {chr(10).join(str(e).split(chr(10))[:2])}")
```

[torch_compilable_check](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/import_utils.py#L1439-L1442) calls `torch._check_with(error_type, cond, msg_callable)` with `ValueError` as an arg; but during strict `torch.export` tracing, Dynamo isn't in a position to convert exception types to proxy objects (`BuiltinVariable(ValueError)` has no `as_proxy()` implementation). I was able to reproduce this issue with the models [Mask2FormerDeformableAttention](https://github.com/huggingface/transformers/blob/main/src/transformers/models/mask2former/modeling_mask2former.py#L942) and [DeformableDetrMultiscaleDeformableAttention](https://github.com/huggingface/transformers/blob/main/src/transformers/models/deformable_detr/modeling_deformable_detr.py#L576) and this should be echoed with others that use `torch_compilable_check` fn from `import_utils.py`.

**Current repro output with concise error messages:**

**Complete error message:**

### Expected behavior

→ Models using `torch_compilable_check` should export successfully with `torch.export.export(strict=True)`.

**Output after the fix:**


ISSUE #44263: The torch.split() return values in GlmMoeDsaIndexer
State: closed  |  Labels: bug
Author: Jintao-Huang  |  Created: 2026-02-24T16:32:23Z

### System Info

transformers:

https://github.com/huggingface/transformers/blob/e2bc54f29a58b2d2ee7e7d6eac949c959e063e0f/src/transformers/models/glm_moe_dsa/modular_glm_moe_dsa.py#L515

vllm:

https://github.com/vllm-project/vllm/blob/a0c70816956298f7dd1d0cf47cfa1a169a413692/vllm/model_executor/models/deepseek_v2.py#L746

deepseek_v3.2

https://github.com/deepseek-ai/DeepSeek-V3.2-Exp/blob/main/inference/model.py#L462

### Who can help?

-

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

-

--- Comment by Jintao-Huang at 2026-02-24T16:33:17Z ---
sorry, I read the wrong line

https://github.com/huggingface/transformers/blob/e2bc54f29a58b2d2ee7e7d6eac949c959e063e0f/src/transformers/models/glm_moe_dsa/modular_glm_moe_dsa.py#L373


ISSUE #44262: from_pretrained no longer uses mmap for CPU weights in transformers 5.x causing full materialization
State: closed  |  Labels:
Author: nikitas-cerebras  |  Created: 2026-02-24T16:20:18Z

I have the following piece of code:

```python
import psutil
from transformers import AutoModel

rss_before = psutil.Process().memory_info().rss
model = AutoModel.from_pretrained(
    "Qwen/Qwen3-Coder-30B-A3B-Instruct",
    dtype="auto",
    device_map="auto",
    max_memory={"cpu": "1024GB", 0: 0},
)
rss_after = psutil.Process().memory_info().rss
print(f"Memory: {(rss_after - rss_before) >> 20} MiB")
```

When running it on transformers 4.57, I get:
```
Loading checkpoint shards: 100%|██████████| 16/16 [00:00<00:00, 17.03it/s]
Memory: 1321 MiB
```

When running it on transformers 5.2, I get:
```
Loading weights: 100%|██████████| 530/530 [00:13<00:00, 39.48it/s, Materializing param=norm.weight]
Memory: 110705 MiB
```

I see that the mode loading was reworked. It seems like on the old transformers version model weights are loaded with memory mapping, while on the new transformers version all the weights are materialized.

I wonder how to achieve the old behavior with the new transformers?

--- Comment by Rocketknight1 at 2026-02-25T13:05:34Z ---
cc @Cyrilvallez I think!

--- Comment by NielBuys at 2026-02-25T14:30:03Z ---
Hi everyone

Is this potentially the same issue causing 4-bit quantization spikes?

I'm seeing what looks like the same materialization behavior when attempting to load Gemma 3 12B with 4-bit quantization in Transformers v5.1.0. Even with device_map="auto", the model appears to materialize at full precision before the quantizer can act, leading to immediate OOM.

I’ve documented the logs and a detailed breakdown of the failure here: [[Link to forum post]](https://discuss.huggingface.co/t/gemma-3-12b-4-bit-quantization-failing-ignored-in-transformers-v5-1-0-gemma3forconditionalgeneration/173278/10)

--- Comment by NielBuys at 2026-02-25T14:42:17Z ---
Maybe another issue with the same problem https://github.com/huggingface/transformers/issues/43749

--- Comment by Cyrilvallez at 2026-03-04T16:14:27Z ---
What you are seeing is merely the fact that `safetensors` mmaps the tensors (even though we used to, and still do, explicitly slice the safetensors slice objects to materialize them with `t = t[...]`).
However, almost all operation that need to access them and/or copy them will then put them into memory. For example, if you try to use `dtype=torch.float16` in your example, it will force a copy with the new dtype, and the memory will become the full model. Now, since MoE models have automatic weight conversions to a new format since v5, we need to copy the weights at load time no matter what, so they end up materializing the memory immediately.
Note that you are NOT saving any memory, i.e. that memory WILL be materialized when running a `forward` anyway.

Consider the following example with Llama, which does not need any weight conversion:

```python
import psutil
import torch
from transformers import AutoModel

model_id = "meta-llama/Llama-3.3-70B-Instruct"
# model_id = "Qwen/Qwen3-Coder-30B-A3B-Instruct"

rss_before = psutil.Process().memory_info().rss / 1024**3
model = AutoModel.from_pretrained(
    model_id,
    dtype="auto",
    # dtype=torch.float16,
)
rss_after = psutil.Process().memory_info().rss / 1024**3
print(f"Memory: {(rss_after - rss_before):.2f} GiB")

tot = 0
for k,v in model.state_dict().items():
    tot += v.element_size() * v.nelement()
print(f"Total model size: {tot / 1024**3:.2f} GiB")

rss_before = psutil.Process().memory_info().rss / 1024**3
print(f"Memory before forward: {(rss_before):.2f} GiB")
foo = torch.randint(100, 200, (1, 20))
_ = model(foo)
rss_after = psutil.Process().memory_info().rss / 1024**3
print(f"Memory after forward: {(rss_after):.2f} GiB")
```

It prints:

```sh
Memory: 0.08 GiB
Total model size: 129.46 GiB
Memory before forward: 0.75 GiB
Memory after forward: 128.91 GiB
```

Now, if you change `dtype="auto"` -> `dtype=torch.float16` (not native dtype, so requires a copy), it prints:

```sh
Memory: 129.51 GiB
Total model size: 129.46 GiB
Memory before forward: 130.19 GiB
Memory after forward: 131.04 GiB
```

And you can notice that everything was materialized since the beginning.

Now, for Qwen it's a bit more particular, as it's a MoE. So before v5 and automatic weight conversion, if you don't change the dtype it would be mmaped, and after forward you would only materialize the experts you used during that forward, resulting in less memory than the full model size. However, as you keep calling `forward`, you will eventually hit all experts, and then memory consumption is the same.

Now, after all of those explanations, Qwen memory consumption in v5 is still too high conmpared to shat it should be if loading the model immediately with e.g. a non-native dtype - I will investigate to see if some memory is wrongly not released as it should!

--- Comment by nikitas-cerebras at 2026-03-05T12:42:57Z ---
@Cyrilvallez Thank you for the explanation! In my particular case I perform some kind of model analysis and don't actually call a model's forward that's why for me this difference is crucial.

I guess **in theory** it should be possible to postpone merging of 2D MoE weights into a 3D tensor until the actual inference. And from the blogpost https://huggingface.co/blog/moe-transformers I was getting an idea that that could be the case based on the quote "_For example, MergeModulelist waits until all experts for a layer are loaded._" under the "_Lazy Materialization of Tensors_" section. But perhaps it means something else. Anyway thanks to your explanation I now understand that there is no such logic.

For anyone having the same issues, I was able to transition to loading the model as:
```python
config = AutoConfig.from_pretrained("Qwen/Qwen3-Coder-30B-A3B-Instruct")
with init_empty_weights(include_buffers=False):
    model = AutoModelForCausalLM.from_config(config)
)
```


ISSUE #44261: [Bug/Discussion] MLA q_a_layernorm Missing config.rms_norm_eps, Causing 1e-5/1e-6 Precision Error
State: closed  |  Labels: bug
Author: Jintao-Huang  |  Created: 2026-02-24T16:14:57Z

### System Info

hello! I noticed that the MLA implementations in transformers/vllm/sglang/megatron have slight differences, leading to precision errors (train/infer/rl...)

vllm:
https://github.com/vllm-project/vllm/blob/a0c70816956298f7dd1d0cf47cfa1a169a413692/vllm/model_executor/models/deepseek_v2.py#L907

sglang:
https://github.com/sgl-project/sglang/blob/e6ad58e5daa1476544f813da93f1f2f5078d387f/python/sglang/srt/models/deepseek_v2.py#L1129

transformers:
https://github.com/huggingface/transformers/blob/e2bc54f29a58b2d2ee7e7d6eac949c959e063e0f/src/transformers/models/deepseek_v3/modular_deepseek_v3.py#L192

### Who can help?

-

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

-

--- Comment by Jintao-Huang at 2026-02-24T16:16:32Z ---
In the transformers implementation, DeepseekV3RMSNorm does not receive rms_norm_eps, so it defaults to 1e-6. However, config.json specifies 1e-5. Not sure which implementation is correct...

--- Comment by vasqu at 2026-02-24T17:07:50Z ---
Seems like the remote code at least goes with not passing the eps as well, i.e. https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py#L664

Maybe someone on the vLLM side has some insights cc @hmellor

--- Comment by hmellor at 2026-02-26T12:50:47Z ---
Which checkpoint specifies `1e-5`? In DeepSeek v3 I see `1e-6` https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/config.json#L46

--- Comment by Jintao-Huang at 2026-03-05T08:21:17Z ---
sorry...

https://huggingface.co/zai-org/GLM-5/blob/main/config.json#L44

https://github.com/huggingface/transformers/blob/e2bc54f29a58b2d2ee7e7d6eac949c959e063e0f/src/transformers/models/glm_moe_dsa/modeling_glm_moe_dsa.py#L49

--- Comment by hmellor at 2026-03-05T08:23:40Z ---
We are discussing this on the vLLM side here https://github.com/vllm-project/vllm/pull/35515

We also realised that this is mainly an issue for GLM models and have reached out to someone on the GLM team for comment

--- Comment by mvanhorn at 2026-03-11T00:21:00Z ---
I've submitted a fix for this in PR #44585. Added `eps=config.rms_norm_eps` to both `q_a_layernorm` and `kv_a_layernorm` in the MLA attention module to align with vLLM/SGLang implementations.

--- Comment by hmellor at 2026-03-12T10:12:43Z ---
It's still up for debate who is correct.

- Transformres has used the same implementation as in the code that DeepSeek originally wrote
- vLLM has added `eps=config.rms_norm_eps` because it seemed sensible and is a no-op for DeepSeek checkpoints

We still need feedback from the GLM team before we can decide.

--- Comment by github-actions[bot] at 2026-04-06T08:28:14Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by selalipop at 2026-04-07T05:49:33Z ---
Was GLM able to clarify?

--- Comment by vasqu at 2026-04-09T16:05:34Z ---
Gentle ping @zRzRzRzRzRzRzR @JaredforReal if you have some more insights

--- Comment by github-actions[bot] at 2026-05-04T08:46:55Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


[Bug] Security Vulnerability

ISSUE #44248: [Bug] Security Vulnerability
State: closed  |  Labels:
Author: 0xManan  |  Created: 2026-02-24T02:13:45Z

I have reported ReDos vulnerability on [Huntr](https://huntr.com/bounties/c93d804a-fa03-4c94-aa29-b83a1eff9499). It's a new issue which hasn't been fixed yet but Huntr's platform bot has marked it as duplicate of some 2024 report which is not relevant to current regex and file. Can you please re-validate it and verify from your end. @Michellehbn

--- Comment by 0xManan at 2026-02-24T12:47:06Z ---
Hey @Rocketknight1 , Can you let me know the reason of closing this issue?


ISSUE #44247: [MPS] Silent correctness issue in bidirectional attention
State: open  |  Labels: WIP, bug
Author: hvaara  |  Created: 2026-02-24T00:08:02Z

### System Info

A bug in PyTorch for the MPS backend (pytorch/pytorch#174861) results in a silent correctness issue in bidirectional attention under certain conditions:

- dtype != `torch.float` (eg. `float16` or `bfloat16`)
- non-masked or boolean mask
- non-causal
- query sequence length <= 8
- query sequence length <= key sequence length
- query head dim = value head dim
- query head dim in {64, 96, 128}
- key sequence length >= 1024 OR (number of key heads < number of query heads AND key sequence length >= 4096)

This is the case for several models in transformers. For example GLM-Image, but I've seen it in other vision and TTS models too, but I can't remember which ones top of mind.

The issue has been in PyTorch since `v2.8.0` released 2025-08-06. I've submitted a fix to PyTorch (pytorch/pytorch#174945), which is on track to make it in the `v2.11.0` release of PyTorch.

I recommend that we add a workaround for versions between `v2.7.1` and `v2.11.0` exclusive. For example by upcasting the qkv tensors to fp32 when calling sdpa under the conditions outlined above. The source for these conditions can be found in the PyTorch source. The issue is in `sdpa_vector_2pass_mps`, so conditions that takes the other code path through `sdpa_general_mps` would lead to correct output.

https://github.com/pytorch/pytorch/blob/07180141f03510ab0b53dd0a544b71a5d5bdba93/aten/src/ATen/native/mps/operations/Attention.mm#L488

I'm happy to provide a PR once we're aligned on mitigation steps.

### Who can help?

@vasqu @ArthurZucker @CyrilVallez @ivarflakstad

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

#### MRE

```python
import torch
from diffusers.pipelines.glm_image import GlmImagePipeline

pipe = GlmImagePipeline.from_pretrained("zai-org/GLM-Image", torch_dtype=torch.bfloat16, device_map="mps")
prompt = "A beautifully designed modern food magazine style dessert recipe illustration, themed around a raspberry mousse cake. The overall layout is clean and bright, divided into four main areas: the top left features a bold black title 'Raspberry Mousse Cake Recipe Guide', with a soft-lit close-up photo of the finished cake on the right, showcasing a light pink cake adorned with fresh raspberries and mint leaves; the bottom left contains an ingredient list section, titled 'Ingredients' in a simple font, listing 'Flour 150g', 'Eggs 3', 'Sugar 120g', 'Raspberry puree 200g', 'Gelatin sheets 10g', 'Whipping cream 300ml', and 'Fresh raspberries', each accompanied by minimalist line icons (like a flour bag, eggs, sugar jar, etc.); the bottom right displays four equally sized step boxes, each containing high-definition macro photos and corresponding instructions, arranged from top to bottom as follows: Step 1 shows a whisk whipping white foam (with the instruction 'Whip egg whites to stiff peaks'), Step 2 shows a red-and-white mixture being folded with a spatula (with the instruction 'Gently fold in the puree and batter'), Step 3 shows pink liquid being poured into a round mold (with the instruction 'Pour into mold and chill for 4 hours'), Step 4 shows the finished cake decorated with raspberries and mint leaves (with the instruction 'Decorate with raspberries and mint'); a light brown information bar runs along the bottom edge, with icons on the left representing 'Preparation time: 30 minutes', 'Cooking time: 20 minutes', and 'Servings: 8'. The overall color scheme is dominated by creamy white and light pink, with a subtle paper texture in the background, featuring compact and orderly text and image layout with clear information hierarchy."
image = pipe(
    prompt=prompt,
    height=32 * 32,
    width=36 * 32,
    num_inference_steps=50,
    guidance_scale=1.5,
    generator=torch.Generator(device="mps").manual_seed(42),
).images[0]

image.save("output_t2i.png")
```

With PyTorch `v2.10.0` results in the following Python stacktrace

```
File ~/dev/huggingface/transformers/src/transformers/generation/utils.py:2884, in GenerationMixin._sample(self, input_ids, logits_processor, stopping_criteria, generation_config, synced_gpus, streamer, **model_kwargs)
   2882     probs = nn.functional.softmax(next_token_scores, dim=-1)
   2883     # TODO (joao): this OP throws "skipping cudagraphs due to ['incompatible ops']", find solution
-> 2884     next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
   2885 else:
   2886     next_tokens = torch.argmax(next_token_scores, dim=-1)

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
```

### Expected behavior

I expected there to be no error and that the output image is generated successfully.

--- Comment by vasqu at 2026-02-24T10:58:59Z ---
Iiuc, then all the listed conditions must apply. Would this mean that if we were to never skip the mask creation, that we wouldn't encounter this issue?

Casting from half to full (and back) sounds quite expensive to me so I'd like to avoid it if possible. And MPS backend does not benefit from anything special between fp32 and half I assume or using no mask?

--- Comment by hvaara at 2026-02-24T11:28:03Z ---
> Iiuc, then all the listed conditions must apply.

Yes

> Would this mean that if we were to never skip the mask creation, that we wouldn't encounter this issue?

Yes, as long as it's not a bool mask,

https://github.com/pytorch/pytorch/blob/07180141f03510ab0b53dd0a544b71a5d5bdba93/aten/src/ATen/native/mps/operations/Attention.mm#L461-L462

would become false, and then you end up going down `sdpa_general_mps`, which is not affected by the correctness issue (although much slower).

> Casting from half to full (and back) sounds quite expensive to me so I'd like to avoid it if possible.

Attention is quite expensive in general on MPS. `sdpa_vector_2pass_mps` is about 40% faster than `sdpa_general_mps`, based on benchmarking stats from the PR that introduced the Metal kernels. I'm unsure what the perf impact would be of running this in fp32 vs using `sdpa_general_mps`. It's possible, but I'm not sure, it would actually be faster to upcast to fp32 and go down `sdpa_vector_2pass_mps`. Perhaps a good candidate for some quick benchmarking.

> And MPS backend does not benefit from anything special between fp32 and half I assume or using no mask?

Non-masked or bool mask is a prereq to use the Metal kernels (which are a lot faster as I mentioned above). Both Metal and MPS kernels support fp32/fp16/bf16.

--- Comment by vasqu at 2026-02-24T12:43:38Z ---
Any memory impacts we could observe on the upcast. In general, I'm up for workarounds as long as we show (benchmark) which one is the better one, speed and memory-wise:
- Non-boolean mask forcing
- Fp32 upcasting

At least these seem to be the current viable options. Fp32 upcasting is potentially expensive on the memory side, should have clarified a bit more.

--- Comment by Cyrilvallez at 2026-03-02T13:04:07Z ---
I think forcing a mask is much simpler on our side. Upcasting to fp32 would require manual tweaks in every modeling code, whereas forcing mask creation can be simply done in the masking API (though it adds yet another code paths due to broen 3rd party unfortunately 🥲)

--- Comment by vasqu at 2026-03-02T13:08:59Z ---
I think it could live in the sdpa integration for upcasting but we definitely have less coverage for the attention interface than the masks 👀

--- Comment by hvaara at 2026-03-02T13:19:49Z ---
Forcing a mask seems like the better option to me too, especially considering tight memory constraints on consumer hardware. Adding a warning might also be beneficial so those affected have a nag to raise awareness of the issue and resolution steps (update PyTorch).

--- Comment by vasqu at 2026-03-02T18:52:59Z ---
Do you want to open a PR @hvaara then? You essentially need to fixup 2 things for this
- https://github.com/huggingface/transformers/blob/41888df59eed5379467262000a4f68493d3fbced/src/transformers/masking_utils.py#L994 --> force eager interface
- https://github.com/huggingface/transformers/blob/41888df59eed5379467262000a4f68493d3fbced/src/transformers/masking_utils.py#L997 --> force `allow_xxx_skip=False`

I just picked the bidirectional mask but would need to happen on every top level mask fn. Hard for me to do a PR because I don't use a Mac 😓

--- Comment by hvaara at 2026-03-02T19:02:21Z ---
Sure, I can take a look. I'll loop you in on the review if you don't mind 😃

--- Comment by vasqu at 2026-03-02T19:03:48Z ---
Sounds good!

--- Comment by hvaara at 2026-03-08T23:18:13Z ---
Looks like at least GLM-Image doesn't use the masking API. Would it be better to add the workaround directly in the attention interface (in `src/transformers/integrations/sdpa_attention.py`)? What's the preferred approach?

--- Comment by vasqu at 2026-03-09T13:54:26Z ---
> Looks like at least GLM-Image doesn't use the masking API

Argh, yea the vision portions don't use a mask (so pretty much any of the VLMs of qwen, zai, and baidu). Could we not simply create a mask with `torch.zeros` in that case? To cover as much as we can, we need to
1. Force mask creations where we can, i.e. in the mask API
2. In the sdpa integration, create a zeros mask if mps x affected torch versions x no mask

It's a bit of a rubber band tbh, but we don't have much choice. Case 1 should cover almost anything and case 2 for the special VLM cases. But @Cyrilvallez wdyt? Maybe you have a different idea

--- Comment by Cyrilvallez at 2026-03-11T09:45:31Z ---
If we need to do 2. anyway, I'd say let's only do 2. then no? That way we keep it confined to the sdpa integration. Even a single function for both linkes issues with mps

--- Comment by hvaara at 2026-03-11T13:04:42Z ---
Since it was not clear in #44359 and #44591: I am already working on this issue.

--- Comment by vasqu at 2026-03-11T13:29:04Z ---
> If we need to do 2. anyway, I'd say let's only do 2. then no? That way we keep it confined to the sdpa integration. Even a single function for both linkes issues with mps

Yep, it was before we had the second issue hit us 😢

> Since it was not clear in https://github.com/huggingface/transformers/pull/44359 and https://github.com/huggingface/transformers/pull/44591: I am already working on this issue.

Yep don't worry, we will keep an eye out for your PR. A lot of coding agents just randomly pick up issues/prs 😅

--- Comment by Cyrilvallez at 2026-03-12T10:41:59Z ---
Yeah no worries @hvaara, we will always prioritize the initial issue raiser if they are willing to contribute, then us, then all the random agent prs 😁

--- Comment by michaelanderson01826-glitch at 2026-03-12T16:06:15Z ---
Yes we'll do provide a fix soon

On Thu, Mar 12, 2026, 10:42 AM Cyril Vallez ***@***.***>
wrote:

> *Cyrilvallez* left a comment (huggingface/transformers#44247)
> <https://github.com/huggingface/transformers/issues/44247#issuecomment-4045704951>
>
> Yeah no worries @hvaara <https://github.com/hvaara>, we will always
> prioritize the initial issue raiser if they are willing to contribute, then
> us, then all the random agent prs 😁
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/44247#issuecomment-4045704951>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/B7HTEELZKMHFMT6NCBSO7PD4QKIBJAVCNFSM6AAAAACV5KSGA6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DANBVG4YDIOJVGE>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>

--- Comment by github-actions[bot] at 2026-04-06T08:28:16Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by hvaara at 2026-04-06T17:00:39Z ---
I'll get around to finish it this week.