ISSUE #44246: import transformers takes long sometimes
State: closed  |  Labels: bug
Author: audricschiltknecht  |  Created: 2026-02-23T22:05:01Z

### System Info

- `transformers` version: 5.2.0
- Platform: macOS-15.7.3-arm64-arm-64bit-Mach-O
- Python version: 3.13.11
- Huggingface_hub version: 1.4.1
- Safetensors version: 0.7.0
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): not installed (NA)
- Using distributed or parallel set-up in script?: Unknown

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Use the following script:
```python
from datetime import datetime
print("Before importing files...")
start = datetime.now()
import transformers
delta = datetime.now() - start
print(f"Import done: {delta.seconds}.{delta.microseconds} seconds")
```

You get:

```
Before importing files...
PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Import done: 10.512374 seconds
```

### Expected behavior

Relatively small import time:

```
Before importing files...
PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Import done: 0.836140 seconds
```

--- Comment by audricschiltknecht at 2026-02-23T22:05:34Z ---
This is really similar to #16863, except I reproduce the issue in a brand new venv with only transformers installed.

--- Comment by audricschiltknecht at 2026-02-23T22:08:02Z ---
Here is an extract of a cprofile's profile:

```
>>> import pstats
>>> p = pstats.Stats("transformers-5.2.0.profile")
>>> p.strip_dirs().sort_stats('cumtime').print_stats(.1)
Mon Feb 23 17:00:31 2026    transformers-5.2.0.profile

         1718741 function calls (1699660 primitive calls) in 11.349 seconds

   Ordered by: cumulative time
   List reduced from 3195 to 320 due to restriction <0.1>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       54    0.004    0.000   21.741    0.403 __init__.py:1(<module>)
    621/1    0.053    0.000   11.350   11.350 {built-in method builtins.exec}
        1    0.000    0.000   11.350   11.350 reproduce.py:1(<module>)
    608/2    0.006    0.000   11.350    5.675 <frozen importlib._bootstrap>:1349(_find_and_load)
    584/2    0.003    0.000   11.350    5.675 <frozen importlib._bootstrap>:1304(_find_and_load_unlocked)
    554/2    0.003    0.000   11.349    5.674 <frozen importlib._bootstrap>:911(_load_unlocked)
    513/2    0.003    0.000   11.349    5.674 <frozen importlib._bootstrap_external>:1017(exec_module)
   1284/4    0.002    0.000   11.341    2.835 <frozen importlib._bootstrap>:480(_call_with_frames_removed)
        1    0.000    0.000    7.401    7.401 import_utils.py:2736(define_import_structure)
    425/1    0.178    0.000    7.365    7.365 import_utils.py:2411(create_import_structure_from_path)
     1400    6.203    0.004    6.211    0.004 {built-in method _io.open}
   509/19    0.002    0.000    3.871    0.204 <frozen importlib._bootstrap>:1390(_handle_fromlist)
    449/6    0.001    0.000    3.871    0.645 {built-in method builtins.__import__}
        1    0.000    0.000    3.828    3.828 dependency_versions_check.py:1(<module>)
        1    0.000    0.000    3.318    3.318 auto_docstring.py:1(<module>)
        1    0.000    0.000    3.196    3.196 generic.py:1(<module>)
      513    0.007    0.000    2.309    0.005 <frozen importlib._bootstrap_external>:1090(get_code)
      513    0.003    0.000    2.183    0.004 <frozen importlib._bootstrap_external>:1211(get_data)
      513    2.128    0.004    2.128    0.004 {built-in method _io.open_code}
        2    0.000    0.000    1.994    0.997 logging.py:1(<module>)
        3    0.000    0.000    1.349    0.450 errors.py:1(<module>)
        2    0.000    0.000    0.814    0.407 _main.py:1(<module>)
        3    0.000    0.000    0.765    0.255 _api.py:1(<module>)
     1253    0.340    0.000    0.680    0.001 import_utils.py:2369(fetch__all__)
        2    0.000    0.000    0.603    0.301 _auth.py:1(<module>)
        1    0.000    0.000    0.594    0.594 _client.py:1(<module>)
      554    0.001    0.000    0.545    0.001 <frozen importlib._bootstrap>:806(module_from_spec)
       37    0.000    0.000    0.529    0.014 <frozen importlib._bootstrap_external>:1315(create_module)
       37    0.528    0.014    0.528    0.014 {built-in method _imp.create_dynamic}
7556/6227    0.003    0.000    0.424    0.000 {built-in method builtins.hasattr}
        1    0.000    0.000    0.421    0.421 hub.py:1(<module>)
```

seems a lot of time is spent in the `define_import_structure` and `create_import_structure_from_path` methods.

--- Comment by Rocketknight1 at 2026-02-24T13:06:11Z ---
Hi @audricschiltknecht, it's not as slow on my machine, so it's trickier for my to benchmark. However, I made some optimizations which should help. Can you try installing from the PR branch with `pip install git+https://github.com/huggingface/transformers.git@faster_create_import_structure_from_path` and let me know if it fixes things?

--- Comment by audricschiltknecht at 2026-02-24T15:50:46Z ---
Thanks for the patch @Rocketknight1
Unfortunately, it does not seem to have improved by a lot:
```
(test-transformers) ~/test-transformers (master)> uv pip install git+https://github.com/huggingface/transformers.git@faster_create_import_structure_from_path
    Updated https://github.com/huggingface/transformers.git (375a659370db5dbf3bf8440fa150e2c8962ccc38)
Resolved 28 packages in 6m 04s
      Built transformers @ git+https://github.com/huggingface/transformers.git@375a659370db5dbf3bf8440fa150e2c8962ccc38
Prepared 1 package in 28.02s
Uninstalled 1 package in 580ms
Installed 1 package in 32ms
 - transformers==5.2.0
 + transformers==5.3.0.dev0 (from git+https://github.com/huggingface/transformers.git@375a659370db5dbf3bf8440fa150e2c8962ccc38)
(test-transformers) ~/test-transformers (master)> python3 -m cProfile -o transformers-faster_create_import_structure_from_path.profile reproduce.py
Before importing files...
PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Import done: 9.308399 seconds
```

--- Comment by Rocketknight1 at 2026-02-24T16:15:43Z ---
Hi @audricschiltknecht, can you give me a bit more detail about the hardware? Also, is transformers installed on a non-SSD hard disc, or a network drive? This section of the codebase does a lot of file scanning, but it usually completes quickly in my testing.

--- Comment by audricschiltknecht at 2026-02-24T18:54:10Z ---
It's a MacBook Pro M4, running Sequoia. `transformers` is installed on the built-in SSD.
However, it is a corporate-provided laptop, so I'm wondering if there are any monitoring softwares that could cause the issue.
I'm looking into the tracing/instrumentation capabilities of MacOS to see if it's possible to get more data, I'll keep you posted.

--- Comment by Rocketknight1 at 2026-02-25T13:45:34Z ---
@audricschiltknecht one other thing that would be very helpful would be if you could run a `git bisect` with `pip install -e` to identify the commit where this slowdown first occurs. This would give us a lot of info about the cause!

--- Comment by audricschiltknecht at 2026-02-27T21:25:42Z ---
@Rocketknight1 so I'm not 100% sure due to the fact that it takes a while to reproduce the issue, but the bisect points 54a123f068c57abe8bc27a507d05d5674f5862bf as the commit with the first observable slowdown.
Given the commit had a pretty heavy refactor of the `_init__.py` file, it still not a completely frivolous possibility.

Before that, I'm seeing import time of max 2-3sec, and after that it jumps to 7-11sec.

--- Comment by LysandreJik at 2026-03-02T15:05:31Z ---
Hey @audricschiltknecht, thanks a lot for your work on this issue.

A few clarification questions:
- Are you using ipython in your example?
- If yes, do you observe the same slowdowns within non-ipython runtimes?
- If yes, can you let me know if running `%config IPCompleter.use_jedi=False` before anything else changes anything?
- If yes, can you also let me know if uninstalling `jedi` solves this?

Thanks for your help!

--- Comment by audricschiltknecht at 2026-03-02T16:07:09Z ---
I'm using cpython directly:
> Python 3.14.2 (main, Jan 13 2026, 19:27:04) [Clang 21.1.4 ]

--- Comment by github-actions[bot] at 2026-03-27T08:12:03Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44242: Load balancing loss not added when output_router_logits=False
State: closed  |  Labels: bug
Author: Matheart  |  Created: 2026-02-23T20:07:48Z

### System Info

version:4.57.3
In file `models/mixtral/modelling_mixtral.py`, the `aux_loss` is not computed and added to the overall loss, when `output_router_logits=False` in the `MixtralConfig`.

This is not intended, since according to the documentation https://huggingface.co/docs/transformers/en/model_doc/mixtral, the auxillary loss should be added as long as `router_aux_loss_coef != 0`. Thus by default `output_router_logits=False`, the model is not by default doing load balancing even when we set a nonzero `router_aux_loss_coef`.

### Who can help?

@SunMarc @ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```py
from transformers import MixtralConfig, MixtralForCausalLM
import torch

# 1. Configure the model to output router logits
config = MixtralConfig(
    vocab_size=32000,
    hidden_size=2048,
    num_hidden_layers=2, # small for demonstration
    num_local_experts=8,
    output_router_logits=False,
    router_aux_loss_coef=0.001   # The scaling factor for the load balancing loss
)

# 2. Initialize the model
model = MixtralForCausalLM(config)

# 3. Create dummy inputs and labels for a training step
input_ids = torch.tensor([[1, 254, 99, 32]])
labels = torch.tensor([[1, 254, 99, 32]]) # Next-token prediction labels

# 4. Perform the forward pass
outputs = model(input_ids=input_ids, labels=labels)

# 5. Read the losses
total_loss = outputs.loss
aux_loss = outputs.aux_loss
router_logits = outputs.router_logits # Tuple of router logits for each layer

print(f"Auxiliary Load Balancing Loss: {aux_loss.item():.4f}")
print(f"Total Loss (Cross Entropy + {config.router_aux_loss_coef} * Aux Loss): {total_loss.item():.4f}")
```

### Expected behavior

```
    print(f"Auxiliary Load Balancing Loss: {aux_loss.item():.4f}")
                                            ^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'item'
```
It can output the loss only when `output_router_logits=True`.

--- Comment by vasqu at 2026-02-25T16:05:00Z ---
> This is not intended, since according to the documentation https://huggingface.co/docs/transformers/en/model_doc/mixtral, the auxillary loss should be added as long as router_aux_loss_coef != 0.

Do you have a more explicit pointer where you found this? When I check in the config it is even explicitly stated here that we need `output_router_logits=True` https://huggingface.co/docs/transformers/en/model_doc/mixtral#transformers.MixtralConfig.output_router_logits.

I opened a PR over here #44264 nonetheless. It seems sensible to me that users expect the loss to be properly used during training; during eval/inference it makes less sense to me

--- Comment by Matheart at 2026-02-27T20:30:38Z ---
So the documentation is `output_router_logits (bool, optional, defaults to False) — Whether or not the router logits should be returned by the model. Enabling this will also allow the model to output the auxiliary loss. See [here](https://huggingface.co/docs/transformers/en/model_doc/mixtral) for more details`, I think it does not specify that this parameter must set to be true, if we want to use the auxillary loss in training.

--- Comment by vasqu at 2026-03-02T12:08:28Z ---
> Enabling this will also allow the model to output the auxiliary loss.

Isn't this exactly what it specifies here? You need to enable this (i.e. set to true) to enable the loss calculation.

--- Comment by Matheart at 2026-03-02T14:27:06Z ---
It is subject to interpretation? For example: The case when the user wants to include proxy loss in loss calculation but don't want to include it as output

--- Comment by vasqu at 2026-03-02T14:42:38Z ---
> Enabling this will also allow the model to output the auxiliary loss

Should this rephrased to "Enabling this will also allow the model to calculate the auxiliary loss"? I think the `output` usage is what is confusing iiuc

--- Comment by Matheart at 2026-03-03T03:00:05Z ---
Maybe better to add a remark that to enable load balancing (I.e. involve proxy loss into loss calculation), one should set that parameter to be true

--- Comment by vasqu at 2026-03-03T08:25:49Z ---
Would you be interested in opening a PR for that? You can ping me and @stevhliu in that one. Glad to avoid any confusion on the docs

--- Comment by mvanhorn at 2026-03-11T00:24:16Z ---
I've submitted a fix for this in PR #44586. Decoupled router logits collection from output visibility so aux_loss is always computed when router_aux_loss_coef > 0, regardless of the output_router_logits setting.

--- Comment by github-actions[bot] at 2026-04-04T08:09:09Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44238: CI: failing slow runs fails to report properly
State: closed  |  Labels: bug
Author: tarekziade  |  Created: 2026-02-23T17:58:30Z

The following issue description was a red herring, the real problem is described at https://github.com/huggingface/transformers/issues/44238#issuecomment-3946684226

Slow runs try to cat `captured_info.txt` which seems to not be always present

https://github.com/huggingface/transformers/pull/43972#issuecomment-3945840080

the GH step that update the report fails after the runs finish.

I reproduced the issue in SSH with

```
PATCH_TESTING_METHODS_TO_COLLECT_OUTPUTS=yes _PATCHED_TESTING_METHODS_OUTPUT_DIR=. pytest -sxv tests/models/qwen3_vl/test_modeling_qwen3_vl.py::Qwen3VLModelTest::test_multi_gpu_data_parallel_forward --make-reports reports
```

and I can see that it does not hit the function that creates the file, most likely because it does not go through a patched assertion = here its `StopIteration`

maybe we could simply touch the file when the tests starts, or make the step more resilient in the absence of it.

--- Comment by tarekziade at 2026-02-23T17:59:08Z ---
cc @ydshieh

--- Comment by tarekziade at 2026-02-23T18:57:31Z ---
That error was a red herring, the real issue seems to be in  `check_bad_commit.py`

It ran for 4 hours (12:29:27 → 16:29:28) , running git bisect — then the ls in "Show results" failed with exit code 2 because the output file was never written.

I think the root cause is a bug in check_bad_commit.py at line 234:

```
return None, "git bisect failed"   # ← returns a TUPLE, not a dict!
```

But the caller at line 360 does:

```
info.update(bad_commit_info)  # bad_commit_info = (None, "git bisect failed")
```

`dict.update()` tries to iterate over `(None, "git bisect failed")` as (key, value) pairs. The first element is None, which can't be unpacked → TypeError: cannot unpack non-iterable NoneType object.

The script crashes, never writes the output file, the ls fails with exit code 2, and the whole chain collapses to "Model CI failed to report results".

maybe we could fix with

```diff
 231      if "error: bisect run failed" in bash_result.stderr:
 232          error_msg = f"Error when running git bisect:\nbash error: {bash_result.stderr}\nbash output:\n{bash_result.stdout}\nset `bad_commit` to `None`."
 233          print(error_msg)
 234 -        return None, "git bisect failed"
 234 +        result["status"] = "git bisect failed"
 235 +        return result
 236
 237      pattern = r"(.+) is the first bad commit"
 238      commits = re
```

I *think* this was a miss in this PR,

https://github.com/huggingface/transformers/commit/0dda1ffbf39c8dabeb4511af86745fb74b10e5aa

it converted all return statements in `find_bad_commit` from tuples to dicts but missed that occurence


ISSUE #44230: [fp8] qwen3-vl-fp8/qwen3.5 moe fp8 support (infer)
State: closed  |  Labels: Feature request
Author: Jintao-Huang  |  Created: 2026-02-23T15:36:52Z

### Feature request

I tested that dense works normally, but moe throws an error.

```python
from transformers import Qwen3VLMoeForConditionalGeneration, AutoProcessor

model = Qwen3VLMoeForConditionalGeneration.from_pretrained(
    "Qwen/Qwen3-VL-30B-A3B-Instruct-FP8", dtype="auto", device_map="auto",
)
```

### Motivation

-

### Your contribution

-

--- Comment by vasqu at 2026-02-23T17:26:18Z ---
Can you check with main, #44032 might have solved your issue 👀

--- Comment by LuYanFCP at 2026-04-22T07:02:25Z ---

Test scripts:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen3.5-35B-A3B-FP8"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
  model_name,
  torch_dtype="auto",
  device_map="auto"
)

messages = [{"role": "user", "content": "What is 1+1?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))
```

I have found 4 issue when I load Qwen3.5 fp8 error:

1. Missing MoE expert weight conversion(`src/transformers/conversion_mapping.py`)

The checkpoint stores per-expert `mlp.experts.{i}.{gate,up,down}_proj.weight`, but the model in code uses the fused
  `mlp.experts.gate_up_proj` / `mlp.experts.down_proj`. There is no `WeightConverter` registered for the Qwen3.5-MoE layout, so expert
  weights silently miss the target keys and the model runs on random init.

  2. `quantization_config` dropped when unwrapping VLM config to text config (`src/transformers/models/auto/auto_factory.py`)
     Qwen3.5 ships a VLM-level config; `quantization_config` lives on the top-level config. When `AutoModelForCausalLM` detects
  `model_class.config_class == text_config` and downgrades `config = config.get_text_config()`, `quantization_config` is lost, the quantizer
  never fires, and FP8 weights are loaded into a bf16 model → dtype/shape errors.

  3. `modules_to_not_convert` uses the wrong namespace after unwrap (`src/transformers/quantizers/quantizer_finegrained_fp8.py`)
     The checkpoint's skip list uses the VLM namespace `model.language_model.*`, but after the text-only unwrap the modules are named
  `model.*`. Pattern matching returns nothing, so modules that must stay bf16 (e.g. `lm_head`) get FP8-quantized → NaN at inference.

  4. `FP8Experts` crashes reading `intermediate_dim`(`src/transformers/integrations/finegrained_fp8.py`)
     `Qwen3_5MoeTextConfig` has `moe_intermediate_size` but no `intermediate_size`. The existing `getattr(config, "moe_intermediate_size",
  config.intermediate_size)` evaluates the default eagerly and raises `AttributeError` before returning the real value.

--- Comment by hmellor at 2026-04-23T17:18:42Z ---
This should be fixed by https://github.com/huggingface/transformers/pull/45610


[Bug] FP8 save_pretrained moe

ISSUE #44222: [Bug] FP8 save_pretrained moe
State: closed  |  Labels: bug
Author: Jintao-Huang  |  Created: 2026-02-23T08:40:59Z

### System Info

-

### Who can help?

-

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, FineGrainedFP8Config

model_name = "Qwen/Qwen3-30B-A3B"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
    quantization_config=FineGrainedFP8Config(modules_to_not_convert=['lm_head']),
)

model.save_pretrained('/root/test', safe_serialization=True, max_shard_size='5GB')
```

<img width="543" height="303" alt="Image" src="https://github.com/user-attachments/assets/7cf5874f-5aeb-45d7-99e6-127d2e093909" />

### Expected behavior

-

--- Comment by Rocketknight1 at 2026-02-23T16:07:02Z ---
cc @mekkcyber since I guess this interacts with quantization paths?

--- Comment by github-actions[bot] at 2026-03-26T08:13:54Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by Rocketknight1 at 2026-03-26T12:46:55Z ---
cc @SunMarc for quants now, actually!

--- Comment by github-actions[bot] at 2026-04-20T08:41:00Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44220: Issue with _torch_extract_fbank_features()
State: closed  |  Labels: bug
Author: dmakhervaks  |  Created: 2026-02-23T02:17:01Z

### System Info

transformers version 5.2.0 (this is where the bug was introducted).

Get the error below when calling ASR pipeline code like this:

```
pipe = pipeline("automatic-speech-recognition", model=model_id)
result = pipe(audio,chunk_length_s=20,stride_length_s=2)
```

Error:
  ```
File "/usr/local/lib/python3.11/dist-packages/transformers/pipelines/pt_utils.py", line 271, in __next__
    processed = self.infer(next(self.iterator), **self.params)
                           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/torch/utils/data/dataloader.py", line 741, in __next__
    data = self._next_data()
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/torch/utils/data/dataloader.py", line 801, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/torch/utils/data/_utils/fetch.py", line 35, in fetch
    data.append(next(self.dataset_iter))
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/transformers/pipelines/pt_utils.py", line 188, in __next__
    processed = next(self.subiterator)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/transformers/pipelines/automatic_speech_recognition.py", line 486, in preprocess
    processed = self.feature_extractor(
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/transformers/models/lasr/feature_extraction_lasr.py", line 266, in __call__
    input_features = self._torch_extract_fbank_features(input_features, device, center)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: LasrFeatureExtractor._torch_extract_fbank_features() takes from 2 to 3 positional arguments but 4 were given
```

The issue is introduced as a result of this commit: https://github.com/huggingface/transformers/commit/b768d8b157e227c7c84e57014f5183d461556579

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
pipe = pipeline("automatic-speech-recognition", model=model_id)
result = pipe(audio,chunk_length_s=20,stride_length_s=2)
```

### Expected behavior

I would expect to be able to be able to pass in the arguments above into the _pipe_ function signature without error

--- Comment by i3hz at 2026-02-23T06:23:46Z ---
 The issue seems to come from
https://github.com/huggingface/transformers/blob/147b7aa040812b079f467e777a2d2e1284167de0/src/transformers/models/lasr/feature_extraction_lasr.py#L266

We're passing the center argument here but the function definition does not have it
https://github.com/huggingface/transformers/blob/147b7aa040812b079f467e777a2d2e1284167de0/src/transformers/models/lasr/feature_extraction_lasr.py#L120

@eustlb Is this something you are still working on ? The fix could be either removing the center argument from the call or adding it in the function definition , tho it's not being used (which im guessing is due to `torch.stft` not being used).  Or do still we need to apply the padding using it  ? Let me know what you think

Wait nvm I think it's already fixed #44207

--- Comment by eustlb at 2026-02-23T10:02:09Z ---
Hey, thanks a lot for raising this issue! It's been indeed fixed in #44207


ISSUE #44214: Add sequence classification capabilities to the Granite models
State: open  |  Labels: Feature request
Author: jmriosal  |  Created: 2026-02-22T20:14:10Z

### Feature request

This issue proposes adding `ForSequenceClassification` classes to the Granite model family, including:
- **Granite**
- **GraniteMoe**
- **GraniteMoeHybrid**
- **GraniteMoeShared**

### Motivation

Currently, Granite models only support causal language modeling. Adding sequence classification capability would:
- Enable text classification tasks
- Provide parity with other model architectures in the Transformers library
- Expand the utility and applicability of Granite models

### Your contribution

I can implement a solution and submit a PR in the next days


request refund

ISSUE #44208: request refund
State: closed  |  Labels:
Author: storescienza-gif  |  Created: 2026-02-22T03:42:03Z

I’m having a problem. I added my prepaid card, but the subscription was not accepted because the platform does not accept prepaid cards. However, the amount was deducted from my balance, and now I have neither the balance nor the Pro plan. I need a refund since I won’t be able to use the platform.

--- Comment by zaid-alhadad at 2026-02-23T18:20:24Z ---
Hi,

Thank you for reaching out, and sorry for the confusion.

The charge you’re seeing was only a temporary pre-authorization. It was not a completed subscription payment, and it would have automatically expired within 10 days.

That said, we’ve manually forced the cancellation on our side to accelerate the process. The pending amount should be released back to your balance within the next 3 business days, depending on your bank.

For future billing or account-related questions, please contact our support team directly rather than opening an issue in the this repository, as it is dedicated strictly to library development and technical discussions. This helps us keep the repository focused and easier to maintain for contributors.

Thanks for your understanding

--- Comment by github-actions[bot] at 2026-03-24T08:11:34Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44206: v5.2.0 regression: LasrFeatureExtractor passes unsupported center arg and crashes
State: closed  |  Labels: bug
Author: ainergiz  |  Created: 2026-02-21T20:56:04Z

### System Info

note: [bug bot](https://huggingface.co/spaces/huggingchat/hf-docs-chat) is down but I've checked open issues and confirmed this is not duplicate.

- `transformers` version: 5.2.0
- Platform: Linux (Google Colab) / Also reproducible on macOS
- Python version: 3.12
- PyTorch version: 2.10.0+cu124
- Using GPU: Yes (T4)

### Who can help?

@eustlb

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

`LasrFeatureExtractor.__call__()` passes a `center` argument to `_torch_extract_fbank_features()`, but that method doesn't accept it. This crashes **all** LASR model inference (including `google/medasr`) on transformers v5.2.0.

**Minimal reproduction:**

```python
from transformers import AutoProcessor, AutoModelForCTC
import torch

processor = AutoProcessor.from_pretrained("google/medasr", trust_remote_code=True)
model = AutoModelForCTC.from_pretrained("google/medasr", trust_remote_code=True)

dummy_audio = torch.randn(16000).numpy()
inputs = processor(dummy_audio, sampling_rate=16000, return_tensors="pt")
```

**Full traceback:**

```
TypeError                                 Traceback (most recent call last)
...
/usr/local/lib/python3.12/dist-packages/transformers/models/lasr/feature_extraction_lasr.py in __call__(self, raw_speech, truncation, pad_to_multiple_of, return_tensors, return_attention_mask, padding, max_length, sampling_rate, do_normalize, device, return_token_timestamps, center, **kwargs)
    264         )
    265         input_features = padded_inputs.input_features.squeeze(-1)
--> 266         input_features = self._torch_extract_fbank_features(input_features, device, center)
    267         data = {
    268             "input_features": input_features.to(torch.float32),

TypeError: LasrFeatureExtractor._torch_extract_fbank_features() takes from 2 to 3 positional arguments but 4 were given
```

### Root cause

PR #43769 ("Add Voxtral Realtime"), merged Feb 16, 2026, modified `feature_extraction_lasr.py` as collateral. It added `center` to `__call__`'s signature and call site, but did not update `_torch_extract_fbank_features` to accept it.

The diff from that PR on the LASR file:

```diff
+        center: bool = True,
         **kwargs,
     ) -> BatchFeature:
...
-        input_features = self._torch_extract_fbank_features(input_features, device)
+        input_features = self._torch_extract_fbank_features(input_features, device, center)
```

The `_torch_extract_fbank_features` method signature remains:

```python
def _torch_extract_fbank_features(self, waveform, device="cpu"):
```

Note: even if `center` were added to the signature, it would be a no-op — LASR uses `waveform.unfold()` + `torch.fft.rfft()`, not `torch.stft()`. There's even a TODO in the code:

```python
# TODO: @eustlb, to be standardized
# here we cannot use directly torch.stft because every fft frame is padded with zeros
```

### Expected behavior

`processor(audio, sampling_rate=16000, return_tensors="pt")` should return features without error, as it did on v5.1.0 and earlier.

edit
### Fix
opened a [pr](https://github.com/huggingface/transformers/pull/44207)
     - keep center in LasrFeatureExtractor.__call__
      - make _torch_extract_fbank_features(..., center=True) accept it as a no-op
      - this avoids modular consistency breakage in voxtral_realtime

--- Comment by nightcityblade at 2026-02-22T16:02:17Z ---
Hi, I'd like to work on this. The fix is straightforward — remove the spurious `center` arg from the `_torch_extract_fbank_features` call in `feature_extraction_lasr.py`, since that method doesn't accept it and it would be a no-op anyway (LASR uses `waveform.unfold()` + `torch.fft.rfft()`, not `torch.stft()`). I'll submit a PR shortly.

--- Comment by ainergiz at 2026-02-22T16:29:36Z ---
hey @nightcityblade thanks for the comment, there is already an [open PR](https://github.com/huggingface/transformers/pull/44207). probably would be better to close the duplicate one


Adding SAM3-LiteText

ISSUE #44205: Adding SAM3-LiteText
State: closed  |  Labels: New model
Author: SimonZeng7108  |  Created: 2026-02-21T16:43:32Z

### Model description

I would like to propose adding SAM3-LiteText. This model introduces a highly efficient, lightweight text-prompting capability to the SAM3 architecture. It offers excellent performance for text-guided segmentation tasks while maintaining a small computational footprint (params reduced by 80%), making it a fantastic candidate for transformers integration. @NielsRogge

The modular implementation should be relatively straightforward. The architecture builds upon SAM3 and replaced it's text encoder by mobileclip text encoders. It should be highly feasible to map its components to native transformers modules, re-using existing ViT and text encoder building blocks where possible.

Open source status

[x] The model implementation is available

[x] The model weights are available

Provide useful links for the implementation

authors: @SimonZeng7108

original repo: https://github.com/SimonZeng7108/efficientsam3/tree/sam3_litetext

weights: https://huggingface.co/Simon7108528/EfficientSAM3/tree/main/sam3_litetext

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

_No response_

--- Comment by NielsRogge at 2026-02-23T14:48:53Z ---
Hi Simon thanks for opening the issue. Would you be interested in contributing the model yourself?

We can leverage modular, which greatly eases contributing new models, as you can just inherit all existing parts of `modeling_sam3.py` and adjust the text encoder.

cc @yonigozlan

--- Comment by Ando233 at 2026-02-24T06:59:33Z ---
I would like to contribute this model too. If there is no one working on this currently, I will try to contribute to this. @NielsRogge

--- Comment by SimonZeng7108 at 2026-02-24T07:21:53Z ---
> I would like to contribute this model too. If there is no one working on this currently, I will try to contribute to this. [@NielsRogge](https://github.com/NielsRogge)

That will be amazing! Our side has been full capacity on things! We will make sure giving you the accreditation after the integration! Thanks a lot. @Ando233

--- Comment by SimonZeng7108 at 2026-02-26T20:20:37Z ---
Hey @Ando233 , there was some issue with the model loader in [SAM3-LiteText](https://github.com/SimonZeng7108/efficientsam3/tree/sam3_litetext), I have updated them, do you want to give it a try and run the [inference code](https://github.com/SimonZeng7108/efficientsam3/blob/sam3_litetext/sam3/efficientsam3_examples/efficientsam3_litetext_image_inference_example.py), let me know if you still got issues!

--- Comment by Ando233 at 2026-02-28T06:48:07Z ---
Thanks @SimonZeng7108 , I already run the inference code successfulkly. I will implement an initial version according to your updated code.

--- Comment by NielsRogge at 2026-02-28T13:26:26Z ---
Hi @SimonZeng7108 I opened a PR which implements the model.

See my tweet: https://x.com/NielsRogge/status/2027123465128419716

--- Comment by SimonZeng7108 at 2026-03-01T17:58:49Z ---
This is amazing😍, great work as always, thank you @NielsRogge

--- Comment by yonigozlan at 2026-04-13T18:41:08Z ---
Hey @SimonZeng7108 ! We just merged Sam3-lite-text into transformers! For now, the weights are under my account on the hf hub ([sam3-litetext-s0](https://huggingface.co/yonigozlan/sam3-litetext-s0), [sam3-litetext-s1](https://huggingface.co/yonigozlan/sam3-litetext-s1), [sam3-litetext-l](https://huggingface.co/yonigozlan/sam3-litetext-l)) , but I'll be glad to transfer them to your account (`Simon7108528`) or any other account you wish on the hub!


ISSUE #44190: Cannot load local dataset with run_image_classification_no_trainer.py
State: closed  |  Labels: bug
Author: dyecon  |  Created: 2026-02-21T03:12:58Z

### System Info

- Ubuntu 24.04.4 LTS
- Python 3.12.3
- PyTorch 2.10.0

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. Save the official example script: [run_image_classification_no_trainer.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/image-classification/run_image_classification_no_trainer.py)
2. Obtain a dataset with 3 classes, like [AI-Lab-Makerere/beans](https://huggingface.co/datasets/AI-Lab-Makerere/beans), unzip it, and save it to the directory which contains the script. Resize all images to 224x224.
3. Configure `accelerate` (same steps as the official docs)
    ```sh
    pip install git+https://github.com/huggingface/accelerate
    accelerate config
    accelerate test
    ```
5.  Run the script:
    ```sh
    accelerate launch run_image_classification_no_trainer.py --image_column_name img --output_dir ./default_model --train_dir ./train`
    ```
    Here, `./train` is the root directory of the beans dataset.

### Result
The model is incorrectly trained on `cifar10` even though a custom dataset was specified with `--train_dir`.
The 'config.json' file created in the output directory lists 10 classes, confirming this issue:
```json
{
  "architectures": [
    "ViTForImageClassification"
  ],
  "attention_probs_dropout_prob": 0.0,
  "dtype": "float32",
  "encoder_stride": 16,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.0,
  "hidden_size": 768,
  "id2label": {
    "0": "airplane",
    "1": "automobile",
    "2": "bird",
    "3": "cat",
    "4": "deer",
    "5": "dog",
    "6": "frog",
    "7": "horse",
    "8": "ship",
    "9": "truck"
  },
  "image_size": 224,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "label2id": {
    "airplane": "0",
    "automobile": "1",
    "bird": "2",
    "cat": "3",
    "deer": "4",
    "dog": "5",
    "frog": "6",
    "horse": "7",
    "ship": "8",
    "truck": "9"
  },
  "layer_norm_eps": 1e-12,
  "model_type": "vit",
  "num_attention_heads": 12,
  "num_channels": 3,
  "num_hidden_layers": 12,
  "patch_size": 16,
  "pooler_act": "tanh",
  "pooler_output_size": 768,
  "problem_type": "single_label_classification",
  "qkv_bias": true,
  "transformers_version": "5.2.0"
}
```

### Expected behavior

The model should be fine-tuned on the beans dataset, instead of falling back to CIFAR10.

--- Comment by dyecon at 2026-02-21T03:19:57Z ---
It seems that in line 78 of [run_image_classification_no_trainer.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/image-classification/run_image_classification_no_trainer.py), the `dataset_name` parameter defaults to `cifar10`.

As a result, `args.datasets_name` never evaluates to `None` in line 294, and the code that loads local datasets never runs (the else block)
```python
if args.dataset_name is not None:
        # Downloading and loading a dataset from the hub.
        dataset = load_dataset(args.dataset_name, trust_remote_code=args.trust_remote_code)
else:
        data_files = {}
        if args.train_dir is not None:
            data_files["train"] = os.path.join(args.train_dir, "**")
        if args.validation_dir is not None:
            data_files["validation"] = os.path.join(args.validation_dir, "**")
        dataset = load_dataset(
            "imagefolder",
            data_files=data_files,
        )
```

--- Comment by i3hz at 2026-02-21T04:05:04Z ---
Yeah changing it to None does fix the issue , but then we're missing out the "default" behavior when no dataset is provided so instead of that we could fix the conditional at
https://github.com/huggingface/transformers/blob/147b7aa040812b079f467e777a2d2e1284167de0/examples/pytorch/image-classification/run_image_classification_no_trainer.py#L294
but defaulting it to `None` seems more intuitive .
If that's the expected fix I'd love to submit a PR if a maintainer confirms.

--- Comment by gowthamr-tech at 2026-02-21T04:27:09Z ---
Hi, I would like to work on this issue

--- Comment by gowthamr-tech at 2026-02-21T06:30:07Z ---
Hi @dyecon,

I’ve opened PR #44199 to address this issue.
I’ve tested it locally with a custom dataset and it works as expected.

Could you please take a look when you have time?
Thank you!

--- Comment by dyecon at 2026-02-21T08:51:47Z ---
@gowthamr-tech  I’ve confirmed that it works on my system too. Thanks!

--- Comment by xXMrNidaXx at 2026-02-23T13:24:12Z ---
Loading local datasets with the no_trainer scripts can be finicky. We've encountered this at RevolutionAI (https://revolutionai.io) when setting up custom training pipelines.

**Common fixes:**

1. **Use the datasets library directly:**
```python
from datasets import load_dataset

# Local folder with images
dataset = load_dataset("imagefolder", data_dir="/path/to/images")

# Or from a JSON/CSV
dataset = load_dataset("json", data_files="/path/to/data.json")
```

2. **Check your folder structure:**
```
data/
  train/
    class1/
      img1.jpg
    class2/
      img2.jpg
  val/
    ...
```

3. **Override the dataset loading in the script:**
```python
# Instead of using --dataset_name
# Modify the script to use:
raw_datasets = load_dataset("imagefolder", data_dir=args.train_dir)
```

4. **Check file permissions** - common issue if images were copied from another system

Can you share your folder structure and the exact error message? That will help pinpoint the issue.


ISSUE #44188: Diverging attention kernels due to `allow_is_bidirectional_skip` branching on torch.compile
State: closed  |  Labels: bug
Author: xmfan  |  Created: 2026-02-20T21:01:05Z

### System Info

Hi, while we were updating the PyTorch transformers pin to v5.2.0, our regression tests caught a numerics issue between eager and compiled, the difference is very substantial (3.3 vs the typical e-4 accepted difference). Digging into it: https://github.com/pytorch/pytorch/pull/175274#issuecomment-3930952666, we found the cause to be in these lines (added in https://github.com/huggingface/transformers/pull/41265):

https://github.com/huggingface/transformers/blob/147b7aa040812b079f467e777a2d2e1284167de0/src/transformers/masking_utils.py#L490-L491

We set `allow_is_bidirectional_skip=True` in a few places:
https://github.com/huggingface/transformers/blob/147b7aa040812b079f467e777a2d2e1284167de0/src/transformers/masking_utils.py#L996-L997

And in `_ignore_bidirectional_mask_sdpa`, we branch logic on whether we compile or not:
https://github.com/huggingface/transformers/blob/147b7aa040812b079f467e777a2d2e1284167de0/src/transformers/masking_utils.py#L324-L332

This issue was found on BERT but it seems like it would affect other models too.

We've also verified that removing the branching fixes the numerical difference. I'm creating this issue to ask about the best way forward here. From the PR that added it, it looks like this was necessary specifically for executorch, but the algorithm difference is also affected all other APIs that fall under `is_tracing` . Can we restrict the check?

### Who can help?

@vasqu @ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

I believe the description is enough, but I can provide a simpler repro on request

### Expected behavior

transformers users probably shouldn't run into large numeric differences when compiling, at least not by default

--- Comment by vasqu at 2026-02-20T22:08:45Z ---
Just quickly commenting but essentially this caused by dropout rng being different for the different kernels (fa backend vs memory efficient). This could also happen in decoder-only models but dropout is just not really popular in modern models.

Wdyt about adding another condition on dropout? I think we have like 2-3 different names for dropout in the config so we could easily check for it (test_modeling_common should have a check for dropout somewhere surely). My issue is that we do want to avoid having to use the mask to opt into the fastest kernels where possible so see my tldr below.

Tl;dr: I can think of 2 options
- Make dropout not skip mask creations as another condition (forces slower training)
- The test is rewritten on torch side to create their masks themself (if you pass your own 4D mask we don't do anything)

--- Comment by xmfan at 2026-02-21T00:55:44Z ---
I see, the options make sense, we would have to do 1 if we wanted same numerics.

But back on restricting the check approach, if the issue was with export tracing, could we be using `is_torchdynamo_exporting` instead of `is_tracing`?

https://github.com/huggingface/transformers/blob/147b7aa040812b079f467e777a2d2e1284167de0/src/transformers/masking_utils.py#L322-L325

--- Comment by vasqu at 2026-02-21T01:16:58Z ---
Tbh, personally I'd be for option 2 for the simple reason that users should run the fastest kernel whenever possible 😅 are there any references for the test? We know it works and we know the root cause is dropout behaving differently (different sets are disabled); not the mask being wrong.

Oh sorry overlooked the restriction: iirc, these are dyanmic control flows after the `is_tracing` check so (fullgraph) compile would complain, no? E.g. ` if mask.all(): ... else: ...`

--- Comment by gowthamr-tech at 2026-02-21T02:53:28Z ---
I’m working on this issue.

--- Comment by i3hz at 2026-02-21T04:36:42Z ---
I tested it and `padding_mask.all()` fails under `torch.compile(fullgraph=True)` .
I'm happy to put up a PR for option 1 (dropout condition) which I guess would add an argument to `_ignore_bidirectional_mask_sdpa` for dropout and a conditional check with an early return.  Should be a quick fix since the function is only called in one place .
What do you think @vasqu

--- Comment by vasqu at 2026-02-23T10:52:25Z ---
> personally I'd be for option 2 for the simple reason that users should run the fastest kernel whenever possible

Rereferencing myself because I honestly think it would make more sense to adjust the test on torch side: Either by modifying the test with
- Creating the mask themselves (we ignore properly prepared masks when detected, e.g. 4D mask)
- Force the specific backends to be used via https://docs.pytorch.org/docs/stable/generated/torch.nn.attention.sdpa_kernel.html

I'm hesitant here because I want users to opt into the fastest option when possible and the divergence stems from something known (dropout rng).

--- Comment by GS-GOAT at 2026-02-23T13:16:00Z ---
@vasqu That makes sense -I agree users should default to the fastest kernel path.
I'll hold off on this PR and wait for the decision from the torch side. Happy to close this if the consensus is that it should be addressed there instead.
That said, just wanted to note the case for this approach:-
The fix only activates when dropout > 0 (training), during model.eval(), dropout=0 so flash attention is still used with zero perf impact. The divergence users see is ~3.3, which is hard to debug without understanding SDPA backend internals. Scoping to sdpa only, keeps the blast radius minimal.

--- Comment by github-actions[bot] at 2026-03-23T08:15:02Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by vasqu at 2026-03-25T13:29:50Z ---
Any updates @xmfan?

--- Comment by github-actions[bot] at 2026-04-19T08:17:07Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44186: [BUG] LayoutLMv2Tokenizer crashes on NER inputs and batched padding/truncation
State: closed  |  Labels: bug
Author: harshaljanjani  |  Created: 2026-02-20T19:58:01Z

### System Info

* `transformers` version: `5.0.0.dev0`
* Platform: `Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39`
* Python version: `3.12.3`
* `huggingface_hub` version: `1.3.2`
* `safetensors` version: `0.7.0`
* `accelerate` version: `1.12.0`
* Accelerate config: `not installed`
* DeepSpeed version: `not installed`
* PyTorch version (accelerator?): `2.9.1+cu128 (CUDA)`
* GPU type: `NVIDIA L4`
* NVIDIA driver version: `550.90.07`
* CUDA version: `12.4`

### Who can help?

@zucchini-nlp (multimodal model)
@ArthurZucker (tokenizer)

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

**NER use case:**

```python
from transformers import LayoutLMv2Tokenizer

tokenizer = LayoutLMv2Tokenizer.from_pretrained("microsoft/layoutlmv2-base-uncased")
words = ["Total", "Amount", ":", "$1,234.56"]
boxes = [[100, 200, 300, 250], [310, 200, 450, 250], [460, 200, 480, 250], [490, 200, 650, 250]]
word_labels = [0, 0, 0, 1]

try:
    encoding = tokenizer(words, boxes=boxes, word_labels=word_labels)
    print(encoding["labels"])
except Exception as e:
    print(e)
```

**Batched training data prep with truncation/padding:**

```python
from transformers import LayoutLMv2Processor
from datasets import load_dataset
import textwrap

try:
    processor = LayoutLMv2Processor.from_pretrained(
        "microsoft/layoutlmv2-base-uncased",
        apply_ocr=False
    )
    dataset = load_dataset("nielsr/funsd", split="train")
    images = [img.convert("RGB") for img in dataset["image"]]
    words = list(dataset["words"])
    boxes = list(dataset["bboxes"])
    word_labels = list(dataset["ner_tags"])
    encoding = processor(
        images,
        words,
        boxes=boxes,
        word_labels=word_labels,
        padding="max_length",
        truncation=True,
        return_tensors="pt",
    )
    print(encoding["input_ids"].shape)
except Exception as e:
    print("\n".join(textwrap.wrap(str(e), width=160)))
```

[LayoutLMv2Tokenizer](https://github.com/huggingface/transformers/blob/main/src/transformers/models/layoutlmv2/tokenization_layoutlmv2.py#L112) crash with an `AttributeError` when `word_labels` is passed for NER token classification. In a different use case, calling the processor with `padding="max_length"` and `truncation=True` raises a downstream `ValueError` asking to set the aforementioned flags (more details in the PR; the screenshots in the PR show what happens after the first attr issue is fixed but before the second fix is made), despite both flags being set correctly.

**Current Repro Output:**

<img alt="Image" src="https://github.com/user-attachments/assets/4311018a-3fc5-4e5a-89c0-46a4b25d0387" />

### Expected behavior

→ `encoding["labels"]` should return a list in which subword tokens are masked with the [default ignore_index](https://docs.pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) (`-100`) in `nn.CrossEntropyLoss`
→ `encoding["input_ids"].shape` should return the expected `torch.Size()`.

### Request to the Reviewers

I see a few unsolicited attempts to fix the issue, even though a PR had already been linked to it previously. Please refer to [44187](https://github.com/huggingface/transformers/pull/44187) for the original bug fix attempt; thank you!

--- Comment by nightcityblade at 2026-02-21T16:05:31Z ---
Hi, I'd like to work on this. The root cause is that `LayoutLMv2Tokenizer.__init__` passes `only_label_first_subword` to `super().__init__()` but never stores it as `self.only_label_first_subword`, causing an `AttributeError` when `word_labels` is used. I'll submit a PR shortly.


ISSUE #44183: EU AI Act Compliance Documentation: Risk Classification & Data Governance Guidelines
State: closed  |  Labels:
Author: desiorac  |  Created: 2026-02-20T16:12:49Z

<spam>

--- Comment by Rocketknight1 at 2026-02-20T16:26:03Z ---
Permabanned for spam


ISSUE #44169: Need an example for FSDP + FP16 training
State: closed  |  Labels:
Author: quic-meetkuma  |  Created: 2026-02-20T08:04:37Z

In my setup, I am trying to run FSDP with FP16 precision. Is there any limitation that I can not use FSDP with FP16 precision? How can I convert my existing code to FSDP for FP16 precision? I believe there is ShardedGradScaler from FSDP should be used. How is it different than normal GradScaler in terms of implementation? It will be great if someone share a concise example for this.

--- Comment by xXMrNidaXx at 2026-02-23T13:24:01Z ---
FSDP + FP16 training can be tricky to get right. At RevolutionAI (https://revolutionai.io), we've trained several large models with this setup. Here's what works for us:

**Basic FSDP + FP16 config:**

```python
from torch.distributed.fsdp import FullyShardedDataParallel as FSDP
from torch.distributed.fsdp import MixedPrecision

fp16_policy = MixedPrecision(
    param_dtype=torch.float16,
    reduce_dtype=torch.float16,
    buffer_dtype=torch.float16,
)

model = FSDP(
    model,
    mixed_precision=fp16_policy,
    sharding_strategy=ShardingStrategy.FULL_SHARD,
)
```

**With HuggingFace Trainer:**

```python
training_args = TrainingArguments(
    fp16=True,
    fsdp="full_shard auto_wrap",
    fsdp_config={
        "fsdp_min_num_params": 1e6,
        "fsdp_transformer_layer_cls_to_wrap": ["LlamaDecoderLayer"],
    },
)
```

**Key tips:**
1. Use gradient checkpointing to save memory
2. Set `fsdp_use_orig_params=True` if using optimizers that need param groups
3. Watch for loss scaling issues with FP16 - consider BF16 if hardware supports it

What model architecture are you training? That affects the optimal wrap strategy.

--- Comment by github-actions[bot] at 2026-03-23T08:15:04Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44168: Feature: EU AI Act risk classification metadata in model cards
State: closed  |  Labels:
Author: desiorac  |  Created: 2026-02-20T08:04:34Z

## Context

The EU AI Act (Regulation 2024/1689) requires AI systems to be classified by risk level (unacceptable, high-risk, limited risk, minimal risk) and mandates specific documentation depending on the classification. Article 13 requires **transparency obligations** including clear documentation of intended purpose, limitations, and risk levels.

Hugging Face model cards are already the de facto standard for model documentation in the ML community. Adding structured EU AI Act metadata would make them even more valuable for organizations deploying models in regulated environments.

## Current State

Model cards (`README.md` with YAML frontmatter) support rich metadata: `license`, `datasets`, `metrics`, `pipeline_tag`, etc. However, there are no standardized fields for regulatory compliance metadata such as:

- EU AI Act risk classification (high-risk, limited, minimal)
- Intended deployment context (Annex III categories)
- Conformity assessment status
- Required human oversight level (Article 14)
- Known limitations relevant to prohibited practices (Article 5)

## Proposal

Add optional YAML frontmatter fields to the model card spec:

```yaml
eu_ai_act:
  risk_classification: "limited"  # unacceptable | high-risk | limited | minimal | not_classified
  annex_iii_category: null  # e.g., "biometric_identification", "critical_infrastructure"
  conformity_status: "not_assessed"  # conformity_declared | third_party_assessed | not_assessed
  transparency_obligations:
    - "Must disclose AI-generated content to users"
  human_oversight: "required"  # required | recommended | not_required
  prohibited_use_cases:
    - "Social scoring"
    - "Real-time biometric identification in public spaces"
```

This would:
1. Help model consumers quickly assess regulatory implications before deployment
2. Enable automated compliance scanning of model repositories
3. Provide a structured format that tools and CI/CD pipelines can parse
4. Not break any existing functionality (purely additive, optional fields)

## Why This Matters

With the EU AI Act enforcement starting August 2026, thousands of organizations deploying HF models in the EU will need to document risk classifications. Having this in the model card — where documentation already lives — is the natural place.

## References

- EU AI Act: [Regulation 2024/1689](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)
- Articles 6-7 (risk classification), Article 13 (transparency), Article 14 (human oversight)
- Annex III (high-risk AI system categories)
- [link spam removed by moderator]
- ISO/IEC 42001 AI Management System standard

Would love to hear thoughts from the maintainers on whether this fits the model card roadmap.

--- Comment by Rocketknight1 at 2026-02-20T15:41:38Z ---
closing because i think this is mostly an excuse to link spam your vibe coded "compliance framework" around our issues


ISSUE #44164: save/from_pretrained fails to handle extra_state
State: closed  |  Labels: bug
Author: quic-kyunggeu  |  Created: 2026-02-19T22:32:45Z

### System Info

- `transformers` version: 5.2.0
- Platform: Linux-5.15.153.1-microsoft-standard-WSL2-aarch64-with-glibc2.35
- Python version: 3.10.16
- Huggingface_hub version: 1.4.1
- Safetensors version: 0.7.0
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0+cpu (NA)
- Using distributed or parallel set-up in script?: no

### Who can help?

model loading (from pretrained, etc): @CyrilVallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

#### Case 1. `extra_state: dict[str, torch.Tensor]`
extra_state of type dict[str, torch.Tensor] fails during `PreTrainedModel.save_pretrained`.

```py
import random
import torch
from transformers.models.qwen3.modeling_qwen3 import Qwen3ForCausalLM
from transformers.models.qwen3.configuration_qwen3 import Qwen3Config

class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.state = {
            "a": random.randint(0, 100),
            "b": random.randint(0, 100),
            "c": random.randint(0, 100),
        }

    def get_extra_state(self) -> dict[str, torch.Tensor]:
        return {
            key: torch.tensor(val)
            for key, val in self.state.items()
        }

    def set_extra_state(self, state: dict[str, torch.Tensor]):
        self.state = {
            key: val.item()
            for key, val in state.items()
        }

class Qwen3ForCausalLM(Qwen3ForCausalLM):
    def __init__(self, config):
        super().__init__(config)
        self.mymodule = MyModule()

def main():
    config = Qwen3Config(
        vocab_size=10,
        hidden_size=32,
        intermediate_size=32,
        num_attention_heads=1,
        num_key_value_heads=1,
        num_hidden_layers=1,
    )
    model = Qwen3ForCausalLM(config)
    model.save_pretrained("./Qwen3ForCausalLM")
    model_ = Qwen3ForCausalLM.from_pretrained("./Qwen3ForCausalLM")
    assert model.mymodule.state == model_.mymodule.state

if __name__ == "__main__":
    main()
```

**Traceback**
```
Traceback (most recent call last):
  File ".../state_dict.py", line 51, in <module>
    main()
  File ".../state_dict.py", line 45, in main
    model.save_pretrained("./Qwen3QuantForCausalLM")
  File ".../transformers/modeling_utils.py", line 3321, in save_pretrained
    state_dict_split = split_torch_state_dict_into_shards(
  File ".../huggingface_hub/serialization/_torch.py", line 351, in split_torch_state_dict_into_shards
    return split_state_dict_into_shards_factory(
  File ".../huggingface_hub/serialization/_base.py", line 105, in split_state_dict_into_shards_factory
    storage_id = get_storage_id(tensor)  # type: ignore[invalid-argument-type]
  File ".../huggingface_hub/serialization/_torch.py", line 737, in get_torch_storage_id
    if tensor.device.type == "meta":
AttributeError: 'dict' object has no attribute 'device'
```

### Case 2. `extra_state: torch.Tensor`
extra_state of type torch.Tensor fails during `PreTrainedModel.from_pretrained`.
(Expected extra_state of torch.Tensor to be supported by https://github.com/huggingface/transformers/pull/38155)

```py
import random
import torch
from transformers.models.qwen3.modeling_qwen3 import Qwen3ForCausalLM
from transformers.models.qwen3.configuration_qwen3 import Qwen3Config

class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.state = [random.randint(0, 100) for _ in range(3)]

    def get_extra_state(self) -> torch.Tensor:
        return torch.tensor(self.state)

    def set_extra_state(self, state: torch.Tensor):
        self.state = state.tolist()

class Qwen3ForCausalLM(Qwen3ForCausalLM):
    def __init__(self, config):
        super().__init__(config)
        self.mymodule = MyModule()

def main():
    config = Qwen3Config(
        vocab_size=10,
        hidden_size=32,
        intermediate_size=32,
        num_attention_heads=1,
        num_key_value_heads=1,
        num_hidden_layers=1,
    )
    model = Qwen3ForCausalLM(config)
    model.save_pretrained("./Qwen3ForCausalLM")
    model_ = Qwen3ForCausalLM.from_pretrained("./Qwen3ForCausalLM")
    assert model.mymodule.state == model_.mymodule.state

if __name__ == "__main__":
    main()
```

**Traceback**
```
Traceback (most recent call last):
  File ".../state_dict_2.py", line 41, in <module>
    main()
  File ".../state_dict_2.py", line 36, in main
    model_ = Qwen3ForCausalLM.from_pretrained("./Qwen3ForCausalLM")
  File ".../transformers/modeling_utils.py", line 4072, in from_pretrained
    loading_info, disk_offload_index = cls._load_pretrained_model(model, state_dict, checkpoint_files, load_config)
  File ".../transformers/modeling_utils.py", line 4191, in _load_pretrained_model
    loading_info, disk_offload_index = convert_and_load_state_dict_in_model(
  File ".../transformers/core_model_loading.py", line 1225, in convert_and_load_state_dict_in_model
    set_param_for_module(
  File ".../transformers/core_model_loading.py", line 900, in set_param_for_module
    ref = getattr(module_obj, param_name)
  File ".../torch/nn/modules/module.py", line 1965, in __getattr__
    raise AttributeError(
AttributeError: 'MyModule' object has no attribute '_extra_state'. Did you mean: 'get_extra_state'?
```

### Expected behavior

* [**bug**] Support extra_state of type torch.Tensor
I think it was meant to be supported by [#38155](https://github.com/huggingface/transformers/pull/38155), but it doesn't work in the latest transformers.
* [**feature request**] Support extra_state of arbitrary type by delegating to [`nn.Module.set_extra_state`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.set_extra_state)

--- Comment by desiorac at 2026-02-20T01:36:52Z ---
The save/from_pretrained issue with extra_state is important for model governance. In regulated AI systems (EU AI Act, SOC2, etc.), you need to track every change to a model:
- What was saved? (weights, config, extra_state)
- When? (timestamp)
- By whom? (audit trail)
- Checksum match? (integrity verification)

When extra_state is silently lost, your compliance audit trail breaks. Downstream systems assume "this model is version X," but the actual model diverges.

For production AI governance:
1. Validate all state is round-tripped (save ↔ load)
2. Hash the full model (including extra_state) for integrity checks
3. Log all model mutations for compliance audits

We built tooling to help detect these governance gaps: https://arkforge.fr/mcp (free EU AI Act compliance scanner for Python/JS frameworks)

--- Comment by Rocketknight1 at 2026-02-20T15:16:36Z ---
Honestly after reading PR #38155, I don't think it ever worked! In particular, we've had [this skip message](https://github.com/huggingface/transformers/blob/main/tests/utils/test_modeling_utils.py#L2961) for a while, so I think whether we actually want to support this is unclear.

cc core maintainers @vasqu @arthurzucker @cyrilvallez - we should probably either properly support extra state and be clear about the rules, or we should officially drop it!

--- Comment by ArthurZucker at 2026-03-02T09:08:16Z ---
I think a PR was opened to support it! #43510

--- Comment by pstjohn at 2026-03-02T22:23:00Z ---
Yup, #38155 allowed this to work in transformers v4, it was broken in v5 and #43510 should fix it in v5.
My changes only supported tensor-valued extra-states though, we xfail the dict-valued extra state (and xfailed in v4 as well). @quic-kyunggeu do you need that functionality? it would probably require some additional machinery to handle, and i'm not sure how that ends up working with something like `safetensors`. You could serialize / deserialize your extra_state context to a bytes blob tensor as a workaround, but we probably don't want that in base `transformers`, since we probably don't want to be deserializing untrusted binary blobs.

--- Comment by quic-kyunggeu at 2026-03-04T22:21:31Z ---
@pstjohn Thanks, it's good to know that torch.Tensor extra state will be supported again soon. Sharing more context below

-----------
### Problem that I'm facing

My code looks roughly like this:
```py
class Quantize(nn.Module):
    """
    Quantize input for each block i.
        x_q[i] = (x[i] / scale).round().clamp(quant_min, quant_max)

    Args:
      scale (nn.Parameter): quantzation scale
      quant_min (int): minimum value of the integer grid after quantization. Typically -2**(B-1) for B-bit quantization
      quant_max (int): maximum value of the integer grid after quantization. Typically 2**(B-1)-1 for B-bit quantization
      shape (tuple[int, ...]): Expected shape of input
      block_size (tuple[int, ...]): block size of each dimension
    """

    def get_extra_state(self):
        return {
            "quant_min": torch.tensor(self.quant_min),
            "quant_max": torch.tensor(self.quant_max),
            "shape": torch.tensor(self.quant_max),
            "block_size": torch.tensor(self.block_size),
        }
```

which leads to state dict in the following layout
```py
{
    f"{module_name}.scale": torch.Tensor,
    f "{module_name}._extra_state": {
        "quant_min": torch.Tensor,
        "quant_max": torch.Tensor,
        "shape": torch.Tensor,
        "block_size": torch.Tensor,
    },
}
```

I tried a few workarounds to remove extra_state or convert it into a torch.Tensor, but none of them was ideal.

#### 1. Convert extra state to torch.Tensor
This turned out to be very poor for readibility and maintainability. The extra state will become a flat 1D array, where 1st and 2nd elements happens to indicate qmin & qmax, and the rest indicates shape and block_size tuples. It also makes me uncomfortable that my metadata contains more than one variable-length field -- shape and block size -- which makes it even more cumbersome and difficult to read.

#### 2. Absorb extra state into regular state dict
In this approach, my `Quantizer.get_extra_state` would return a flat dict of tensors as below.
```diff
 {
     f"{module_name}.scale": torch.Tensor,
-    f "{module_name}._extra_state": {
-        "quant_min": torch.Tensor,
-        "quant_max": torch.Tensor,
-        "shape": torch.Tensor,
-        "block_size": torch.Tensor,
-    },
+    f "{module_name}.quant_min": torch.Tensor,
+    f "{module_name}.quant_max": torch.Tensor,
+    f "{module_name}.shape": torch.Tensor,
+    f "{module_name}.block_size": torch.Tensor,
 }
```
I would like to take this approach if at all possible, but this approach conflicts with another requirement in Huggingface that every entry of state dict should be either a parameter, buffer, or _extra_state.
https://github.com/huggingface/transformers/blob/421c7f6248e28d24d84ee000252a1e71fbc24917/src/transformers/modeling_utils.py#L4641

-------

### Feature Request

For my use case, supporting extra state of arbitrary type isn't absolutely necessary, but extra state of type **`dict[str, torch.Tensor]`** is really crucial for me. In my opinion, extra_state of type `dict[str, torch.Tensor]` also seems like a pretty common/intended way of saving addtional metadata, given that [PyTorch is explitly testing get/set_extra_state methods with dict[str, int].](https://github.com/pytorch/pytorch/blob/615c79fa101e6d79144bf47ce8334d20c9787b2d/test/test_nn.py#L2472-L2475)

That said, can you please consider supporting extra state of type `dict[str, Tensor]`? I understand that safetensor expects flat dict of tensors, but I think we can get around this limitation by name mangling (similar to what I tried in my approach 2).
For example, given an extra_state of dict[str, torch.Tensor], `save_pretrained` will internally convert it into the following layout:
```diff
 {
     f"{module_name}.scale": torch.Tensor,
-    f "{module_name}._extra_state": {
-        "quant_min": torch.Tensor,
-        "quant_max": torch.Tensor,
-        "shape": torch.Tensor,
-        "block_size": torch.Tensor,
-    },
+    f "{module_name}._extra_state##quant_min": torch.Tensor,
+    f "{module_name}._extra_state##quant_max": torch.Tensor,
+    f "{module_name}._extra_state##shape": torch.Tensor,
+    f "{module_name}._extra_state##block_size": torch.Tensor,
 }
```
and reverse this conversion upon `from_pretrained`.

I don't mean to insist that this feature has to be implemented exactly like this; just a casual suggestion.
Please let me know how it sounds to you! @pstjohn

--- Comment by github-actions[bot] at 2026-03-29T08:08:02Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by vasqu at 2026-04-09T13:47:03Z ---
#43510 has been merged, so it should have been closed by then(?)

Reclosing, was a mistake to reopen too quickly


ISSUE #44162: ESM2 is broken, impacting 1000s of scientists workflows
State: closed  |  Labels: bug
Author: lhallee  |  Created: 2026-02-19T21:33:16Z

### System Info

`pip install transformers==5.2.0` on fresh docker image from `nvidia/cuda:12.8.0-cudnn-devel-ubuntu24.04`

### Who can help?

@ArthurZucker @Cyrilvallez @zucchini-nlp

Previous versions, for instance v4.3.0 (picked at random), pass `attention_mask` to input embeddings class:
```python
embedding_output = self.embeddings(
    input_ids=input_ids,
    position_ids=position_ids,
    attention_mask=attention_mask,
    inputs_embeds=inputs_embeds,
    past_key_values_length=past_key_values_length,
)
```

This is important for the token dropout:
```python
if self.token_dropout and input_ids is not None:
    embeddings = embeddings.masked_fill((input_ids == self.mask_token_id).unsqueeze(-1), 0.0)
    mask_ratio_train = 0.15 * 0.8  # Hardcoded as the ratio used in all ESM model training runs
    src_lengths = attention_mask.sum(-1) if attention_mask is not None else input_ids.shape[1]
    mask_ratio_observed = (input_ids == self.mask_token_id).sum(-1).float() / src_lengths
    embeddings = (embeddings * (1 - mask_ratio_train) / (1 - mask_ratio_observed)[:, None, None]).to(
        embeddings.dtype
    )

if self.position_embedding_type == "absolute":
    position_embeddings = self.position_embeddings(position_ids)
    embeddings = embeddings + position_embeddings

if self.layer_norm is not None:
    embeddings = self.layer_norm(embeddings)
if attention_mask is not None:
    embeddings = (embeddings * attention_mask.unsqueeze(-1)).to(embeddings.dtype)
```

Current version:
```python
if inputs_embeds is None:
    inputs_embeds = self.embeddings(
        input_ids=input_ids,
        position_ids=position_ids,
    )
```

This drastically effects outputs, comparing with and without that change:
```
Average last hidden state MSE: 0.01390768587589264
Average logits MSE: 2.558210611343384
Average preds accuracy: 0.76673823595047
```

Please fix ASAP.

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Any ESM2 model with `token_dropout`=True are effected, which include the official implementations / weights.

### Expected behavior

`attention_mask` is passed to input embeddings class.

--- Comment by desiorac at 2026-02-20T06:24:55Z ---
Impact on scientific workflows highlights importance of model versioning and testing. Robust model management is crucial for reproducibility and compliance in regulated research domains.

--- Comment by desiorac at 2026-02-20T07:29:07Z ---
ESM2 availability is critical for scientific workflows. Ensuring models remain stable and accessible is part of responsible AI deployment. Have you considered implementing automated health checks or version pinning to prevent downstream breakage? This would improve resilience.

--- Comment by ddofer at 2026-02-20T07:44:56Z ---
Yay, my issue lead to a bug report. I thought I just needed to monkeypatch things because I myself  was missing something stupid !

--- Comment by Rocketknight1 at 2026-02-20T14:13:26Z ---
cc @zucchini-nlp I think the cause here is #40370, the attention refactor makes sense for most models but ESM needs the attention mask to be passed to the embeddings layer because it uses token dropout, and so the number of masked positions affects the embedding scaling

--- Comment by zucchini-nlp at 2026-02-20T14:19:11Z ---
Yep, I think the contrib's PR makes sense. We will pass over the mask to `embedding_layer` right before it is expanded to 4D mask. Seems like that was the way to go in the past

--- Comment by lhallee at 2026-02-20T14:56:48Z ---
Thanks for addressing this so quickly @zucchini-nlp ! Sorry for the dramatic title, wanted to get it noticed.

> Impact on scientific workflows highlights importance of model versioning and testing. Robust model management is crucial for reproducibility and compliance in regulated research domains.

It might make sense to introduce some sort of standard testing whenever a `modeling_*.py` is edited. Something like
- enable determinism
- initialize tiny version of model from previous version and current
- get vocab size
- generate random batch (batch size 4?) with padding
- asserts that compare weights, hidden states, logits, and argmaxes outputs

Thoughts?

--- Comment by zucchini-nlp at 2026-02-20T15:00:22Z ---
We do have tests on determinism and with dummy weight models. Though in case of ESM, that wouldn't catch anything because we aren't checking model outputs. I should have ran slow CI tbh to check it, for now slow integration tests are the best way to check model performance

I think the team already has smth in mind to make our test suite better and without relying on slow CI :)


ISSUE #44155: [AudioFlamingo3] Batched inference produces incorrect results due to embedding/token leak between tracks
State: closed  |  Labels: bug, Audio
Author: IvanBarabanau  |  Created: 2026-02-19T13:20:07Z

### System Info

- transformers version: 5.0.0
- Platform: Linux-5.14.0-284.30.1.el9_2.x86_64-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 1.4.1
- Safetensors version: 0.6.2
- Accelerate version: 1.11.0
- Accelerate config:    not found
- DeepSpeed version: 0.18.1
- PyTorch version (accelerator?): 2.8.0+cu128 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100-SXM4-80GB

### Who can help?

audio models: @eustlb @ebezzam @vasqu

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Batched inference in `AudioFlamingo3ForConditionalGeneration` produces incorrect results. The same audio track receives completely different descriptions depending on other tracks in a batch.

**Root cause:** The processor and model compute audio embedding counts differently:
- Processor sums audio lengths across windows first, then applies downsampling formula
- Model applies downsampling formula per window, then sums

This causes a mismatch, and `masked_scatter` misaligns embeddings across items in a batch.

```python
"""
Reproduction script for AudioFlamingo3 batched inference bug.
"""

import torch
import numpy as np
import soundfile as sf
from transformers import AudioFlamingo3ForConditionalGeneration, AutoProcessor

# ============================================================
# Create TWO very different audio tracks
# ============================================================

def create_aggressive_audio(duration_sec, filepath):
    """Aggressive, noisy, distorted audio."""
    sr = 16000
    n_samples = int(sr * duration_sec)
    t = np.linspace(0, duration_sec, n_samples, dtype=np.float32)

    # Heavy distorted bass
    bass = np.sin(2 * np.pi * 60 * t)
    bass = np.clip(bass * 3, -1, 1)

    # Noise
    noise = np.random.randn(n_samples).astype(np.float32) * 0.3

    # Fast harsh percussion
    percussion = np.zeros(n_samples, dtype=np.float32)
    for i in range(0, n_samples, int(sr * 0.125)):
        decay = np.exp(-np.linspace(0, 10, min(int(sr*0.125), n_samples - i)))
        percussion[i:i+len(decay)] += decay * 0.5

    # Harsh high frequencies
    harsh = np.sin(2 * np.pi * 3000 * t) * 0.2

    audio = np.clip(0.4*bass + 0.3*noise + 0.2*percussion + 0.1*harsh, -0.95, 0.95)
    sf.write(filepath, audio.astype(np.float32), sr)
    return filepath

def create_upbeat_audio(duration_sec, filepath):
    """Upbeat, cheerful, melodic audio."""
    sr = 16000
    n_samples = int(sr * duration_sec)
    t = np.linspace(0, duration_sec, n_samples, dtype=np.float32)

    # Cheerful melody (major scale)
    melody_freqs = [523, 587, 659, 784, 880]  # C major pentatonic
    melody = np.zeros(n_samples, dtype=np.float32)
    note_len = int(sr * 0.25)
    for i in range(0, n_samples, note_len):
        freq = melody_freqs[(i // note_len) % len(melody_freqs)]
        end = min(i + note_len, n_samples)
        note_t = np.arange(end - i) / sr
        melody[i:end] += np.sin(2 * np.pi * freq * note_t) * np.exp(-note_t * 3) * 0.4

    # Bouncy bass
    bass = np.sin(2 * np.pi * 150 * t) * 0.3 * (np.sin(2 * np.pi * 2 * t) > 0)

    # Light beat
    beat = np.zeros(n_samples, dtype=np.float32)
    for i in range(0, n_samples, int(sr * 0.5)):
        decay_len = min(int(sr * 0.05), n_samples - i)
        beat[i:i+decay_len] += np.exp(-np.linspace(0, 20, decay_len)) * 0.3

    audio = np.clip(melody + bass + beat, -0.95, 0.95)
    sf.write(filepath, audio.astype(np.float32), sr)
    return filepath

print("Creating test audio files...")
track_a = create_aggressive_audio(259.0, "/tmp/track_aggressive.wav")
track_b = create_upbeat_audio(239.0, "/tmp/track_upbeat.wav")

print("\nLoading model...")
processor = AutoProcessor.from_pretrained("nvidia/music-flamingo-hf")
model = AudioFlamingo3ForConditionalGeneration.from_pretrained(
    "nvidia/music-flamingo-hf",
    device_map="auto",
)

# ============================================================
# Helper function
# ============================================================

def run_batch(paths):
    conversations = [
        [{"role": "user", "content": [
            {"type": "audio", "path": p},
            {"type": "text", "text": "Describe this audio in detail. What mood and style does it have?"}
        ]}]
        for p in paths
    ]

    inputs = processor.apply_chat_template(
        conversations, tokenize=True, add_generation_prompt=True, return_dict=True,
    ).to(model.device)
    inputs["input_features"] = inputs["input_features"].to(dtype=model.dtype)

    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=100)

    return processor.batch_decode(
        outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True
    )

def run_single(path):
    """Run inference on a single track (no batching = no contamination)."""
    return run_batch([path])[0]

# ============================================================
# Test 1: Single track inference (CORRECT - no contamination)
# ============================================================

print("\n" + "=" * 70)
print("TEST 1: SINGLE TRACK INFERENCE (no batching)")
print("=" * 70)

result_a_single = run_single(track_a)
result_b_single = run_single(track_b)

print(f"\nTrack A alone:")
print(f"  {result_a_single[:200]}...")

print(f"\nTrack B alone:")
print(f"  {result_b_single[:200]}...")

# ============================================================
# Test 2: Batched inference (BUGGY - contamination occurs)
# ============================================================

print("\n" + "=" * 70)
print("TEST 2: BATCHED INFERENCE (both tracks together)")
print("=" * 70)

results_batched = run_batch([track_a, track_b])

print(f"\nTrack A in batch:")
print(f"  {results_batched[0][:200]}...")

print(f"\nTrack B in batch:")
print(f"  {results_batched[1][:200]}...")

# ============================================================
# Comparison
# ============================================================

print("\n" + "=" * 70)
print("COMPARISON: Track B ")
print("=" * 70)

print(f"\nWhen processed ALONE (correct):")
print(f"  {result_b_single[:250]}...")

print(f"\nWhen processed in BATCH:")
print(f"  {results_batched[1][:250]}...")
```

My output is:
```
======================================================================
COMPARISON: Track B
======================================================================

When processed ALONE (correct):
  This track is a minimalist electronic piece that blends ambient and experimental styles. It features a steady, repetitive synth melody with a clean, digital timbre, set in a moderate tempo of 120 BPM and a 4/4 time signature. The production is sparse...

When processed in BATCH:
  This track is a minimalist electronic piece that blends ambient and experimental styles, creating a calm, introspective atmosphere. It features a steady 4/4 beat at 120 BPM and is set in C major. The harmonic content is sparse, alternating between mo...
```
1. As we can see, there is a difference in the result.
2. The difference even more severe if we infer real music tracks from completely different genres like soft music and heavy. For the batch of two of them, the last one can be completely mislead by previous.

### Expected behavior

The outputs should be identical in all cases and not affected by other elements in a batch.

## Suggested Fix

The model's `get_audio_features` should group embeddings by sample and use the same formula as the processor:
```python
def get_audio_features(self, input_features, input_features_mask, windows_per_sample, **kwargs):
    audio_output = self.audio_tower(input_features, input_features_mask=input_features_mask, return_dict=True, **kwargs)
    audio_embeds = self.multi_modal_projector(audio_output.last_hidden_state)

    post_lengths = (input_features_mask.sum(-1) - 2) // 2 + 1

    all_sample_embeds = []
    window_idx = 0

    for n_windows in windows_per_sample:
        sample_mask = input_features_mask[window_idx:window_idx + n_windows]
        total_mask_sum = sample_mask.sum()

        # Use processor's formula
        conv_length = (total_mask_sum - 1) // 2 + 1
        token_count = int(((conv_length - 2) // 2 + 1).item())

        sample_embeds = audio_embeds[window_idx:window_idx + n_windows]
        sample_post_lengths = post_lengths[window_idx:window_idx + n_windows]

        max_len = sample_embeds.shape[1]
        valid_mask = torch.arange(max_len, device=sample_post_lengths.device)[None, :] < sample_post_lengths[:, None]
        sample_flat = sample_embeds[valid_mask][:token_count]

        all_sample_embeds.append(sample_flat)
        window_idx += n_windows

    return torch.cat(all_sample_embeds, dim=0)
```

This requires passing `windows_per_sample` from the processor to the model, which may need API changes.

```python
# Constants from processor
SAMPLING_RATE = 16000
CHUNK_LENGTH = 30.0
MAX_AUDIO_LEN = 600
WINDOW_SIZE = int(SAMPLING_RATE * CHUNK_LENGTH)
MAX_WINDOWS = int(MAX_AUDIO_LEN // CHUNK_LENGTH)

def get_windows_per_sample(batch_paths):
    """Calculate number of windows for each audio file."""
    windows_per_sample = []
    for path in batch_paths:
        audio, sr = librosa.load(path, sr=SAMPLING_RATE)
        n_samples = len(audio)
        n_windows = max(1, (n_samples + WINDOW_SIZE - 1) // WINDOW_SIZE)
        if n_windows > MAX_WINDOWS:
            n_windows = MAX_WINDOWS
        windows_per_sample.append(n_windows)
    return windows_per_sample
```

--- Comment by ebezzam at 2026-02-19T16:58:05Z ---
@IvanBarabanau thanks for the reproducers and suggested fix! (Esp. the `create_aggressive_audio` and `create_upbeat_audio` methods, I don't why but I find it super cool/funny how you created the different audio 😛)

Could you open a PR with your suggested fix and passing `windows_per_sample` to the model? For the latter, if you find that it's really necessary, you'll have to:
- add it to the processor outputs
- add it here as well: https://github.com/huggingface/transformers/blob/1e31876d0eef7d046ad5d2813f67a73026fcec9c/src/transformers/models/audioflamingo3/processing_audioflamingo3.py#L201
- add it to the forward inputs: https://github.com/huggingface/transformers/blob/1e31876d0eef7d046ad5d2813f67a73026fcec9c/src/transformers/models/audioflamingo3/modular_audioflamingo3.py#L189
- add the corresponding logic here: https://github.com/huggingface/transformers/blob/1e31876d0eef7d046ad5d2813f67a73026fcec9c/src/transformers/models/audioflamingo3/modular_audioflamingo3.py#L296

--- Comment by github-actions[bot] at 2026-03-22T08:04:17Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

--- Comment by vasqu at 2026-03-25T13:30:44Z ---
gentle ping @IvanBarabanau @ebezzam

--- Comment by ebezzam at 2026-03-25T14:41:06Z ---
@vasqu we've actually found the issue with the model author when integrating their new model #43538, so this issue can be closed!

See this comment: https://github.com/huggingface/transformers/pull/43538/changes#r2948358594


ISSUE #44153: [Bug] Glm46VImageProcessorFast.get_number_of_image_patches() ignores self.size, uses hardcoded longest_edge
State: closed  |  Labels:
Author: SteadfastAsArt  |  Created: 2026-02-19T10:57:12Z

## Bug Description

`get_number_of_image_patches()` in both `image_processing_glm46v_fast.py` and `image_processing_glm46v.py` ignores `self.size` and falls back to a **hardcoded** default when `images_kwargs` does not include `size`:

```python
# image_processing_glm46v_fast.py line 182 (same in slow processor, line 463)
size = images_kwargs.get("size", {"shortest_edge": 112 * 112, "longest_edge": 28 * 28 * 15000})
#                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#                                  Hardcoded 11,760,000 — ignores self.size entirely
```

This means a processor loaded from a config that sets a different `longest_edge` will silently use the wrong value.

## Affected Model: GLM-OCR

`zai-org/GLM-OCR` sets `longest_edge = 9,633,792` in its `preprocessor_config.json`, but `get_number_of_image_patches()` uses `11,760,000` (the class default).

```python
proc = AutoImageProcessor.from_pretrained("zai-org/GLM-OCR")
print(proc.size)
# → {'shortest_edge': 12544, 'longest_edge': 9633792}

# But get_number_of_image_patches() uses 11,760,000 — wrong!
n = proc.get_number_of_image_patches(3000, 3000, {})
# Returns 29584 patches → 7396 tokens  (using 11,760,000)
# Correct value would be 24336 patches → 6084 tokens  (using 9,633,792)
```

## Downstream Impact

vLLM uses `_get_num_multimodal_tokens()` (which calls `get_number_of_image_patches()` with empty `images_kwargs`) to profile the KV cache before serving. For GLM-OCR the inflated estimate causes:

```
RuntimeError: split_with_sizes expects sum of split_sizes to equal the tensor's size
along the split_dimension, but got split_sizes=[7396] and tensor's size=6084
along dimension 0.
```

Related vLLM issue: https://github.com/vllm-project/vllm/issues/34040

## Fix

Replace the hardcoded fallback with `self.size` in both files:

```python
# Before (broken):
size = images_kwargs.get("size", {"shortest_edge": 112 * 112, "longest_edge": 28 * 28 * 15000})

# After (correct):
size = images_kwargs.get("size", self.size)
```

This is a one-line change in each of the two processor files.

## Environment

- `transformers==5.1.0`
- `python==3.13`
- Model: `zai-org/GLM-OCR`

--- Comment by SteadfastAsArt at 2026-02-19T10:59:03Z ---
**Update:** After checking, this bug is already fixed in transformers **v5.2.0** (released 2026-02-16):

```python
# image_processing_glm46v.py line 463 in v5.2.0:
size = images_kwargs.get("size", self.size)  # ✓ Fixed
```

The bug was present in v5.1.0 (released 2026-02-05). Users on v5.1.0 hitting this issue with GLM-OCR + vLLM should upgrade to v5.2.0.

Closing as already fixed.


ISSUE #44121: [Model Request] Add OpenAI Weight-Sparse Transformer (circuit-sparsity / circuitgpt)
State: open  |  Labels: New model
Author: dtiourine  |  Created: 2026-02-18T06:33:19Z

### Model description

Hello,

OpenAI recently released research on [Weight-sparse transformers](https://openai.com/index/understanding-neural-networks-through-sparse-circuits/). These models are specifically trained with weight sparsity for mechanistic interpretability and circuit analysis.

I would like to contribute an official implementation to `transformers` to make these models more accessible for the interpretability research community.  Currently, running these models via the Hub requires `trust_remote_code=True`.

I would like to open a PR to work on this if the maintainers are open to it. Proposing to call this `CircuitGPT` based on the circuitgpt tag used by the authors on their official HF upload.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

Paper: https://arxiv.org/abs/2511.13653

Repo: https://github.com/openai/circuit_sparsity

Hugging Face Hub: https://huggingface.co/openai/circuit-sparsity

--- Comment by Rocketknight1 at 2026-02-18T14:12:19Z ---
Are there any checkpoints for this? We generally don't add models until significant pre-trained checkpoints exist - if we added every experimental architecture to `transformers` then the library would explode 😅

--- Comment by dtiourine at 2026-02-18T17:56:35Z ---
Yes, the authors published a 0.4B checkpoint on the HF Hub ([openai/circuit-sparsity](https://huggingface.co/openai/circuit-sparsity)) called `csp_yolo2`, and additional model weights/checkpoints are available in their [GitHub repo](https://github.com/openai/circuit_sparsity/).


ISSUE #44117: TOKENIZER_MAPPING_NAMES sometimes returns None, but from_pretrained assumes otherwise
State: closed  |  Labels: bug
Author: DavidMChan  |  Created: 2026-02-17T23:23:08Z

### System Info

When loading a tokenizer with AutoTokenizer (src/transformers/models/auto/tokenization_auto.py), on L652 in from_pretained the code tries to remove "Fast" from the tokenizer mapping for old-style models:
```
if (
            tokenizer_auto_map is None
            and tokenizer_config_class is not None
            and config_model_type is not None
            and config_model_type != ""
            # Here:
            and TOKENIZER_MAPPING_NAMES.get(config_model_type, "").replace("Fast", "")
            != tokenizer_config_class.replace("Fast", "")
        ):
```
Permalink: https://github.com/huggingface/transformers/blob/ffbea2db927e4a0fb4bc8615c909ae277d8df0e6/src/transformers/models/auto/tokenization_auto.py#L652

However, sometimes ` TOKENIZER_MAPPING_NAMES.get(config_model_type, "")` returns `None`, instead of an empty string (if the config_model_type is in the dict, but has value None), leading to models failing to load with an attribute error
```
AttributeError: 'NoneType' object has no attribute 'replace'
```

This could be fixed as follows:
```
and (TOKENIZER_MAPPING_NAMES.get(config_model_type, "") or "")
```

Or by some other reasonable check.

### Who can help?

Looks like this was introduced in #42894 - @ArthurZucker  and @itazap

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Run:

```
from transformers import AutoProcessor
processor = AutoProcessor.from_pretrained('google/siglip2-so400m-patch14-384')
```

### Expected behavior

It shouldn't throw an attribute error :)

--- Comment by Rocketknight1 at 2026-02-18T14:05:31Z ---
Fixed in https://github.com/huggingface/transformers/pull/44127 I think! Feel free to ping me or reopen if the issue is still there after you install from `main`.

--- Comment by jkj-dev-rg at 2026-02-25T13:27:02Z ---
@Rocketknight1
Im also getting the same error now, currently im using the latest vesion 5.2
PS C:\Users\jyoth> docker exec -it vibrant_jennings pip show transformers
Name: transformers
Version: 5.2.0
Summary: Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /opt/venv/lib/python3.10/site-packages
Requires: huggingface-hub, numpy, packaging, pyyaml, regex, safetensors, tokenizers, tqdm, typer-slim
Required-by:

Error trace back:

{"status":"error","message":"Full traceback:

Traceback (most recent call last):
  File "/app/src/modules/embedding/siglip_embedding_server.py", line 368, in load_model
    model, processor, image_processor, tokenizer, device =
      load_siglip_robust(dest_dir, EXPECTED_DIM, local_files_only=True)

  File "/app/src/modules/embedding/siglip_embedding_server.py", line 103, in load_siglip_robust
    _cached_processor = AutoProcessor.from_pretrained(model_name_or_path, **processor_kwargs)

  File "/opt/venv/lib/python3.10/site-packages/transformers/models/auto/processing_auto.py", line 402, in from_pretrained
    return processor_class.from_pretrained(...)

  File "/opt/venv/lib/python3.10/site-packages/transformers/processing_utils.py", line 1400, in from_pretrained
    args = cls._get_arguments_from_pretrained(...)

  File "/opt/venv/lib/python3.10/site-packages/transformers/processing_utils.py", line 1514, in _get_arguments_from_pretrained
    tokenizer = cls._load_tokenizer_from_pretrained(...)

  File "/opt/venv/lib/python3.10/site-packages/transformers/processing_utils.py", line 1461, in _load_tokenizer_from_pretrained
    tokenizer = auto_processor_class.from_pretrained(...)

  File "/opt/venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 652, in from_pretrained
    and TOKENIZER_MAPPING_NAMES.get(config_model_type, "").replace("Fast", "")
AttributeError: 'NoneType' object has no attribute 'replace'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/src/modules/embedding/siglip_embedding_server.py", line 103, in load_siglip_robust
    _cached_processor = AutoProcessor.from_pretrained(model_name_or_path, **processor_kwargs)

  (same Transformers call chain as above...)

  File "/opt/venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 652, in from_pretrained
    and TOKENIZER_MAPPING_NAMES.get(config_model_type, "").replace("Fast", "")
AttributeError: 'NoneType' object has no attribute 'replace'
"}

same code base run in local but not in docker, both local and docker  have same transformers version,
in local model artifacts were provided after downloading from hf  siglip2-giant-opt-patch16-384, same model were mounted -volume to docker to load the model.

--- Comment by ArthurZucker at 2026-02-25T16:30:21Z ---
we'll release to fix this

--- Comment by jkj-dev-rg at 2026-02-25T18:29:30Z ---
I pulled the main from git and installed the package. Now its working fine.
BTW May I know the reason behind the issue. It worked in local environment with same package versions but not in docker build environment. @ArthurZucker

--- Comment by IDoCodingStuffs at 2026-02-25T18:51:11Z ---
AutoProcessor is broken until this lands (below fails on `transformers>=5.1.0`, works on `transformers==4.57.6`):

```
model_id = "google/siglip2-large-patch16-512"
processor = AutoProcessor.from_pretrained(model_id,)
```

--- Comment by ArthurZucker at 2026-03-02T09:06:04Z ---
Happy to hear @jkj-dev-rg