ISSUE #43856: Inefficient memory usage during Qwen3 MoE training
State: closed  |  Labels: bug
Author: yurkoff-mv  |  Created: 2026-02-09T09:48:03Z

### System Info

- `transformers` version: 4.57.3
- Platform: Linux-6.8.0-90-generic-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 0.36.0
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0+cu126 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA H100 80GB HBM3

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

[train_qlora_memory.py](https://github.com/user-attachments/files/25179495/train_qlora_memory.py)
[adapter_config.json](https://github.com/user-attachments/files/25179496/adapter_config.json)

Run with:
```
CUDA_VISIBLE_DEVICES=0 PYTORCH_ALLOC_CONF=expandable_segments:True,garbage_collection_threshold:0.8,max_split_size_mb:128 python train_qlora_memory.py "Qwen3/Qwen3-30B-A3B-Instruct-2507" --max_seq_length 1024 --batch_size 1 --max_steps 5 --path_config adapter_config.json --use_liger --use_gc
```

Output:
```
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.00 MiB. GPU 0 has a total capacity of 79.19 GiB of which 2.50 MiB is free. Process 2253701 has 79.18 GiB memory in use. Of the allocated memory 51.31 GiB is allocated by PyTorch, and 27.15 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
```
Output confirming memory inefficiency: `27.15 GiB is reserved by PyTorch but unallocated`!
--

By the way, quantizing the model using **bitsandbytes** is also[ inefficient and leads to high memory consumption](https://github.com/bitsandbytes-foundation/bitsandbytes/issues/1849#issuecomment-3870385400): with **4-bit quantization** of the model **30B**, the actual memory consumption after loading the model is **46.75 GB** instead of the expected **16 GB**!

### Expected behavior

```
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:19<00:00,  1.24s/it]
LoraConfig(task_type=None, peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, peft_version='0.18.0', base_model_name_or_path=None, revision=None, inference_mode=False, r=64, target_modules={'q_proj', 'o_proj', 'k_proj', 'v_proj', 'down_proj'}, exclude_modules=None, lora_alpha=64, lora_dropout=0.0, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', trainable_token_indices=None, loftq_config={}, eva_config=None, corda_config=None, use_dora=False, alora_invocation_tokens=None, use_qalora=False, qalora_group_size=16, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False), lora_bias=False, target_parameters=None, arrow_config=None, ensure_weight_tying=False)
trainable params: 1,160,773,632 || all params: 31,692,896,256 || trainable%: 3.6626
Gradient Checkpointing enabled
step   0 seq_length 1024 loss 11.360670 time 18.22s
step   1 seq_length 1024 loss 11.471191 time 14.65s
step   2 seq_length 1024 loss 11.253231 time 13.13s
step   3 seq_length 1024 loss 11.337120 time 11.62s
step   4 seq_length 1024 loss 11.379625 time 11.16s
cuda memory avg: 33269MB
cuda memory max: 45740MB
total time: 76.52s
file size: 4429.8MB
```

--- Comment by ArthurZucker at 2026-02-09T15:49:12Z ---
Hey, can you try to upgrade your `transformers` version?

--- Comment by yurkoff-mv at 2026-02-16T17:41:50Z ---
The problem persists in `transformers==4.57.6`

--- Comment by Cyrilvallez at 2026-02-16T19:08:51Z ---
We meant on latest transformers v5! Latest is now v5.2.0

--- Comment by yurkoff-mv at 2026-02-17T05:46:57Z ---
I also had to update the Liger Kernel library to version  0.7.0. But now OOM occurs much earlier (in `prepare_model_for_kbit_training`):
```
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 531/531 [00:24<00:00, 21.93it/s, Materializing param=model.norm.weight]
Traceback (most recent call last):
  File "/workspace/tests/train_qlora.py", line 259, in <module>
    train(
  File "/workspace/tests/train_qlora.py", line 103, in train
    model = prepare_model_for_kbit_training(model)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/peft/utils/other.py", line 175, in prepare_model_for_kbit_training
    param.data = param.data.to(torch.float32)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.50 GiB. GPU 0 has a total capacity of 79.19 GiB of which 318.50 MiB is free. Process 372740 has 78.87 GiB memory in use. Of the allocated memory 70.88 GiB is allocated by PyTorch, and 7.40 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
```

My environment:
```
Package                            Version
---------------------------------- ------------
accelerate                         1.12.0
bitsandbytes                       0.49.2
datasets                           4.3.0
liger-kernel                       0.7.0
peft                               0.18.1
safetensors                        0.7.0
tokenizers                         0.22.1
torch                              2.9.0+cu126
torchao                            0.14.1+cu126
transformers                       5.2.0
triton                             3.5.0
trl                                0.24.0
```

--- Comment by ArthurZucker at 2026-02-17T09:47:00Z ---
`prepare_model_for_kbit_training(model)` is for peft and seems to cast your weights to float32 -> more ram.
I would suggest making sure that post weight loading -> you have the expected ram usage, then you can open the issue on Peft!

--- Comment by github-actions[bot] at 2026-03-14T08:04:28Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #43854: Unable to load `zai-org/GLM-4.7-Flash` model correctly in the unit tests
State: closed  |  Labels: bug
Author: kaixuanliu  |  Created: 2026-02-09T07:48:27Z

### System Info

```
- `transformers` version: 5.2.0.dev0
- Platform: Linux-5.4.292-1.el8.elrepo.x86_64-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 1.4.0
- Safetensors version: 0.5.3
- Accelerate version: 1.3.0
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: FSDP
        - mixed_precision: no
        - use_cpu: False
        - debug: False
        - num_processes: 4
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - enable_cpu_affinity: False
        - fsdp_config: {'fsdp_activation_checkpointing': False, 'fsdp_auto_wrap_policy': 'TRANSFORMER_BASED_WRAP', 'fsdp_cpu_ram_efficient_loading': True, 'fsdp_offload_params': False, 'fsdp_reshard_after_forward': True, 'fsdp_state_dict_type': 'SHARDED_STATE_DICT', 'fsdp_transformer_layer_cls_to_wrap': 'Gemma3DecoderLayer', 'fsdp_version': 2}
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0+cu128 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100 80GB PCIe
```

### Who can help?

text models: @ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. pip install git+https://github.com/huggingface/transformers.git
2. run this example code:
```
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "zai-org/GLM-4.7-Flash"
messages = [{"role": "user", "content": "hello"}]
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
model = AutoModelForCausalLM.from_pretrained(
    pretrained_model_name_or_path=MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
inputs = inputs.to(model.device)
generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
output_text = tokenizer.decode(generated_ids[0][inputs.input_ids.shape[1]:])
print(output_text)

```
And it will output log w/ content:
```
Key                                                 | Status     |  |
----------------------------------------------------+------------+--+-
model.layers.47.embed_tokens.weight                 | UNEXPECTED |  |
model.layers.47.hnorm.weight                        | UNEXPECTED |  |
model.layers.47.enorm.weight                        | UNEXPECTED |  |
model.layers.47.shared_head.head.weight             | UNEXPECTED |  |
model.layers.47.mlp.gate.e_score_correction_bias    | UNEXPECTED |  |
model.layers.47.mlp.shared_experts.up_proj.weight   | UNEXPECTED |  |
model.layers.47.self_attn.q_a_layernorm.weight      | UNEXPECTED |  |
model.layers.47.mlp.shared_experts.gate_proj.weight | UNEXPECTED |  |
model.layers.47.post_attention_layernorm.weight     | UNEXPECTED |  |
model.layers.47.input_layernorm.weight              | UNEXPECTED |  |
model.layers.47.self_attn.kv_b_proj.weight          | UNEXPECTED |  |
model.layers.47.mlp.experts.gate_up_proj            | UNEXPECTED |  |
model.layers.47.mlp.gate.weight                     | UNEXPECTED |  |
model.layers.47.self_attn.q_b_proj.weight           | UNEXPECTED |  |
model.layers.47.mlp.shared_experts.down_proj.weight | UNEXPECTED |  |
model.layers.47.self_attn.o_proj.weight             | UNEXPECTED |  |
model.layers.47.eh_proj.weight                      | UNEXPECTED |  |
model.layers.47.self_attn.kv_a_proj_with_mqa.weight | UNEXPECTED |  |
model.layers.47.shared_head.norm.weight             | UNEXPECTED |  |
model.layers.47.mlp.experts.down_proj               | UNEXPECTED |  |
model.layers.47.self_attn.q_a_proj.weight           | UNEXPECTED |  |
model.layers.47.self_attn.kv_a_layernorm.weight     | UNEXPECTED |  |
```

### Expected behavior

All the model weight be correctly loaded.

--- Comment by zucchini-nlp at 2026-02-09T08:44:08Z ---
I think we need to ask someone from Zai team to be sure. The model has mismatching `num_hidden_layers` in config file and in the saved checkpoint
cc @zRzRzRzRzRzRzR ?

--- Comment by zRzRzRzRzRzRzR at 2026-02-10T13:51:28Z ---
This is the MTP layer, not the model layer
The model layers are 0-46, and layer 47 should be ignored when I submit the PR.

--- Comment by zucchini-nlp at 2026-02-10T13:59:52Z ---
lets' add it in `keys_to_ignore_on_load`? So that it doesn't fire with unactionable warnings

--- Comment by ArthurZucker at 2026-02-10T14:00:58Z ---
```python
@auto_docstring
class GlmMoeDsaModel(GlmMoeDsaPreTrainedModel):
    _keys_to_ignore_on_load_unexpected = [r"model\.layers\.78.*"]
```
yep probably a typo if its for dsa

but for GLMLite:
```python

@auto_docstring
class Glm4MoeLiteModel(Glm4MoeLitePreTrainedModel):
    _keys_to_ignore_on_load_unexpected = [r"model\.layers\.47.*"]
```

--- Comment by ArthurZucker at 2026-02-10T14:02:26Z ---
will fix

--- Comment by kaixuanliu at 2026-02-11T01:40:07Z ---
@zRzRzRzRzRzRzR So this should not cause the wrong output, right? But here is a mis-match expect output in this test case: `pytest -rA tests/models/glm4_moe_lite/test_modeling_glm4_moe_lite.py::Glm4MoeIntegrationTest::test_compile_static_cache`, can you help check?


huggingface

ISSUE #43846: huggingface
State: closed  |  Labels:
Author: Pediboi666  |  Created: 2026-02-08T21:37:04Z

https://github.com/likaia/nginxpulse/commit/f363839f502181fdfa6ff0a90471c6136d8fbe70


huggingface

ISSUE #43845: huggingface
State: closed  |  Labels:
Author: Pediboi666  |  Created: 2026-02-08T21:36:36Z

https://github.com/likaia/nginxpulse/commit/9940b2c008aba73d976992c0020b2c1249b59220


ISSUE #43844: Gradient abnormally increases when training a randomly initialized model with HfDeepSpeedConfig + ZeRO-3
State: closed  |  Labels: bug
Author: wxhcore  |  Created: 2026-02-08T15:32:38Z

### System Info

Environment information:
- Operating System: Linux
- Python version: 3.10
- `torch==2.7.1`
- `deepspeed==0.18.3`
- `transformers==4.57.3`

Full dependencies can be found in the project's [`pyproject.toml`](https://github.com/wxhcore/bumblecore/blob/main/pyproject.toml).

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

### Reproduction Steps

1. **Clone the [bumblecore](https://github.com/wxhcore/bumblecore) repository and set up the environment**
   ```bash
   git clone https://github.com/wxhcore/bumblecore.git
   cd bumblecore
   conda create -n bumblecore_env python=3.10 -y && conda activate bumblecore_env
   pip install -e .
   ```

2. **Modify the pretraining script to enable ZeRO-3**
   Edit `./scripts/pretrain.sh` to set the training data path and add the DeepSpeed config:
   ```bash
   --dataset_path ./datasets/pretrain_zh_demo.json \
   --deepspeed_config_path ./configs/deepspeed/ds_z3_config.json \
   ```

3. **Remove the ZeRO-3 training stage restriction**
   Open [`src/bumblecore/training/base_trainer.py`](https://github.com/wxhcore/bumblecore/blob/6b5dc6cb4ed5fac32c5016b7d8bc4b81cd248e0d/src/bumblecore/training/base_trainer.py#L172) and change:
   ```python
   if zero_stage == 3 and self.config.training_stage != "pretrain":
   ```
   to:
   ```python
   if zero_stage == 3:
   ```
   (This allows `HfDeepSpeedConfig` to be used during pretraining.)

4. **Start training**
   ```bash
   bash scripts/pretrain.sh
   ```
   This will launch training with a randomly initialized model + ZeRO-3 + `HfDeepSpeedConfig`, reproducing the abnormal gradient increase.

### Expected behavior

### Bug Report: Abnormal Gradient Increase When Training a Randomly Initialized Model with `HfDeepSpeedConfig` + ZeRO-3

#### Description
When using DeepSpeed ZeRO-3 together with `HfDeepSpeedConfig(ds_config)`, **gradients become abnormally large during training if the model is randomly initialized (i.e., not loaded from pretrained weights)**.

We conducted several controlled experiments:
- ✅ **Pretrained or randomly initialized models** + ZeRO-3 (**without** `HfDeepSpeedConfig`) → gradients normal
- ✅ **Pretrained model** + `HfDeepSpeedConfig` + ZeRO-3 → gradients normal
- ✅ Any setup with **ZeRO-2** (**without** `HfDeepSpeedConfig`)  → gradients normal
- ❌ **Randomly initialized model** + `HfDeepSpeedConfig` + ZeRO-3 → **abnormal gradient increase**

This indicates the issue is specific to the combination of **random initialization + `HfDeepSpeedConfig` + ZeRO-3**.

> Gradient distribution comparison:
> - Normal case:
>   ![Normal gradients](https://github.com/user-attachments/assets/16ff1ee5-e0c2-4940-8f1b-a45350fc3962)
> - Abnormal case:
>   ![Abnormal gradients](https://github.com/user-attachments/assets/7c3c080f-43cb-442f-a076-6b1c198b301d)

#### Preliminary Analysis
According to the [Transformers DeepSpeed integration guide](https://huggingface.co/docs/transformers/v5.1.0/en/main_classes/deepspeed#non-trainer-deepspeed-integration), `HfDeepSpeedConfig` must be instantiated before model creation to enable ZeRO-3 parameter partitioning.

However, **when the model is randomly initialized**, this mechanism may interfere with proper parameter initialization or registration order, leading to incorrect gradient computation under ZeRO-3.

#### Expected Behavior
Training a randomly initialized model with `HfDeepSpeedConfig` + ZeRO-3 should produce **stable gradients**, consistent with other configurations—**not an abnormal increase**.

--- Comment by wxhcore at 2026-02-08T15:51:09Z ---
Hi @tohtana,
This issue appears to involve an interaction between HfDeepSpeedConfig (from Transformers) and ZeRO-3 when training randomly initialized models. The gradients become abnormally large only in this specific setup.
Could you please take a look and advise whether this might be related to DeepSpeed’s parameter initialization or partitioning logic under ZeRO-3?
Thank you very much for your time!

--- Comment by tohtana at 2026-02-08T22:05:50Z ---
Thank you for reporting this. I opened a PR to fix it. See #43847 for the details.

--- Comment by wxhcore at 2026-02-09T12:39:45Z ---
> Thank you for reporting this. I opened a PR to fix it. See [#43847](https://github.com/huggingface/transformers/pull/43847) for the details.

Thank you very much for your help!

--- Comment by github-actions[bot] at 2026-03-11T08:07:42Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


Proposal to add Qwen3-ASR support

ISSUE #43837: Proposal to add Qwen3-ASR support
State: open  |  Labels: New model
Author: mbtariq82  |  Created: 2026-02-08T12:00:34Z

### Model description

Model: Qwen3-ASR
Repository: https://github.com/QwenLM/Qwen3-ASR
Paper: https://arxiv.org/abs/2601.21337
License: Apache 2.0

Qwen3-ASR would make a good addition to Transformers. It uses models that are already integrated in the library such as Qwen2TokenizerFast and WhisperFeatureExtractor. Transformers already has other ASR models such as Qwen2-Audio, Parakeet and SeamlessM4T.

Would the maintainers be open to this addition? I've created a basic PR adding the processor and some corresponding tests for now. If I get the go ahead, I'll add the rest of the model.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

https://huggingface.co/spaces/Qwen/Qwen3-ASR


Fett

ISSUE #43835: Fett
State: closed  |  Labels:
Author: Pediboi666  |  Created: 2026-02-08T11:12:19Z

Fett

_Ursprünglich gepostet von @Pediboi666 in https://github.com/huggingface/transformers/pull/42845#pullrequestreview-3769644934_


ISSUE #43834: [i18n-<languageCode>] Translating docs to <languageName>
State: closed  |  Labels: WIP
Author: Pediboi666  |  Created: 2026-02-08T11:03:54Z

<!--Deutsch
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [x] [index.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.md) https://github.com/huggingface/transformers/pull/20180
- [x] [quicktour.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.md) (waiting for initial PR to go through)
- [ ] [installation.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.md).

## Tutorial section
- [ ] [pipeline_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.md)
- [ ]  [autoclass_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/autoclass_tutorial.md)
- [ ]  [preprocessing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.md)
- [ ]  [training.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.md)
- [ ]  [accelerate.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.md)
- [ ]  [model_sharing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.md)
- [ ]  [multilingual.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.md)

<!--
Keep on adding more as you go 🔥
-->


ISSUE #43828: With torch.autocast, Phi-tiny-MoE-instruct raises an dtype mismatch error
State: closed  |  Labels: bug
Author: vinnamkim  |  Created: 2026-02-08T03:08:45Z

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 5.1.0
- Platform: Linux-5.15.0-168-generic-x86_64-with-glibc2.35
- Python version: 3.11.14
- Huggingface_hub version: 1.4.1
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: 0.18.5
- PyTorch version (accelerator?): 2.9.1+cu128 (CUDA)
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: Yes
- GPU type: NVIDIA A100-SXM4-80GB

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

This is a minimal Python script to reproduce
```python
import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-tiny-MoE-instruct", device_map="cuda", dtype="bfloat16")

x = torch.randint(0, 100, [1, 16], device="cuda")

with torch.autocast("cuda", torch.bfloat16):
    y = model(x)

print(y)
```

### Expected behavior

Expect no error but there is dtype mismatch in the grouped GEMM
```
Loading weights: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 485/485 [00:02<00:00, 239.58it/s, Materializing param=model.norm.weight]
Traceback (most recent call last):
  File "/local/mnt/workspace/users/vinnkim/QAT-LlaMA/test.py", line 12, in <module>
    y = model(x)
        ^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/utils/generic.py", line 834, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/models/phimoe/modeling_phimoe.py", line 845, in forward
    outputs: MoeModelOutputWithPast = self.model(
                                      ^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/utils/generic.py", line 1001, in wrapper
    outputs = func(self, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/models/phimoe/modeling_phimoe.py", line 682, in forward
    hidden_states = decoder_layer(
                    ^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/modeling_layers.py", line 93, in __call__
    return super().__call__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/models/phimoe/modeling_phimoe.py", line 584, in forward
    hidden_states = self.mlp(hidden_states)
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/models/phimoe/modeling_phimoe.py", line 541, in forward
    final_hidden_states = self.experts(hidden_states, selected_experts, routing_weights)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/integrations/moe.py", line 349, in forward
    return experts_forward(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/integrations/moe.py", line 251, in grouped_mm_experts_forward
    gate_up_out = _grouped_linear(
                  ^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/integrations/moe.py", line 192, in _grouped_linear
    out = torch._grouped_mm(input, weight.transpose(-2, -1), offs=offs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::BFloat16
```

--- Comment by ArthurZucker at 2026-02-09T12:45:29Z ---
thanks for reporting!


ISSUE #43827: Summarization/Translation docs still reference pipeline() after v5 pipeline removals
State: closed  |  Labels: bug
Author: math-hiyoko  |  Created: 2026-02-08T02:11:03Z

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 5.2.0.dev0
- Platform: Linux-6.6.105+-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 1.3.7
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0+cpu (NA)
- Using distributed or parallel set-up in script?: parallel set-up

### Who can help?

@stevhliu

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

According to the Transformers v5 migration guide, `SummarizationPipeline` and `TranslationPipeline` were removed in v5:

https://github.com/huggingface/transformers/blob/main/MIGRATION_GUIDE_V5.md#pipelines

However, the official task documentation for summarization and translation still contains inference examples using `transformers.pipeline()`.
https://huggingface.co/docs/transformers/en/tasks/summarization#inference
https://huggingface.co/docs/transformers/en/tasks/translation#inference

This is confusing for users on v5, because the shown examples no longer work as written (or suggest that the pipeline API is still supported for these tasks).

### Expected behavior

Remove `pipeline()`-based examples for summarization/translation


ISSUE #43825: pipeline() error message incorrectly suggests translation tasks are supported in v5
State: closed  |  Labels: bug
Author: math-hiyoko  |  Created: 2026-02-08T01:31:10Z

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 5.2.0.dev0
- Platform: Linux-6.6.105+-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 1.3.7
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0+cpu (NA)
- Using distributed or parallel set-up in script?: parallel

### Who can help?

@Rocketknight1

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When calling `transformers.pipeline()` with an unknown task, the error message lists `translation_XX_to_YY` as an available task. However, in Transformers v5, the default `pipeline()` does not provide the `"translation"` pipeline in `supported_tasks`, so attempting to use `translation_XX_to_YY` leads to a `KeyError: 'translation'`.

Also, calling `pipeline("translation")` raises:
`KeyError: "Invalid translation task translation, use 'translation_XX_to_YY' format"`
but since `"translation"` is not actually supported, the expected error should be closer to:
`KeyError: "Unknown task translation, available tasks are ..."`

Colab link: https://colab.research.google.com/drive/1aKTGHuuDhpdxAJlJB8txOmcyAKeNoz2i?usp=sharing

### Expected behavior

KeyError: "Unknown task hoge, available tasks are ['any-to-any', 'audio-classification', 'automatic-speech-recognition', 'depth-estimation', 'document-question-answering', 'feature-extraction', 'fill-mask', 'image-classification', 'image-feature-extraction', 'image-segmentation', 'image-text-to-text', 'image-to-image', 'keypoint-matching', 'mask-generation', 'ner', 'object-detection', 'question-answering', 'sentiment-analysis', 'table-question-answering', 'text-classification', 'text-generation', 'text-to-audio', 'text-to-speech', 'token-classification', 'video-classification', 'visual-question-answering', 'vqa', 'zero-shot-audio-classification', 'zero-shot-classification', 'zero-shot-image-classification', 'zero-shot-object-detection']

--- Comment by Rocketknight1 at 2026-02-09T12:38:20Z ---
The translation pipeline has been removed in v5! This is because the quality of translations from Seq2Seq models was generally worse than using LLMs instead, and we didn't want pipelines to mislead users. We probably need to remove all the leftover translation references in the pipeline files entirely.

--- Comment by math-hiyoko at 2026-02-09T13:08:45Z ---
@Rocketknight1

Thanks for the clarification, I understand that the translation pipeline itself is removed in v5.

The actual issue here is:
- `pipeline()` still lists `translation_XX_to_YY` tasks in the "available tasks" message, which makes it look like translation is supported.
```python
pipeline('hoge')
```
```text
KeyError: "Unknown task hoge, available tasks are ['any-to-any', 'audio-classification', 'automatic-speech-recognition', 'depth-estimation', 'document-question-answering', 'feature-extraction', 'fill-mask', 'image-classification', 'image-feature-extraction', 'image-segmentation', 'image-text-to-text', 'image-to-image', 'keypoint-matching', 'mask-generation', 'ner', 'object-detection', 'question-answering', 'sentiment-analysis', 'table-question-answering', 'text-classification', 'text-generation', 'text-to-audio', 'text-to-speech', 'token-classification', 'video-classification', 'visual-question-answering', 'vqa', 'zero-shot-audio-classification', 'zero-shot-classification', 'zero-shot-image-classification', 'zero-shot-object-detection', 'translation_XX_to_YY']"
```

- When explicitly passing `translation_XX_to_YY`, the resulting error message is misleading.
```python
pipeline('translation_en_to_fr')
```
```text
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipython-input-124266547.py in <cell line: 0>()
----> 1 pipeline('translation_en_to_fr')

2 frames
/usr/local/lib/python3.12/dist-packages/transformers/pipelines/base.py in check_task(self, task)
   1349             tokens = task.split("_")
   1350             if len(tokens) == 4 and tokens[0] == "translation" and tokens[2] == "to":
-> 1351                 targeted_task = self.supported_tasks["translation"]
   1352                 task = "translation"
   1353                 return task, targeted_task, (tokens[1], tokens[3])

KeyError: 'translation'
```

PR #43826 addresses this by fixing the pipeline error message and preventing the misleading KeyError, which resolves the core confusion.

--- Comment by Rocketknight1 at 2026-02-09T18:37:00Z ---
Yes, this cleanup is good, but there are a few other areas that need fixing too! I made a PR at https://github.com/huggingface/transformers/pull/43869 which includes your PR at #43826 as well as some other changes. Can you take a look and tell me if there's anything else you think I missed?

--- Comment by github-actions[bot] at 2026-03-10T08:07:03Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #43824: ImportError: cannot import name 'Qwen2_5_VLForConditionalGeneration' from 'transformers'
State: closed  |  Labels: bug
Author: waltwalt36  |  Created: 2026-02-08T00:48:13Z

### System Info

```
Traceback (most recent call last):
  File "/home/wbrione/.conda/envs/watt_ai/bin/transformers", line 3, in <module>
    from transformers.cli.transformers import main
  File "/home/wbrione/.conda/envs/watt_ai/lib/python3.10/site-packages/transformers/cli/transformers.py", line 22, in <module>
    from transformers.cli.serve import Serve
  File "/home/wbrione/.conda/envs/watt_ai/lib/python3.10/site-packages/transformers/cli/serve.py", line 360, in <module>
    class Serve:
  File "/home/wbrione/.conda/envs/watt_ai/lib/python3.10/site-packages/transformers/cli/serve.py", line 588, in Serve
    validator: TypeAdapter,
NameError: name 'TypeAdapter' is not defined
```

### Who can help?

@yonigozlan @molbap

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
!pip install --upgrade --force-reinstall torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124

import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

import torch
import os
import json
import time
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

# Load AWQ quantized model
# Note: Using "auto" for torch_dtype as recommended in the documentation
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-72B-Instruct-AWQ",
    torch_dtype = "auto",
    device_map = "auto"
)
```

### Expected behavior

Import Qwen2 from transformers so I can run the rest of my code.

--- Comment by pragnyanramtha at 2026-02-08T09:24:35Z ---
i'm looking into this, i'll take an attempt to fix it

--- Comment by pragnyanramtha at 2026-02-08T09:53:24Z ---
i think i found the issue,

```transformers/cli/serve``` has a bug where TypeAdapter was used in a type hint but fail-safe logic for missing dependencies (like pydantic) caused NameError at import time if evaluated eagerly.

--- Comment by waltwalt36 at 2026-02-09T04:03:28Z ---
Bet so I need to install Pydantic and I'm good?

--- Comment by waltwalt36 at 2026-02-09T04:08:06Z ---
Also, my research leader told me to install these requirements.txt file but I got another error

```txt
requirements.txt

accelerate==1.12.0
aiohappyeyeballs==2.6.1
aiohttp==3.13.3
aiosignal==1.4.0
anyio==4.12.1
argon2-cffi==25.1.0
argon2-cffi-bindings==25.1.0
arrow==1.4.0
asttokens==3.0.1
async-lru==2.1.0
async-timeout==5.0.1
attrs==25.4.0
audioread==3.1.0
autoawq==0.2.9
av==16.1.0
babel==2.18.0
beautifulsoup4==4.14.3
bleach==6.3.0
certifi==2026.1.4
cffi==2.0.0
charset-normalizer==3.4.4
comm==0.2.3
cuda-bindings==12.9.4
cuda-pathfinder==1.3.3
datasets==4.5.0
debugpy==1.8.20
decorator==5.2.1
decord==0.6.0
defusedxml==0.7.1
dill==0.4.0
exceptiongroup==1.3.1
executing==2.2.1
fastjsonschema==2.21.2
filelock==3.20.3
fqdn==1.5.1
frozenlist==1.8.0
fsspec==2025.10.0
h11==0.16.0
hf-xet==1.2.0
httpcore==1.0.9
httpx==0.28.1
huggingface_hub==0.36.2
idna==3.11
imageio-ffmpeg==0.6.0
ipykernel==7.2.0
ipython==8.38.0
isoduration==20.11.0
jedi==0.19.2
Jinja2==3.1.6
joblib==1.5.3
json5==0.13.0
jsonpointer==3.0.0
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
jupyter-events==0.12.0
jupyter-lsp==2.3.0
jupyter_client==8.8.0
jupyter_core==5.9.1
jupyter_server==2.17.0
jupyter_server_terminals==0.5.4
jupyterlab==4.5.3
jupyterlab_pygments==0.3.0
jupyterlab_server==2.28.0
lark==1.3.1
lazy_loader==0.4
librosa==0.11.0
llvmlite==0.46.0
MarkupSafe==3.0.3
matplotlib-inline==0.2.1
mistune==3.2.0
mpmath==1.3.0
msgpack==1.1.2
multidict==6.7.1
multiprocess==0.70.18
nbclient==0.10.4
nbconvert==7.17.0
nbformat==5.10.4
nest-asyncio==1.6.0
networkx==3.4.2
notebook_shim==0.2.4
numba==0.63.1
numpy==2.2.6
nvidia-cublas-cu12==12.8.4.1
nvidia-cuda-cupti-cu12==12.8.90
nvidia-cuda-nvrtc-cu12==12.8.93
nvidia-cuda-runtime-cu12==12.8.90
nvidia-cudnn-cu12==9.10.2.21
nvidia-cufft-cu12==11.3.3.83
nvidia-cufile-cu12==1.13.1.3
nvidia-curand-cu12==10.3.9.90
nvidia-cusolver-cu12==11.7.3.90
nvidia-cusparse-cu12==12.5.8.93
nvidia-cusparselt-cu12==0.7.1
nvidia-nccl-cu12==2.27.5
nvidia-nvjitlink-cu12==12.8.93
nvidia-nvshmem-cu12==3.4.5
nvidia-nvtx-cu12==12.8.90
overrides==7.7.0
packaging @ file:///home/task_176104877067765/conda-bld/packaging_1761049113113/work
pandas==2.3.3
pandocfilters==1.5.1
parso==0.8.5
pexpect==4.9.0
pillow==12.1.0
platformdirs==4.5.1
pooch==1.9.0
prometheus_client==0.24.1
prompt_toolkit==3.0.52
propcache==0.4.1
psutil==7.2.2
ptyprocess==0.7.0
pure_eval==0.2.3
pyarrow==23.0.0
pycparser==3.0
Pygments==2.19.2
python-dateutil==2.9.0.post0
python-json-logger==4.0.0
pytz==2025.2
PyYAML==6.0.3
pyzmq==27.1.0
qwen-vl-utils==0.0.8
referencing==0.37.0
regex==2026.1.15
requests==2.32.5
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rfc3987-syntax==1.1.0
rpds-py==0.30.0
safetensors==0.7.0
scikit-learn==1.7.2
scipy==1.15.3
Send2Trash==2.1.0
six==1.17.0
soundfile==0.13.1
soupsieve==2.8.3
soxr==1.0.0
stack-data==0.6.3
sympy==1.14.0
terminado==0.18.1
threadpoolctl==3.6.0
tinycss2==1.4.0
tokenizers==0.22.2
tomli==2.4.0
torch==2.10.0
torchvision==0.25.0
tornado==6.5.4
tqdm==4.67.3
traitlets==5.14.3
transformers==4.56.1
triton==3.6.0
typing_extensions==4.15.0
tzdata==2025.3
uri-template==1.3.0
urllib3==2.6.3
wcwidth==0.6.0
webcolors==25.10.0
webencodings==0.5.1
websocket-client==1.9.0
xxhash==3.6.0
yarl==1.22.0
zstandard==0.25.0
```

```
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/home/task_176104877067765/conda-bld/packaging_1761049113113/work'
```

--- Comment by pragnyanramtha at 2026-02-09T04:09:15Z ---
Hopefully yes, but there are other optional dependencies you need to install too, like gptqmodel, qwen_vl_utils , accelerate etc

But you may see missing module errors that may actually tell you what to install

--- Comment by pragnyanramtha at 2026-02-09T04:12:43Z ---
Uff, i think your dependencies might be corrupted, i would highly recommend deleting the venv and initialising a new one

You could try :
```
pip install --no-cache-dir -r requirements.txt
```

--- Comment by waltwalt36 at 2026-02-09T04:15:07Z ---
I did make a fresh venv 😭, I'm going to try to install the optional dependencies and see what happens

--- Comment by waltwalt36 at 2026-02-09T04:25:04Z ---
Bruh, what other dependencies do I need? I installed Pydantic and still nada. Is there a doc somewhere with all the dependencies I need?

--- Comment by fbordignon at 2026-02-18T21:00:43Z ---
Installed the following and it seems to work:
pydantic
fastapi
uvicorn
openai

--- Comment by pragnyanramtha at 2026-02-19T02:45:18Z ---
@fbordignon, i think those are the optional dependencies for serving.

Somthing similar could be achieved by
```
pip install "transformers[serve]"
```

--- Comment by github-actions[bot] at 2026-03-15T08:06:27Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #43819: [BUG] DAC.from_latents does not match the forward pass with missing STE
State: closed  |  Labels: bug
Author: harshaljanjani  |  Created: 2026-02-07T14:44:24Z

### System Info

* `transformers` version: `5.0.0.dev0`
* Platform: `Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39`
* Python version: `3.12.3`
* `huggingface_hub` version: `1.3.2`
* `safetensors` version: `0.7.0`
* `accelerate` version: `1.12.0`
* Accelerate config: `not installed`
* DeepSpeed version: `not installed`
* PyTorch version (accelerator?): `2.9.1+cu128 (CUDA)`
* GPU type: `NVIDIA L4`
* NVIDIA driver version: `550.90.07`
* CUDA version: `12.4`

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import torch
from datasets import load_dataset, Audio
from transformers import DacModel, AutoProcessor

model_id = "descript/dac_16khz"
device = "cuda" if torch.cuda.is_available() else "cpu"
model = DacModel.from_pretrained(model_id).to(device).eval()
processor = AutoProcessor.from_pretrained(model_id)
librispeech_dummy = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
librispeech_dummy = librispeech_dummy.cast_column("audio", Audio(sampling_rate=processor.sampling_rate))
audio_sample = librispeech_dummy[0]["audio"]["array"]
inputs = processor(
  raw_audio=audio_sample,
  sampling_rate=processor.sampling_rate,
  return_tensors="pt",
).to(device)
input_values = inputs["input_values"]

with torch.no_grad():
  encoded = model.encoder(input_values)
  quant_repr_fwd, audio_codes, proj_latents, _, _ = model.quantizer(encoded)
  quant_repr_from_lat, _ = model.quantizer.from_latents(proj_latents)
  print(quant_repr_fwd)
  print(quant_repr_from_lat)
  print(torch.max(quant_repr_fwd - quant_repr_from_lat).abs().item())
```

[DAC.from_latents](https://github.com/huggingface/transformers/blob/main/src/transformers/models/dac/modeling_dac.py#L391-L400) in [DacResidualVectorQuantizer](https://github.com/huggingface/transformers/blob/main/src/transformers/models/dac/modeling_dac.py#L265) does not correctly apply the straight-through estimator before `out_proj`, unlike the correct [DacVectorQuantize.forward](https://github.com/huggingface/transformers/blob/main/src/transformers/models/dac/modeling_dac.py#L148-L150) pattern. This also breaks CI; the `tests/models/dac/test_modeling_dac.py::DacIntegrationTest` tests fail.

**Current Output:**

### Expected behavior

→ `DAC.from_latents` should exactly match the quantizer forward pass output.
→ `tests/models/dac/test_modeling_dac.py::DacIntegrationTest::test_quantizer_from_latents_integration_0_dac_16khz && DacIntegrationTest::test_quantizer_from_latents_integration_1_dac_24khz && DacIntegrationTest::test_quantizer_from_latents_integration_2_dac_44khz` integration tests pass without regressions

**Output After the Fix:**


ISSUE #43818: [Video-LLaVA] `Video-LLaVA-7B-hf` video_tower is missing temporal attention AND shares nearly identical weights with image_tower
State: closed  |  Labels: bug
Author: jong980812  |  Created: 2026-02-07T13:41:05Z

### System Info

### Problem

The HF-converted model `LanguageBind/Video-LLaVA-7B-hf` has two critical problems in its video tower:

1. **Missing `temporal_attn` layers**: The original `LanguageBind/Video-LLaVA-7B` video tower contains per-layer temporal attention for cross-frame reasoning. These are completely absent in the `-hf` version.
2. **video_tower and image_tower have nearly identical weights**: Only 3 out of ~300 parameter tensors differ between the two towers. This should not be the case — the original model uses separately pretrained LanguageBind-Video and LanguageBind-Image encoders with distinct weights.

### Evidence

**1. Original model has temporal attention in the video tower**

In `LanguageBind/Video-LLaVA-7B` ([model.safetensors.index.json](https://huggingface.co/LanguageBind/Video-LLaVA-7B/blob/main/model.safetensors.index.json)), the video tower contains `temporal_attn` layers per encoder block:

```
model.video_tower.video_tower.encoder.layers.X.temporal_attn.k_proj.weight
model.video_tower.video_tower.encoder.layers.X.temporal_attn.v_proj.weight
model.video_tower.video_tower.encoder.layers.X.temporal_attn.q_proj.weight
model.video_tower.video_tower.encoder.layers.X.temporal_attn.out_proj.weight
model.video_tower.video_tower.encoder.layers.X.temporal_layer_norm.weight
model.video_tower.video_tower.encoder.layers.X.temporal_layer_norm.bias
```

The `-hf` version uses `CLIPVisionModel` for both towers — no temporal attention exists.

**2. Weight comparison: video_tower ≈ image_tower**

```python
import torch

video_params = dict(model.video_tower.named_parameters())
image_params = dict(model.image_tower.named_parameters())

same, diff = [], []
for name in video_params:
    if name in image_params:
        if torch.equal(video_params[name], image_params[name]):
            same.append(name)
        else:
            diff.append(name)

print(f"Same: {len(same)}, Different: {len(diff)}")
```

**Result:**
- **Different (only 3):**
  - `vision_model.embeddings.class_embedding`
  - `vision_model.post_layernorm.weight`
  - `vision_model.post_layernorm.bias`
- **Same: all remaining ~300 parameters**

This means the video tower and image tower are effectively the **same model**. In the original `Video-LLaVA-7B`, these towers should have substantially different weights because they were pretrained separately (LanguageBind-Video on video-text pairs, LanguageBind-Image on image-text pairs).

### Impact

- The `-hf` model is **not a faithful conversion** of the original Video-LLaVA.
- Users loading the model via `VideoLlavaForConditionalGeneration.from_pretrained("LanguageBind/Video-LLaVA-7B-hf")` are running inference with what is essentially **two copies of the same image encoder** with no temporal modeling.
- Benchmark results from the `-hf` version do not reflect the actual Video-LLaVA architecture or performance described in the paper.
- Follow-up research using this model as a baseline is comparing against a degraded, incorrectly converted model.

### cc @zucchini-nlp

Could you clarify how `Video-LLaVA-7B-hf` was converted? Specifically:
- How were the video tower weights handled during conversion, given that the original model contains `temporal_attn` layers that don't exist in `CLIPVisionModel`?
- Were the LanguageBind-Video weights intentionally mapped to a standard CLIP architecture, or were the image tower weights duplicated into the video tower?
- Were the temporal attention weights simply discarded, or was there a merging/distillation step?

The weight comparison suggests the video tower may have received the same (or nearly the same) weights as the image tower, which would mean the conversion lost both the architectural differences and the distinct pretrained representations.

### Environment

- transformers version: 4.46
- Model: `LanguageBind/Video-LLaVA-7B-hf`

---
---
@zucchini-nlp

### Who can help?

@zucchini-nlp

Could you clarify how `Video-LLaVA-7B-hf` was converted? Specifically:
- How were the video tower weights handled during conversion, given that the original model contains `temporal_attn` layers that don't exist in `CLIPVisionModel`?
- Were the LanguageBind-Video weights intentionally mapped to a standard CLIP architecture, or were the image tower weights duplicated into the video tower?
- Were the temporal attention weights simply discarded, or was there a merging/distillation step?

The weight comparison suggests the video tower may have received the same (or nearly the same) weights as the image tower, which would mean the conversion lost both the architectural differences and the distinct pretrained representations.

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

### Reproduction

```python
import torch
from transformers import VideoLlavaForConditionalGeneration

model = VideoLlavaForConditionalGeneration.from_pretrained(
    "LanguageBind/Video-LLaVA-7B-hf",
    torch_dtype=torch.float16
)

# Architecture check: both are plain CLIPVisionModel
print(model.video_tower)
print(model.image_tower)

# Weight check
video_params = dict(model.video_tower.named_parameters())
image_params = dict(model.image_tower.named_parameters())

diff = [name for name in video_params
        if name in image_params and not torch.equal(video_params[name], image_params[name])]

print(f"Only {len(diff)} parameters differ:")
for n in diff:
    print(f"  {n}")
# Expected: most weights should differ
# Actual: only 3 differ
```

### Expected behavior

### Expected Behavior

The `-hf` conversion should:
1. Include temporal attention layers in the video tower matching the original architecture
2. Load the correct LanguageBind-Video pretrained weights (distinct from LanguageBind-Image)

Or at minimum, the model card should clearly state that this is **not equivalent** to the original model.

--- Comment by zucchini-nlp at 2026-02-09T09:25:52Z ---
Hey @jong980812 ,

This is [the script](https://github.com/huggingface/transformers/blob/main/src/transformers/models/video_llava/convert_video_llava_weights_to_hf.py) used to convert the model. It was converted a long time ago so I cannot remember clearly the details, tho I think we did an equivalence test to see if the generated output will be same in HF impl and the official release

I am looking at the VideoLlava code rn their repo and indeed seems like the video tower needs a `temporal_attn` layer. Do you have a reproducer showing that the outputs from HF model and the official model are different?

Also note that we don't own the model repo on the hub and thus we can't update the weights or configs. From what I now, video-llava authors are not active and the model is not super common in usage. If the conversion is indeed corrupted, we can only fix the above script and inform users that the `LanguageBind/Video-LLaVA-7B-hf` isn't correct

--- Comment by jong980812 at 2026-02-09T10:10:23Z ---
Thanks for the quick response and for sharing the conversion script!

I understand the constraints on your side regarding the model repo ownership. However, I'd like to point out that this isn't just an output discrepancy issue — **the model architecture itself is structurally different**.

Here are two concrete pieces of evidence:

## 1. Architecture Comparison

The HF version (`Video-LLaVA-7B-hf`) uses a plain `CLIPVisionModel` as the video tower, which is identical to the image tower — it lacks `temporal_attn` and `temporal_layer_norm1` layers entirely:

```
(video_tower): CLIPVisionModel(
    (vision_model): CLIPVisionTransformer(
      (embeddings): CLIPVisionEmbeddings(
        (patch_embedding): Conv2d(3, 1024, kernel_size=(14, 14), stride=(14, 14), bias=False)
        (position_embedding): Embedding(257, 1024)
      )
      (pre_layrnorm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
      (encoder): CLIPEncoder(
        (layers): ModuleList(
          (0-23): 24 x CLIPEncoderLayer(
            (self_attn): CLIPSdpaAttention(
              (k_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (v_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (q_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (out_proj): Linear(in_features=1024, out_features=1024, bias=True)
            )
            (layer_norm1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
            (mlp): CLIPMLP(
              (activation_fn): QuickGELUActivation()
              (fc1): Linear(in_features=1024, out_features=4096, bias=True)
              (fc2): Linear(in_features=4096, out_features=1024, bias=True)
            )
            (layer_norm2): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
          )
        )
      )
      (post_layernorm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
    )
  )
```

In contrast, the original `Video-LLaVA-7B` uses `LanguageBindVideoTower`, which includes `temporal_attn` (with its own K/Q/V/Out projections) and `temporal_layer_norm1` in every encoder layer:

```
(video_tower): LanguageBindVideoTower(
   (video_tower): CLIPVisionTransformer(
     (embeddings): CLIPVisionEmbeddings(
       (patch_embedding): Conv2d(3, 1024, kernel_size=(14, 14), stride=(14, 14), bias=False)
       (position_embedding): Embedding(257, 1024)
     )
     (patch_dropout): PatchDropout()
     (pre_layrnorm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
     (encoder): CLIPEncoder(
       (layers): ModuleList(
         (0-23): 24 x CLIPEncoderLayer(
           (self_attn): CLIPAttention(
             (k_proj): Linear(in_features=1024, out_features=1024, bias=True)
             (v_proj): Linear(in_features=1024, out_features=1024, bias=True)
             (q_proj): Linear(in_features=1024, out_features=1024, bias=True)
             (out_proj): Linear(in_features=1024, out_features=1024, bias=True)
           )
           (layer_norm1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
           (mlp): CLIPMLP(
             (activation_fn): GELUActivation()
             (fc1): Linear(in_features=1024, out_features=4096, bias=True)
             (fc2): Linear(in_features=4096, out_features=1024, bias=True)
           )
           (layer_norm2): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
           (temporal_attn): CLIPAttention(
             (k_proj): Linear(in_features=1024, out_features=1024, bias=True)
             (v_proj): Linear(in_features=1024, out_features=1024, bias=True)
             (q_proj): Linear(in_features=1024, out_features=1024, bias=True)
             (out_proj): Linear(in_features=1024, out_features=1024, bias=True)
           )
           (temporal_layer_norm1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
         )
       )
     )
   )
 )
```

This means the HF version is missing a significant portion of the video-specific parameters.

## 2. Feature-level Comparison

I extracted features from **400 videos** using both models under identical settings. The results confirm a substantial divergence:

```
--- Basic Stats ---
F1 mean: 0.002946, std: 0.187537, min: -1.630859, max: 3.878906
F2 mean: -0.002434, std: 0.323142, min: -2.918253, max: 6.173318

--- Element-wise Diff ---
Max diff: 2.431636
Mean diff: 0.176825

--- Cosine Similarity (per sample) ---
Mean: 0.721119, Min: 0.699564, Max: 0.736440
```

- **Mean cosine similarity: ~0.72** (far from 1.0)
- **Max element-wise diff: 2.43**, Mean diff: 0.18

These are not minor numerical differences — they indicate fundamentally different representations.

---

Given this, even if the equivalence test passed at the time of conversion, the current HF weights appear to be missing the temporal attention parameters that are core to LanguageBind-Video's design. I believe this should be clearly communicated to users, whether through a note on the model card, an update to the conversion script, or both.

I'd appreciate any clarification on how this can be addressed. Thank you!

--- Comment by zucchini-nlp at 2026-02-09T10:50:24Z ---
I see! Do you know if we can adapt  the existing model code without breaking it for the "corrupted" checkpoint? If the diff is only in the vision tower and the core logic of merging vision+text embeddings stays the same, let's update the model code and conversion script accordingly

And add a note in model doc page explaining that to get the correct output matching official model ckpt, one needs to convert the model themselves. Since we don't own the hf repo, that's all we can do to inform users

--- Comment by github-actions[bot] at 2026-03-16T08:18:31Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #43813: Typo - should be "orig_conversion.quantization_operation"
State: closed  |  Labels:
Author: ishaan-shivhare  |  Created: 2026-02-07T02:29:14Z

https://github.com/huggingface/transformers/blob/dd360ad2364382e7ef3c19d1011cb0a4e9b418ff/src/transformers/integrations/peft.py#L303

Please have a look. There's another similar typo in this file on line 264

--- Comment by lohitkotni at 2026-02-07T02:59:22Z ---
Hey I'm new here. Can I work on this issue?

--- Comment by redpanda1995 at 2026-02-07T19:05:13Z ---
sent a PR for this - https://github.com/huggingface/transformers/pull/43821

--- Comment by github-actions[bot] at 2026-03-09T08:09:01Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #43805: chore(test): add a `set_seed` pytest fixture
State: closed  |  Labels:
Author: tarekziade  |  Created: 2026-02-06T16:14:29Z

follow-up from https://github.com/huggingface/transformers/pull/43794

we should add in our fixture a `set_seed` so we *always* set the same seed in all of the model tests to improve determinism.

cc @Rocketknight1

--- Comment by amahuli03 at 2026-02-11T18:12:26Z ---
Hi, does this issue still need to be fixed? I'm happy to give it a try

--- Comment by tarekziade at 2026-02-11T20:04:50Z ---
Thanks @amahuli03 I have a draft going on here - https://github.com/huggingface/transformers/pull/43817  I am off a couple of weeks but will get back to it then.

--- Comment by tarekziade at 2026-03-03T15:27:37Z ---
We'll defer doing this for now


openai/whisper-large-v2 can't run

ISSUE #43792: openai/whisper-large-v2 can't run
State: closed  |  Labels: bug
Author: LIANGQI0811  |  Created: 2026-02-06T09:16:29Z

### System Info

```python
tt = pipeline(model="openai/whisper-large-v2")
tt("https://hf-mirror.com/datasets/Narsil/asr_dummy/resolve/main/mlk.flac")
```

print error message like this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/developer/.local/lib/python3.10/site-packages/transformers/pipelines/automatic_speech_recognition.py", line 266, in __call__
    return super().__call__(inputs, **kwargs)
  File "/home/developer/.local/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1266, in __call__
    return next(
  File "/home/developer/.local/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 126, in __next__
    item = next(self.iterator)
  File "/home/developer/.local/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 271, in __next__
    processed = self.infer(next(self.iterator), **self.params)
  File "/home/developer/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 741, in __next__
    data = self._next_data()
  File "/home/developer/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 801, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/developer/.local/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 35, in fetch
    data.append(next(self.dataset_iter))
  File "/home/developer/.local/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 188, in __next__
    processed = next(self.subiterator)
  File "/home/developer/.local/lib/python3.10/site-packages/transformers/pipelines/automatic_speech_recognition.py", line 486, in preprocess
    extra["num_frames"] = processed.pop("num_frames")
  File "/usr/lib/python3.10/_collections_abc.py", line 962, in pop
    value = self[key]
  File "/home/developer/.local/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 90, in __getitem__
    return self.data[item]
KeyError: 'num_frames'

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

tt = pipeline(model="openai/whisper-large-v2")
tt("https://hf-mirror.com/datasets/Narsil/asr_dummy/resolve/main/mlk.flac")

### Expected behavior

will run and result data

--- Comment by gSulpizio at 2026-02-06T10:33:09Z ---
`KeyError: 'num_frames'` an issue that occurred in older versions. The pipeline expects the feature extractor to return a `num_frames ` key, but it is not being generated or passed correctly in your version. `pip install --upgrade transformers accelerator librosa` should fix it.

--- Comment by LIANGQI0811 at 2026-02-09T02:11:00Z ---
it't looks work now


ISSUE #43784: NameError: name 'nn' is not defined when importing sentence-transformers with latest transformers
State: closed  |  Labels:
Author: Alan-Jowett  |  Created: 2026-02-06T00:18:08Z

## System Info

```
transformers version: latest (installed via pip today, 2026-02-05)
sentence-transformers version: latest
torch version: 2.x (from pytorch/pytorch Docker image)
Python version: 3.11+
OS: Linux (Docker container)
```

## Who can help?

@ArthurZucker @Rocketknight1

## Information

- [ ] The official example scripts
- [x] My own modified scripts

## Tasks

- [ ] An officially supported task in the `examples` folder
- [x] My own task or dataset

## Reproduction

Minimal reproduction:

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
```

Or directly:

```python
from transformers import AutoModel
```

Results in:

```
NameError: name 'nn' is not defined
```

The error occurs during the import chain when `transformers` attempts to load a module that references `nn.Module` or similar `torch.nn` constructs without first importing `torch.nn as nn`.

## Expected behavior

The import should succeed without errors. This worked with previous versions of `transformers`.

## Additional context

This appears to be a similar regression to #38269 ("NameError: name 'Replicate' is not defined"), which was caused by a missing import in `transformers/integrations/tensor_parallel.py`.

The current error (`nn` not defined) suggests another module has a similar missing import issue, likely referencing `nn.Module` without the corresponding `import torch.nn as nn` statement.

**Workaround**: Pinning `transformers<4.53.0` (or to the last known working version) resolves the issue.

--- Comment by Alan-Jowett at 2026-02-06T00:21:02Z ---
**Update**: This appears to be related to the **transformers v5.0.0** release (January 26, 2026).

The last working CI build was on February 4, 2026, which likely used a cached Docker layer with transformers 4.x. Fresh builds now pull v5.0.0, which causes this import error.

The error occurs when `sentence-transformers` (or other downstream packages) imports from `transformers`, and a module in v5.0.0 references `nn.Module` without the proper `import torch.nn as nn` statement.

**Temporary workaround**: Pin `transformers<5.0.0` until this is resolved.

--- Comment by ArthurZucker at 2026-02-06T08:43:20Z ---
cc @tomaarsen as well!

--- Comment by tomaarsen at 2026-02-06T09:27:09Z ---
Hello @Alan-Jowett!

Thanks for raising this. I'm sadly unable to reproduce this right now. I created a fresh virtualenvironment with (only) the latest `sentence-transformers` and `transformers`, and the code you linked seems to work for me:
```python
>>> import transformers
>>> from transformers import AutoModel
>>> from sentence_transformers import SentenceTransformer
>>>
>>> model = SentenceTransformer("all-MiniLM-L6-v2")
Loading weights: 100%|████████████████████| 103/103 [00:00<00:00, 3319.53it/s, Materializing param=pooler.dense.weight]
BertModel LOAD REPORT from: sentence-transformers/all-MiniLM-L6-v2
Key                     | Status     |  |
------------------------+------------+--+-
embeddings.position_ids | UNEXPECTED |  |

Notes:
- UNEXPECTED    :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
```

Could you perhaps find which line in `transformers` uses an un-imported `nn`? Perhaps the import is locked behind an `if is_..._available(): from torch import nn`, causing your issue.

- Tom Aarsen

--- Comment by Alan-Jowett at 2026-02-06T15:34:19Z ---
Thank you for taking a look into this issue. I will try stripping down my container to produce a minimal reproduction of the issue.

Alan Jowett

--- Comment by Alan-Jowett at 2026-02-06T15:56:10Z ---
Seems like it's triggered by an old version of PyTorch?

```dockerfile
# Minimal reproduction for transformers v5.0.0 NameError: name 'nn' is not defined
# Issue: https://github.com/huggingface/transformers/issues/43784
#
# The bug occurs when:
# 1. PyTorch version is < 2.4 (e.g., 2.2.1 from pytorch/pytorch base image)
# 2. transformers >= 5.0.0 is installed
# 3. transformers detects old PyTorch and "disables" it
# 4. transformers/integrations/accelerate.py line 62 uses `nn.Module` without importing it
#
# Build: docker build -t transformers-nn-bug .
# Run:   docker run --rm transformers-nn-bug
#
# Expected output:
#   Disabling PyTorch because PyTorch >= 2.4 is required but found 2.2.1
#   ...
#   File ".../transformers/integrations/accelerate.py", line 62, in <module>
#     ) -> tuple[int, list[str], list[nn.Module]]:
#   NameError: name 'nn' is not defined

FROM pytorch/pytorch@sha256:11691e035a3651d25a87116b4f6adc113a27a29d8f5a6a583f8569e0ee5ff897

# This base image has PyTorch 2.2.1, which is < 2.4 required by transformers 5.0.0

# Install latest transformers (v5.0.0+) - the bug is in transformers, not sentence-transformers
RUN pip install --no-cache-dir "transformers>=5.0.0" sentence-transformers

# Minimal test - just importing sentence_transformers triggers the bug
# because it imports from transformers.integrations.peft which chains to accelerate.py
CMD ["python", "-c", "from sentence_transformers import SentenceTransformer; print('Success!')"]
```

```
Disabling PyTorch because PyTorch >= 2.4 is required but found 2.2.1
PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Python: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]
PyTorch: 2.2.1
Transformers: 5.1.0
Traceback (most recent call last):
  File "/test.py", line 7, in <module>
    from sentence_transformers import SentenceTransformer
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/__init__.py", line 15, in <module>
    from sentence_transformers.cross_encoder import (
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/cross_encoder/__init__.py", line 3, in <module>
    from .CrossEncoder import CrossEncoder
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/cross_encoder/CrossEncoder.py", line 35, in <module>
    from sentence_transformers.cross_encoder.fit_mixin import FitMixin
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/cross_encoder/fit_mixin.py", line 20, in <module>
    from sentence_transformers.datasets.NoDuplicatesDataLoader import NoDuplicatesDataLoader
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/datasets/__init__.py", line 13, in <module>
    from .ParallelSentencesDataset import ParallelSentencesDataset
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/datasets/ParallelSentencesDataset.py", line 19, in <module>
    from sentence_transformers import SentenceTransformer
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 45, in <module>
    from .peft_mixin import PeftAdapterMixin
  File "/opt/conda/lib/python3.10/site-packages/sentence_transformers/peft_mixin.py", line 5, in <module>
    from transformers.integrations.peft import PeftAdapterMixin as PeftAdapterMixinTransformers
  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/peft.py", line 22, in <module>
    from ..conversion_mapping import (
  File "/opt/conda/lib/python3.10/site-packages/transformers/conversion_mapping.py", line 20, in <module>
    from .core_model_loading import (
  File "/opt/conda/lib/python3.10/site-packages/transformers/core_model_loading.py", line 34, in <module>
    from .integrations.accelerate import get_device, offload_weight
  File "/opt/conda/lib/python3.10/site-packages/transformers/integrations/accelerate.py", line 62, in <module>
    ) -> tuple[int, list[str], list[nn.Module]]:
NameError: name 'nn' is not defined
```

--- Comment by vasqu at 2026-02-09T11:39:30Z ---
We have bumped the minimum to torch 2.4 so please upgrade the torch version. It's not something we will maintain going forward

--- Comment by github-actions[bot] at 2026-03-08T08:02:58Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #43782: Qwen3VLForConditionalGeneration.from_pretrained weight_only = True error
State: closed  |  Labels: bug
Author: oscars17  |  Created: 2026-02-05T21:52:13Z

### System Info

platform: ubuntu 24.04.03
python 3.10
transformers 5.0
torch 2.10
accelerate 1.12
docker image: FROM nvidia/cuda:12.2.2-cudnn8-runtime-ubuntu22.04
https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct

### Who can help?

@yonigozlan @molbap

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`
 _pickle.UnpicklingError: Weights only load failed. In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
web-1    | Please file an issue with the following so that we can make `weights_only=True` compatible with your use case: WeightsUnpickler error:
`

I get the error when i try to run the code below

import logging
import os
import torch
from typing import Optional, Dict
from PIL import Image
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from app.core.config import settings

logger = logging.getLogger(__name__)

class LocalQwenVisionService:
    _instance = None

    def __new__(cls, model_path: Optional[str] = None):
        if cls._instance is None:
            cls._instance = super().__new__(cls)
            cls._instance._initialized = False
        return cls._instance

    def __init__(self, model_path: Optional[str] = None):
        if self._initialized:
            return

        self.model_path = model_path or settings.QWEN_MODEL_PATH
        if not self.model_path or not os.path.exists(self.model_path):
            raise ValueError(f"Model path invalid: {self.model_path}")

        logger.info(f"🚀 Loading Qwen3-VL from: {self.model_path}")

        self.model = Qwen3VLForConditionalGeneration.from_pretrained(
            self.model_path,
            dtype=torch.float16,
            attn_implementation="sdpa",
            local_files_only=True,
            weights_only=False
        )

        self.processor = AutoProcessor.from_pretrained(
            self.model_path,
            local_files_only=True
        )

        logger.info(f"✅ Model loaded on: {next(self.model.parameters()).device}")
        self._initialized = True

    def analyze_image(
            self,
            image_path: str,
            prompt: Optional[str] = None
    ) -> Dict:
        """✅ Analyze image and return response"""

        if not os.path.exists(image_path):
            return {"status": "error", "error": f"Image not found: {image_path}"}

        if not prompt:
            prompt = (
                "Analyze this image and provide:\n"
                "1. Description of content\n"
                "2. Main objects and locations\n"
                "3. Atmosphere and mood\n"
                "4. Tags/categories\n"
                "5. One-sentence summary"
            )

        try:
            image = Image.open(image_path).convert("RGB")

            messages = [
                {
                    "role": "user",
                    "content": [
                        {"type": "image", "url": image},
                        {"type": "text", "text": prompt}
                    ]
                }
            ]

            inputs = self.processor(
                messages,
                tokenize=True,
                add_generation_prompt=True,
                return_dict=True,
                return_tensors='pt'
            )

            generated_ids = self.model.generate(
                **inputs,
                max_new_tokens=1024
            )

            generated_ids_trimmed = [
                out_ids[len(in_ids):]
                for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
            ]

            response_text = self.processor.batch_decode(
                generated_ids_trimmed,
                skip_special_tokens=True,
                clean_up_tokenization_spaces=False
            )[0]

            return {
                "status": "success",
                "message": response_text
            }

        except Exception as e:
            logger.error(f"❌ Error: {e}")
            return {"status": "error", "error": str(e)}

    def close(self):
        """Clean up resources"""
        logger.info("🛑 Closing model")
        if hasattr(self, "model"):
            del self.model
        torch.cuda.empty_cache()

What i tried
ENV TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 - to docker file didnt help
weights_only=False - inside Qwen3VLForConditionalGeneration.from_pretrained - didnt help

import numpy
import torch.serialization

torch.serialization.add_safe_globals([
    (numpy._core.multiarray.scalar, 'numpy.core.multiarray.scalar'),
    numpy.dtype,
    numpy.dtypes.Float64DType
])
 - didnt help

### Expected behavior

I have no clue where should i dig to fix the problem.

--- Comment by pragnyanramtha at 2026-02-21T05:59:42Z ---
The issue is in src/transformers/modeling_utils.py line 4211 (on main). When loading .bin checkpoint files in the non-DeepSpeed code path, load_state_dict(ckpt_file) is called without forwarding the weights_only parameter, so it always defaults to True, regardless of what you pass to from_pretrained().

The DeepSpeed path (line 4184) already correctly passes load_config.weights_only, but the regular path doesn't:

# DeepSpeed path (correct) - line 4184
load_state_dict(ckpt_file, map_location="cpu", weights_only=load_config.weights_only)

# Regular .bin path (bug) - line 4211
load_state_dict(ckpt_file)  # weights_only defaults to True, ignoring user's setting
The fix is a one-liner — pass weights_only=load_config.weights_only to the regular .bin loading path to match the DeepSpeed path.

Workaround until this is merged: use use_safetensors=True in your from_pretrained() call if the model provides .safetensors files (Qwen3-VL-8B-Instruct does). Safetensors loading doesn't use torch.load/pickle at all, so the weights_only issue is bypassed entirely.

--- Comment by Rocketknight1 at 2026-02-23T14:21:07Z ---
This is a very strange bug because it's a torch weight loading error, but that model stores its weights as `safetensors`. Are you sure you haven't downloaded a corrupted checkpoint somewhere?

--- Comment by github-actions[bot] at 2026-03-20T08:08:34Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #43761: [v5 regression] CLIPVisionModel.forward returns hidden_states=None even when output_hidden_states=True
State: closed  |  Labels: bug
Author: yukiu00  |  Created: 2026-02-05T09:35:32Z

### System Info

### Description

I am reporting a potential regression found while testing `transformers` **v5.0.0**.
We noticed that `CLIPVisionModel.forward()` returns `hidden_states=None` even when `output_hidden_states=True` is explicitly passed. This behavior is different from v4.x, where hidden states were correctly returned.

### Reproduction / Context

We encountered this issue during Llava convergence tests in the **Liger Kernel** repository.
Specifically, the issue was identified in:
- **Liger-Kernel PR:** https://github.com/linkedin/Liger-Kernel/pull/1061
- **Related Issue:** https://github.com/linkedin/Liger-Kernel/issues/1011

In our test suite, we noticed the failure when creating a new model instance after reloading modules. The `CLIPVisionModel` fails to return hidden states.

Upon investigating the code changes in `transformers`, it appears that `output_hidden_states` is no longer handled correctly in `ClipEncoder.forward`.
- **v5.0.0 (Current):** [modeling_clip.py#L485](https://github.com/huggingface/transformers/blob/08810b1e278938278c50153ee1edfd7a20a759da/src/transformers/models/clip/modeling_clip.py#L485)
- **v4.57.6 (Previous):** [modeling_clip.py#L506](https://github.com/huggingface/transformers/blob/753d61104116eefc8ffc977327b441ee0c8d599f/src/transformers/models/clip/modeling_clip.py#L506)

It seems `output_hidden_states` is present in `TransformersKwargs` fields but might not be utilized in the `.forward()` method after the recent modernizing changes.

### Suspected Cause
This might be related to the changes introduced in PR #41546.

Could you please confirm if this is an intended change? If so, what is the recommended way to retrieve hidden states in v5?

Thanks!

### Who can help?

@yonigozlan @molbap @zucchini-nlp

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
import torch
from transformers import CLIPVisionModel, CLIPVisionConfig

# 1. Setup a minimal CLIPVisionModel
config = CLIPVisionConfig(
    hidden_size=32,
    intermediate_size=64,
    num_hidden_layers=2,
    num_attention_heads=4,
    image_size=30,
    patch_size=10
)
model = CLIPVisionModel(config)

# 2. Create dummy input
pixel_values = torch.randn(1, 3, 30, 30)

# 3. Run forward pass with output_hidden_states=True
outputs = model(pixel_values, output_hidden_states=True)

# 4. Check the result
print(f"Transformers version: {importlib.metadata.version('transformers')}")
print(f"Hidden states: {outputs.hidden_states}")

if outputs.hidden_states is None:
    raise ValueError("Regression: hidden_states is None despite output_hidden_states=True")
```

### Expected behavior

When `output_hidden_states=True` is passed to `model()`, `outputs.hidden_states` should be a tuple of tensors containing the hidden states from all layers. It should not be `None`.

--- Comment by molbap at 2026-02-05T10:05:44Z ---
Hello @yukiu00 , `output_hidden_states` is indeed removed but not the tracking itself of hidden_states. it is handled through `_can_record_outputs`

```python
    _can_record_outputs = {
        "hidden_states": CLIPEncoderLayer,
        "attentions": CLIPAttention,
    }
```

so it is tracked. What happens is a different issue and is linked to when a model is _reloaded_, I had to dig a bit. Exact reproducer is:

```python
import importlib, torch
from transformers import CLIPVisionConfig, CLIPVisionModel
import transformers.models.clip.modeling_clip as modeling_clip

config = CLIPVisionConfig(hidden_size=32, intermediate_size=64, num_hidden_layers=2, num_attention_heads=4, image_size=30, patch_size=10)
pixel_values = torch.randn(1, 3, 30, 30)

m1 = CLIPVisionModel(config)
o1 = m1(pixel_values=pixel_values, output_hidden_states=True)
print("before reload:", o1.hidden_states is None)

importlib.reload(modeling_clip)

m2 = CLIPVisionModel(config)
o2 = m2(pixel_values=pixel_values, output_hidden_states=True)
print("after reload:", o2.hidden_states is None)
```

The reason why that happens is that `isinstance(module, specs.target_class) ` fails. Opening a PR to fix it

--- Comment by yukiu00 at 2026-02-05T10:18:03Z ---
Thanks for the quick investigation @molbap!
To clarify my understanding: the output_hidden_states parameter itself was intentionally removed in v5, and hidden states tracking now works through the new _can_record_outputs mechanism. The actual bug is that isinstance() checks fail after module reload, which breaks this tracking.
Is that correct?
Looking forward to the fix PR. Let me know if there's anything I can help with for testing.

--- Comment by molbap at 2026-02-05T12:07:50Z ---
Yes, your understanding is correct. For testing, yes thanks, you may checkout the linked PR and let me know if it solves your issue? I mean just  `git checkout fix_recording_on_reload` and `pip install -e .` to have the local install, if you want to test

--- Comment by yukiu00 at 2026-02-05T12:43:49Z ---
@molbap

Thanks for the quick fix! I tested the `fix_recording_on_reload` branch and confirmed it resolves the issue:

- Minimal reproduction code: ✓ `hidden_states` correctly returned after reload
- Liger-Kernel Llava convergence test: ✓ Passed

--- Comment by oil666oil at 2026-02-05T17:29:01Z ---
This solves a huge pain point for me. Shared my automated deployment scripts for this on my GitHub.

--- Comment by zucchini-nlp at 2026-02-06T17:07:10Z ---
Ig this is fixed now in `main` branch


ISSUE #43756: Smollm3 drops 3/4 RoPE layers while 1/4 is intended by whats claimed in the blog post.
State: closed  |  Labels: bug
Author: tobiaskatsch  |  Created: 2026-02-05T06:01:24Z

### System Info

- `transformers` version: v5.0.0
- Python version: N/A (bug in library code)
- PyTorch version: N/A (bug in library code)

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

**Relevant files:**
- Modeling: [`src/transformers/models/smollm3/modeling_smollm3.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/smollm3/modeling_smollm3.py)

**Blog statement** ([SmolLM3 blog post](https://huggingface.co/blog/smollm3)):
> "removes positional encoding every 4th layer"

This means 25% of layers should be NoPE (1 out of every 4 layers has no RoPE).

**Original training config** ([smollm3-configs/stage1_8T.yaml](https://huggingface.co/datasets/HuggingFaceTB/smollm3-configs/blob/main/stage1_8T.yaml)):

```yaml
no_rope_layer: 4
```

This integer means "skip RoPE every 4th layer". During conversion to HuggingFace format, this becomes a list where `1` marks "this is a no-rope layer":

**The HF config** ([`configuration_smollm3.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/smollm3/configuration_smollm3.py)):

```python
no_rope_layers = [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, ...]
#                 0  1  2  3  4  5  6  7  8  9  10 11
```

The naming is semantically correct: `1` at index 3, 7, 11... marks layers that should have **no RoPE** ✓

**The bug** ([`modeling_smollm3.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/smollm3/modeling_smollm3.py)):

```python
self.use_rope = config.no_rope_layers[layer_idx]
```

This interprets the value directly, but `no_rope_layers[i] = 1` means "layer i has NO rope", so `use_rope` should be the **negation**.

With this logic:
- Layer 3: `no_rope_layers[3] = 1` → `use_rope = True` → RoPE applied ❌
- Layer 0: `no_rope_layers[0] = 0` → `use_rope = False` → No RoPE ❌

**Result:** 75% of layers have no positional encoding (layers 0,1,2, 4,5,6, 8,9,10...) and only 25% have RoPE.

| Source | Description | NoPE % |
|--------|-------------|--------|
| Blog | "removes positional encoding every 4th layer" | 25% NoPE |
| Config | `no_rope_layers = [0,0,0,1,...]` | 25% intended NoPE |
| Code | `use_rope = no_rope_layers[i]` (inverted!) | **75% actual NoPE** |

### Expected behavior

The code should apply RoPE to 75% of layers and skip RoPE for 25% of layers (every 4th layer), matching the blog description.

**Suggested fix:**

```python
self.use_rope = not config.no_rope_layers[layer_idx]
```

Or alternatively, rename the config field to clarify the semantics (e.g., `rope_layers` instead of `no_rope_layers`).

--- Comment by eliebak at 2026-02-05T14:06:22Z ---
hi, the model was trained with 25% NoPE. in [configuration_smollm3.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/smollm3/configuration_smollm3.py) we are describing how `no_rope_layers` works:
```
        no_rope_layers (`List[int]`, *optional*):
            List with at least the same length as the number of layers in the model.
            A `1` at an index position indicates that the corresponding layer will use RoPE,
            while a `0` indicates that it's a NoPE layer.
```
so `self.use_rope = config.no_rope_layers[layer_idx]` is the correct way to implement this.

I agree that the naming of `no_rope_layers`is ambiguous but i'd prefer to leave it as is to not break the `config.json` of all the smollm3 variant.

--- Comment by tobiaskatsch at 2026-02-05T18:49:18Z ---
I recommend you to update the blog post then, because "removes positional encoding every 4th layer" is clearly misleading.
And could you please clearify whether this was intended or a bug in the config? Did you intentionally train the model like that or was this a accident? Because the choice is hard for me to understand given the ablation eval data you have shared. Thanks!

--- Comment by tobiaskatsch at 2026-02-05T18:55:42Z ---
But appraciate ovearll that you put together the smollm3 recipe, really helpful stuff in there! Just came across this subtle difference btween blog and code i wanted to point out here. Best regards!

--- Comment by eliebak at 2026-02-05T19:35:38Z ---
Thanks for the kind words on Smollm! So there is no bug, 25% NoPE is "removing PE every 4 layers". If you check the 'config.json' of smollm3 you will see 'no_rope_layers = [ 1,1,1,0,...]'

--- Comment by tobiaskatsch at 2026-02-05T19:46:15Z ---
ok thx you are right!