PULL REQUEST #45595: Add unified Cache-layer management for GLM-5 DSA Indexer keys
State: closed  |  Merged: False
Author: louzongzhi  |  Base: main
Labels:
Created: 2026-04-23T06:35:39Z

## What does this PR do?

This PR migrates the GLM-5 DSA **IndexCache** key cache from a self-managed `register_buffer` (as introduced in #45424) into the standard `past_key_values` (`Cache`) per-layer infrastructure. Indexer keys now share the same lifecycle as attention KV caches, enabling transparent support for beam-search reordering, cache cropping, batch selection/repetition, and offloading.

## Background & Motivation

GLM-5 integrates the DeepSeek Sparse Attention (DSA) Indexer. In #45424, we added support for the **IndexCache** mechanism (THUDM/IndexCache, arXiv:2603.12201) to accelerate inference by caching indexer keys and allowing Shared (S) layers to reuse indices from Full (F) layers.

However, the implementation in #45424 managed these cached keys inside `GlmMoeDsaIndexer` via `self.register_buffer("_cached_keys", None, persistent=False)`. While this works for basic generation, it creates architectural friction when building **downstream models on top of GLM-5**:

- **Lifecycle drift**: `Cache.reset`, `reorder_cache`, `crop`, `batch_repeat_interleave`, and `batch_select_indices` do not propagate to the Indexer's isolated buffer.
- **Offloading blind spot**: The indexer keys are invisible to the Cache offloading / prefetching system.
- **Duplicated logic**: Prefill-vs-decode concatenation is manually implemented inside the Indexer, duplicating what `DynamicCache` already does.

To simplify our own downstream model code and benefit the broader GLM-5 ecosystem, we are upstreaming this unification first.

## Changes

### `src/transformers/cache_utils.py`

- **`CacheLayerMixin`**: Added `indexer_keys` attribute and abstract method `update_cached_keys()`.
- **`DynamicLayer`**: Implemented `update_cached_keys()` by concatenating along the sequence dimension (`dim=1`); synchronized `crop`, `batch_repeat_interleave`, `batch_select_indices`, `reset`, and `reorder_cache` to also operate on `indexer_keys`.
- **`StaticLayer`**: Added passthrough `update_cached_keys()` returning the input as-is.
- **`Cache`**: Added `update_cached_keys(cached_keys, layer_idx)` and `reset_cached_keys(layer_idx)` to dispatch per layer, mirroring the existing `update()` / `reset()` API.

### `src/transformers/models/glm_moe_dsa/modeling_glm_moe_dsa.py`

- **`GlmMoeDsaIndexer`**:
  - Removed `self.register_buffer("_cached_keys", None, persistent=False)`.
  - `forward()` now accepts `past_key_values: Cache | None` instead of `use_cache: bool`.
  - Prefill (`seq_len > 1`) triggers `past_key_values.reset_cached_keys(layer_idx)` before computing scores.
  - Decode (`seq_len == 1`) appends keys via `past_key_values.update_cached_keys(k, layer_idx)`.
- **`GlmMoeDsaAttention`**: Updated `self.indexer(...)` call to pass `past_key_values=past_key_values` directly.

## Behavior equivalence

| #45424 (self-managed IndexCache) | This PR (unified Cache layer) |
|---|---|
| `if seq_len > 1: self._cached_keys = None` | `if seq_len > 1: past_key_values.reset_cached_keys(layer_idx)` |
| `if use_cache: cat([self._cached_keys, k])` | `if past_key_values is not None: past_key_values.update_cached_keys(k, layer_idx)` |
| `else: k_cached = k` | `else: k_cached = k` |

## Local verification

Verified locally with a tiny `hidden_size=256` config:
- Prefill + decode cache growth
- Multi-layer isolation
- Beam-search `reorder_cache`
- `batch_select_indices` / `batch_repeat_interleave`
- `crop` truncation
- Shared indexer (`skip_topk`) reuse path
- No-cache fallback (`past_key_values=None`)
- `Cache.reset()` clears `indexer_keys`
- End-to-end multi-turn chat: `indexer_keys` accumulate correctly across turns

## Backward compatibility

- No public API changes; `generate()` behavior is unchanged.
- The removed `_cached_keys` buffer was never part of saved state (`persistent=False`), so existing checkpoints remain fully compatible.

## Follow-up context

We are actively building a new LLM architecture on top of the GLM-5 backbone, leveraging the DSA Indexer and IndexCache design. Unifying the Indexer cache into the standard `Cache` layer is prerequisite infrastructure work for open-sourcing that model. We will share more details once training converges. Stay tuned!

<!-- Remove if not applicable -->

Fixes # (N/A)

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
**we ask that new users do not submit pure code agent PRs** at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [x] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez

Library:

- attention: @vasqu @ArthurZucker @CyrilVallez
- generate: @gante

--- Comment by github-actions[bot] at 2026-04-23T06:36:49Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm_moe_dsa

--- Comment by Rocketknight1 at 2026-04-23T12:26:18Z ---
cc @arthurzucker @vasqu who reviewed the last issue! #45424

--- Comment by louzongzhi at 2026-04-23T14:03:31Z ---
> cc @ArthurZucker @vasqu who reviewed the last issue! #45424

Hi @Rocketknight1 @ArthurZucker @vasqu, thanks for the attention!

I'd like to clarify the background of IndexCache and the context of this PR.

**IndexCache is a mechanism explicitly described in the GLM-5 papers and official code.** It allows Shared (S) layers to reuse top-k indices from Full (F) layers to accelerate inference. You can find the details in [arXiv:2602.15763](https://arxiv.org/abs/2602.15763) and [arXiv:2603.12201](https://arxiv.org/abs/2603.12201), and the official implementation is at [THUDM/IndexCache](https://github.com/THUDM/IndexCache).

**The GLM-5 implementation in transformers did not include IndexCache until [#45424](https://github.com/huggingface/transformers/pull/45424).** Without it, Shared layers were unable to reuse indices, which deviated from the official behavior described in the papers. In [#45424](https://github.com/huggingface/transformers/pull/45424), I added IndexCache support following the official implementation, so the functional behavior now aligns with the official repo.

**A side note on `self._cached_keys`:** The change from a plain Python attribute (`self._cached_keys = None`) to `register_buffer(..., persistent=False)` in [#45424](https://github.com/huggingface/transformers/pull/45424) was an incidental fix, not part of the IndexCache feature itself. The original plain attribute had clear infrastructure issues: it doesn't sync with the model device (`.to(device)` / `.cuda()` won't move it), it can lead to inconsistent states across ranks in distributed training (DDP/FSDP), and it isn't tracked by `state_dict`. The `register_buffer` change simply fixed those issues.

**This PR (#45595) does not change the functional logic of IndexCache.** That logic remains the same as in the THUDM official implementation (i.e., the behavior before my [#45424](https://github.com/huggingface/transformers/pull/45424) submission). **What this PR does is migrate the cache management from the internal `register_buffer` in `GlmMoeDsaIndexer` into the standard `past_key_values` (`DynamicCache`/`StaticCache`) lifecycle.** This lets indexer keys automatically participate in `Cache.reset`, `reorder_cache`, `crop`, `batch_repeat_interleave`, and offloading, rather than requiring downstream models to handle these operations manually.

If anyone has questions about the IndexCache behavior or the migration approach, I'm happy to explain further. Thanks!

--- Comment by louzongzhi at 2026-04-24T11:40:03Z ---
> Hey! ty for the PR that's not the way we want to add it! I have a draft PR for this let me link it 🤗

By "the way we want to add it," do you mean implementing something similar to the `DeepseekV4Cache` in your draft ([#45616](https://github.com/huggingface/transformers/pull/45616))?

That was actually my initial approach too, but I noticed `Qwen3NextDynamicCache` was removed recently, which made me really confused about the preferred pattern.

--- Comment by louzongzhi at 2026-04-26T12:13:55Z ---
> Hey! ty for the PR that's not the way we want to add it! I have a draft PR for this let me link it 🤗

I’ll close this PR for now and resubmit after the transformers implementation of deepseek_v4 is released.

--- Comment by ArthurZucker at 2026-05-14T06:48:46Z ---
it is now released! 🤗 But I'll try to push it with the dsv 3.2 PR!


PULL REQUEST #45594: fix(utils): Resolve backbone utils test regressions
State: closed  |  Merged: True
Author: harshaljanjani  |  Base: main
Labels:
Created: 2026-04-23T06:14:09Z

### What does this PR do?

The following regressions were identified and fixed in this PR:

→ [0c4fe5c ("Delete duplicate code in backbone utils")](https://github.com/huggingface/transformers/pull/43323) removed `use_pretrained_backbone` from `load_backbone`; pretrained weights are now loaded via `from_pretrained` on the parent. The tests in [test_backbone_utils.py#L217](https://github.com/huggingface/transformers/blob/main/tests/utils/test_backbone_utils.py#L217) weren't updated likely because `tests/utils/` wasn't in the `run-slow` scope of that PR. Updated the test to match the new design (`load_backbone` always initializes with random weights).
→ [dff54ad ("out_indices always a list")](https://github.com/huggingface/transformers/pull/30941) made `out_indices` always return a list via [backbone_utils.py#L57](https://github.com/huggingface/transformers/blob/main/src/transformers/backbone_utils.py#L57). The assertion at [test_backbone_utils.py#L165](https://github.com/huggingface/transformers/blob/main/tests/utils/test_backbone_utils.py#L165) was written before that change in [#28784](https://github.com/huggingface/transformers/pull/28784) and still expected a tuple. Updated the assertion to expect a list.

cc: @zucchini-nlp @Rocketknight1

**CI coverage fixed by this PR** (checked in [the latest CI run](https://github.com/huggingface/transformers/actions/runs/24758510131/job/72437302961)):

```
tests/generation/test_utils.py::GenerationIntegrationTests::test_green_red_watermark_generation, tests/utils/test_backbone_utils.py::BackboneUtilsTester::test_load_backbone_from_config, tests/utils/test_backbone_utils.py::BackboneUtilsTester::test_load_backbone_in_new_model
```

**Current output:**

**Output after fix:**

### Code Agent Policy
- [x] I confirm that this is not a pure code agent PR.

### Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
    [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
    [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you fix any necessary existing tests?

--- Comment by Rocketknight1 at 2026-04-23T12:35:43Z ---
cc @ydshieh @tarekziade maybe? We may want to consider ditching these tests if they're all slow and we don't even notice when they break!

--- Comment by zucchini-nlp at 2026-04-23T12:40:08Z ---
Hm, I usually try to check daily CI for things that I might have broken, so prob this one slipped off the CI-bot feedback? Been quite a while since the last time "timm" was updated...

--- Comment by harshaljanjani at 2026-04-27T09:23:27Z ---
Good day; just checking in to see if there are any updates on the direction we should take regarding this :)

--- Comment by harshaljanjani at 2026-04-28T10:46:49Z ---
Will look into the same and revert, thank you :)

--- Comment by github-actions[bot] at 2026-04-29T05:27:45Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: timm_backbone

--- Comment by harshaljanjani at 2026-04-29T05:28:49Z ---
@zucchini-nlp Done, sorry for the delay!

--- Comment by zucchini-nlp at 2026-05-01T08:35:07Z ---
run-slow: timm_backbone

--- Comment by github-actions[bot] at 2026-05-01T08:36:25Z ---
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/25208246336)

This comment contains `run-slow`, running the specified jobs:

models: ["models/timm_backbone"]
quantizations: []

--- Comment by github-actions[bot] at 2026-05-01T08:43:12Z ---
## CI Results
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/25208246336)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [d92de1f2](https://github.com/huggingface/transformers/commit/d92de1f25e1ffd6b21abdcab2b95ed974e2565ae) | workflow commit (merge commit) |
| PR | [73326e24](https://github.com/harshaljanjani/transformers/commit/73326e24ab9d555a8001416c600163745a6636c8) | branch commit (from PR) |
| main | [756dffe0](https://github.com/huggingface/transformers/commit/756dffe0d664419d2f620e131ed426373cfd6f12) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !

--- Comment by HuggingFaceDocBuilderDev at 2026-05-01T08:46:04Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45594). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by harshaljanjani at 2026-04-29T05:32:28Z ---
Just removed this one test without relocation since `use_pretrained_backbone` was removed in #43323.


PULL REQUEST #45592: fix padding side issue for fast_vlm tests
State: closed  |  Merged: True
Author: kaixuanliu  |  Base: main
Labels:
Created: 2026-04-23T02:33:50Z

This PR fixes failed test case: `tests/models/fast_vlm/test_modeling_fast_vlm.py::FastVlmForConditionalGenerationIntegrationTest::test_small_model_integration_test_batch`
@ydshieh pls help review, thx!

--- Comment by github-actions[bot] at 2026-04-23T02:35:02Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: fast_vlm

--- Comment by ydshieh at 2026-04-28T15:07:04Z ---
Run in CI runner, all ✅  (for single gpu runner) and indeed this PR also fixes the failing test on our Nvidia CI.


PULL REQUEST #45591: [nemotron_h] respect _no_reinit flag on dt_bias and out_proj.weight
State: closed  |  Merged: True
Author: vai-minzhou  |  Base: main
Labels:
Created: 2026-04-23T01:47:23Z

## Summary

`NemotronHPreTrainedModel._init_weights` unconditionally overwrites two trained parameters every time it is invoked:
- `NemotronHMamba2Mixer.dt_bias` — reset to a fresh `inv_softplus(random dt)` draw
- `{…}.out_proj.weight` — reset to a kaiming-uniform scaled by `1/sqrt(num_hidden_layers)`

It sets `module.dt_bias._no_reinit = True` after the copy, but that flag is only checked for the `nn.Linear.bias` branch of the same function — never for `dt_bias` itself, and `out_proj.weight` doesn't set the flag at all.

On `transformers>=5.0`, `_init_weights` runs a second time after `from_pretrained` has finished loading the checkpoint (the post-load pass that initialises tensors still on `meta`). For `NemotronHForCausalLM` that silently overwrites the on-disk values for `dt_bias` and `out_proj.weight` with fresh random ones, while all other tensors keep their trained values.

The resulting model outputs repetitive filler streams like ` and and and , and and ,` for any input — sanity is preserved only when loading through vLLM (which bypasses `_init_weights`) or via an older transformers release.

## Reproduction

```python
import json, pathlib, torch
from safetensors.torch import load_file
from transformers import AutoConfig, AutoModelForCausalLM

path = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"   # any Nemotron-H ckpt
cfg = AutoConfig.from_pretrained(path, trust_remote_code=True)
cfg._attn_implementation = "eager"
m = AutoModelForCausalLM.from_pretrained(path, config=cfg, torch_dtype=torch.bfloat16)

idx = json.load(open(pathlib.Path(path) / "model.safetensors.index.json"))["weight_map"]
k = "backbone.layers.0.mixer.dt_bias"
on_disk = load_file(f"{path}/{idx[k]}")[k]
in_mem  = m.backbone.layers[0].mixer.dt_bias
print((on_disk.float() - in_mem.float().cpu()).abs().max().item())
# → ~26.8 before this patch, 0 after
```

Prompting ``"Hello, how are you? I am"`` on an unpatched load returns ``' and' ' in' ' the' ' first' ','`` as top-5 next tokens — a symptom of Mamba2 with randomised `dt_bias` and mis-scaled `out_proj`. After the patch, trained values are preserved and the model generates normally.

## The fix

Both changes live in `NemotronHPreTrainedModel._init_weights`:

1. `dt_bias` branch: early-return if `dt_bias._no_reinit` is already set (the flag is set at the end of the current branch, so the first pass initialises normally, the second pass becomes a no-op).
2. `out_proj.weight` branch: skip when `p._no_reinit` is set, and set `p._no_reinit = True` after the initial kaiming scale so a second invocation is a no-op.

Fresh-init training is unaffected — only the second (post-load) invocation is made idempotent. Same edit is mirrored into `modular_nemotron_h.py` and `modeling_nemotron_h.py`.

## Test plan
- [x] Unpatched load: `|on_disk - in_mem|.max()` for layer-0 dt_bias ≈ 26.8, next-token logits return stop-word garbage.
- [x] Patched load: diff is 0, next-token logits look sane, eval on our NemotronH-based classifier no longer collapses to 1000/1000 parse failures.
- [ ] CI: run `tests/models/nemotron_h/` — no behaviour change for fresh-init, only the idempotence of the re-init pass changes.

Please let me know if you'd like the fix to take a different shape (e.g. short-circuit `_init_weights` entirely when the module's parameters are all materialised, or move the guard to a shared utility in `modeling_utils`). Happy to adjust.

--- Comment by Rocketknight1 at 2026-04-23T12:20:34Z ---
Hey, I'm not sure about this PR! We already have the `_is_hf_initialized` attribute, so I'm worried about the `no_reinit` flag that does the same thing, even though it seems like it's already in the codebase. Can you dig a little deeper and figure out why we don't just use `_is_hf_initialized` here?

--- Comment by vai-minzhou at 2026-04-24T02:01:01Z ---
Thanks for the review! You're right — `_is_hf_initialized` is the canonical flag and is already set on checkpoint-loaded parameters by `from_pretrained`. Pushed a new commit that:

- replaces each `_no_reinit` check with `_is_hf_initialized`, guarded per-parameter (`A_log`, `D`, `dt_bias`, and `out_proj.weight`), so a loaded value survives any subsequent `_init_weights` pass;
- drops the `_no_reinit = True` assignments that this PR previously added (no longer needed — the checkpoint loader already sets the flag).

Left the pre-existing `getattr(module.bias, "_no_reinit", False)` check on `nn.Linear.bias` untouched since it's orthogonal to this fix.

Repro that motivated the original bug: with the old code, loading a finetuned NemotronH checkpoint left `|diff(loaded, effective)| ≈ 26` on `dt_bias` and substantial drift on `out_proj.weight`, because `_init_weights` ran a second time after the safety pass and drew fresh random values. With this fix those params are byte-identical to the checkpoint.

--- Comment by vai-minzhou at 2026-05-01T00:00:12Z ---
Just pushed [`5aee35f`](https://github.com/huggingface/transformers/pull/45591/commits/5aee35fa56b813b91d34ff76cd0a5cec21c4c22f) that swaps the remaining `_no_reinit` getter on `nn.Linear.bias` to `_is_hf_initialized`. Confirmed safe — after my earlier patch removed the only `_no_reinit = True` assignment, that flag is set nowhere in the repo (`grep -rn "_no_reinit *= *True" .` returns nothing), so the old check was effectively dead. Using `_is_hf_initialized` makes the file consistent and gains a small upside: a checkpoint-loaded bias (rare for the layers in question, but possible) now survives a re-init pass.

--- Comment by github-actions[bot] at 2026-05-01T00:01:22Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: nemotron_h

--- Comment by Rocketknight1 at 2026-05-01T13:25:00Z ---
run-slow: nemotron_h

--- Comment by github-actions[bot] at 2026-05-01T13:26:29Z ---
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/25215936567)

This comment contains `run-slow`, running the specified jobs:

models: ["models/nemotron_h"]
quantizations: []

--- Comment by github-actions[bot] at 2026-05-01T13:35:59Z ---
## CI Results
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/25215936567)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [0030eaa5](https://github.com/huggingface/transformers/commit/0030eaa50e5bb1bc07813799f481fc2111b17558) | workflow commit (merge commit) |
| PR | [5aee35fa](https://github.com/vai-minzhou/transformers/commit/5aee35fa56b813b91d34ff76cd0a5cec21c4c22f) | branch commit (from PR) |
| main | [807d9d79](https://github.com/huggingface/transformers/commit/807d9d798e2af2b9b2b2b8fa758c26316ac2df72) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !

--- Comment by HuggingFaceDocBuilderDev at 2026-05-01T13:36:12Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45591). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by Rocketknight1 at 2026-05-01T13:37:04Z ---
LGTM now, I think we can merge!

--- Comment by Rocketknight1 at 2026-04-24T13:16:43Z ---
The code is still referencing `_no_reinit` here - can we just replace with `_is_hf_initialized` everywhere in the file, or will that cause problems?


PULL REQUEST #45590: fix #45588: guard s_aux against None in flash_attention_forward
State: closed  |  Merged: False
Author: ghost  |  Base: main
Labels: Code agent slop
Created: 2026-04-23T01:06:37Z

## Fix for #45588

### Bug
`flash_attention_forward` unconditionally calls `s_aux.to(query.dtype)`, but `s_aux` defaults to `None` and sink-less models (e.g. Gemma 4) never pass `s_aux`, causing:

```
AttributeError: 'NoneType' object has no attribute 'to'
```

### Fix
Add a guard to only convert `s_aux` when it is not `None`:

```python
# Before
s_aux=s_aux.to(query.dtype)

# After
s_aux=s_aux.to(query.dtype) if s_aux is not None else None
```

This pattern was already used in `flash_paged.py` (see PR #40434).

### Testing
- [x] Python syntax check passed
- [x] Code follows existing pattern in codebase

### Notes
- Huggingface transformers v5.6.0
- Affects sink-less models using flash_attention_2

---
*Automated high-quality fix*

--- Comment by github-actions[bot] at 2026-04-23T01:06:46Z ---
This PR was flagged by our automated quality checks. If you're a genuine
contributor, please reply here and a maintainer will review your PR.

Common reasons for flagging:
- New GitHub account
- Unusually high number of repository forks in a 24-hour window

We appreciate your contribution and apologize if this is a false positive!


PULL REQUEST #45589: Fix `AttributeError` on `s_aux=None` in `flash_attention_forward`
State: closed  |  Merged: True
Author: jamesbraza  |  Base: main
Labels: for patch
Created: 2026-04-23T00:57:13Z

Fixes https://github.com/huggingface/transformers/issues/45588

@ArthurZucker  @yonigozlan @molbap


[docs] cb memory management

PULL REQUEST #45587: [docs] cb memory management
State: closed  |  Merged: True
Author: stevhliu  |  Base: main
Labels:
Created: 2026-04-22T22:12:16Z

follows up on the comment [here](https://github.com/huggingface/transformers/pull/44896#pullrequestreview-4028794356) to document the memory side of the system

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T22:23:06Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45587). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by remi-or at 2026-04-28T02:12:42Z ---
```suggestion
The manager infers the number of blocks at startup from free GPU memory. The manager solves an equation that accounts for KV tensors, attention masks, activations, and bookkeeping indices, then sizes the pool to fit inside `max_memory_percent` (default 0.9) of the available memory.

--- Comment by remi-or at 2026-04-28T02:13:37Z ---
Perhaps replace `[soft reset](#soft-reset)` with `[offload](#offload)`

--- Comment by remi-or at 2026-04-28T02:17:48Z ---
`a soft reset` -> `offloading`


Add Audio-Visual Flamingo model

PULL REQUEST #45586: Add Audio-Visual Flamingo model
State: open  |  Merged: False
Author: lashahub  |  Base: main
Labels:
Created: 2026-04-22T19:05:22Z

This PR adds support for **Audio-Visual Flamingo (AVF)**, an open audio-visual large language model for joint understanding and reasoning over audio, images, and videos.

The paper, model weights, and project page will be released in May 2026.

In Transformers, AVF pairs a **SigLIP** vision tower with an **AF-Whisper** audio encoder and a **Qwen2.5-7B** causal language model, with separate projectors for visual and audio features. For joint video-audio inputs, AVF aligns synchronized visual and audio chunks, interleaves them along the time axis, applies **Constrained Rotary Time Embeddings (CRTE)**, and feeds the fused sequence to the language model. It also supports **Dynamic-S2** preprocessing for high-resolution images and sampled video frames.

This PR introduces:
- `AudioVisualFlamingoConfig`
- `AudioVisualFlamingoForConditionalGeneration`
- `AudioVisualFlamingoProcessor`
- Joint video-audio handling from a single container via `load_audio_in_video=True`
- Dynamic-S2 visual preprocessing
- Temporal audio-visual interleaving with CRTE
- Modeling, processing, tests, and documentation

Example usage once the checkpoint is public:

```python
from transformers import AudioVisualFlamingoForConditionalGeneration, AutoProcessor

model_id = "nvidia/audio-visual-flamingo-hf"

processor = AutoProcessor.from_pretrained(
    model_id,
    padding_side="left",
    use_fast=False,
    load_audio_in_video=True,
    num_video_frames=128,
    audio_chunk_length="max_3600",
)
model = AudioVisualFlamingoForConditionalGeneration.from_pretrained(
    model_id,
    device_map="auto",
    load_audio_in_video=True,
).eval()

conversation = [
    {
        "role": "user",
        "content": [
            {"type": "video", "video": "video.mp4"},
            {
                "type": "text",
                "text": "Describe both the visual scene and the spoken or environmental audio content.",
            },
        ],
    }
]

inputs = processor.apply_chat_template(
    conversation,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(outputs[:, inputs.input_ids.shape[1] :], skip_special_tokens=True)[0])
```

--- Comment by github-actions[bot] at 2026-04-22T21:32:35Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: audiovisualflamingo, auto


PULL REQUEST #45585: qa: bumped mlinter and allow local override
State: closed  |  Merged: True
Author: tarekziade  |  Base: main
Labels:
Created: 2026-04-22T18:22:35Z

# What does this PR do?

- bumps mlinter to 0.1.1
- add a local config that can be used to quickly override the linter rules

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T18:32:42Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45585). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by vasqu at 2026-04-23T12:25:37Z ---
```suggestion
RULES_TOML_PATH = ROOT / "utils" / "rules.toml"
```
nit: any reason not to be explicit here?

--- Comment by vasqu at 2026-04-23T12:27:22Z ---
Ah ig maybe because you wanted it to be consistent across files(?)

--- Comment by vasqu at 2026-04-23T12:27:48Z ---
Let's add our licence for here as well

--- Comment by vasqu at 2026-04-23T12:29:18Z ---
Oh same re licence just noticed

--- Comment by tarekziade at 2026-04-23T14:24:23Z ---
sure. both works, yours look better :)

--- Comment by tarekziade at 2026-04-23T14:25:19Z ---
Not really...


Update dev version

PULL REQUEST #45583: Update dev version
State: closed  |  Merged: True
Author: vasqu  |  Base: main
Labels:
Created: 2026-04-22T16:00:35Z

As per title cc @ArthurZucker @Cyrilvallez

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T16:10:52Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45583). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


PULL REQUEST #45582: generate: drop stale num_return_sequences warning on continuous batching path
State: closed  |  Merged: True
Author: joaquinhuigomez  |  Base: main
Labels:
Created: 2026-04-22T15:51:03Z

The continuous-batching branch in `generate` warned that `num_return_sequences` was unsupported alongside `num_beams`, but `generate_batch()` already honors `generation_config.num_return_sequences` when expanding requests. The warning fires for any run that explicitly sets `num_return_sequences` even though the feature works, cluttering logs and misleading users.

Drop the `num_return_sequences` half of the warning; keep the `num_beams` guard since beam search is still unsupported on the CB path.

Fixes #45563

--- Comment by remi-or at 2026-04-23T06:18:06Z ---
@bot /style

--- Comment by github-actions[bot] at 2026-04-23T06:18:43Z ---
Style fix bot fixed some files and pushed the changes.

--- Comment by HuggingFaceDocBuilderDev at 2026-04-23T08:16:23Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45582). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


PULL REQUEST #45581: Add automated reviewer assignment script
State: closed  |  Merged: False
Author: saiganesh47  |  Base: main
Labels: Code agent slop
Created: 2026-04-22T15:18:23Z

This script automates reviewer assignment based on CODEOWNERS and lines of code changed in a pull request.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
**we ask that new users do not submit pure code agent PRs** at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [ ] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


[`Privacy Filter`] Add model

PULL REQUEST #45580: [`Privacy Filter`] Add model
State: closed  |  Merged: True
Author: vasqu  |  Base: main
Labels: New model
Created: 2026-04-22T15:17:00Z

As per title

--- Comment by github-actions[bot] at 2026-04-22T15:18:16Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, gpt_oss, openai_privacy_filter

--- Comment by vasqu at 2026-04-22T15:22:19Z ---
Checked locally that everything passes

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T15:27:32Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45580). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


Update assign_reviewers.py

PULL REQUEST #45579: Update assign_reviewers.py
State: closed  |  Merged: False
Author: saiganesh47  |  Base: main
Labels:
Created: 2026-04-22T15:13:44Z

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
**we ask that new users do not submit pure code agent PRs** at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [ ] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Remove attribute_map from GptOssConfig

PULL REQUEST #45578: Remove attribute_map from GptOssConfig
State: open  |  Merged: False
Author: AmineDiro  |  Base: main
Labels:
Created: 2026-04-22T13:50:39Z

Added in #45473,  it clobbers num_local_experts when checkpoints carry both keys (breaks tiny-GptOssForCausalLM loading in PEFT/TRL CI) : https://github.com/huggingface/transformers/pull/45473/changes/BASE..20c2a54ea0f7669d1c5af1fa512ca91379e48df6#r3116765952

Was added for API parity with Mixtral/MiniMax `{"num_experts": "num_local_experts"}`.  Looked like a natural consistency change to land alongside the EP fixes.

Transformers CI didn't fail. The bug only fires when a `config.json` carries both `num_experts` and `num_local_experts`. GPT-OSS-20 has only the latter, so the transformers tests pass. TRL/PEFT CI pin on `trl-internal-testing/tiny-GptOssForCausalLM`, whose config has both [(num_experts: 4, num_local_experts: 128](https://huggingface.co/trl-internal-testing/tiny-GptOssForCausalLM/commit/ffe015e42676a3b43e8ff4534b67d3f3eeb0f25a)). With the attribute_map, the num_experts=4 kwarg silently rewrites num_local_experts to 4, so the built model has 4 experts while the
checkpoint holds 128, that's why we got the shape MISMATCH.

Although I saw @albertvillanova fixing the `num_local_experts` 👀  in   https://huggingface.co/trl-internal-testing/tiny-GptOssForCausalLM/commit/ffe015e42676a3b43e8ff4534b67d3f3eeb0f25a

# What does this PR do?
- Removes config attr_map for num_experts and num_local_experts

## Who can review?
 @ArthurZucker @BenjaminBossan  @qgallouedec  @albertvillanova

--- Comment by github-actions[bot] at 2026-04-22T13:51:47Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gpt_oss

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T14:01:44Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45578). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by Rocketknight1 at 2026-04-23T11:36:50Z ---
cc @arthurzucker @benjaminbossan from the previous issue

--- Comment by BenjaminBossan at 2026-04-23T13:48:40Z ---
My understanding is that this should no longer be necessary, but I'll leave it up to others to make the call.

--- Comment by AmineDiro at 2026-04-25T07:20:02Z ---
I can just close this PR if it's not needed. We just need to confirm API parity with Mixtral/MiniMax :) ?


PULL REQUEST #45577: Allow for registered experts from kernels hub
State: closed  |  Merged: True
Author: winglian  |  Base: main
Labels:
Created: 2026-04-22T13:41:37Z

# What does this PR do?

When loading custom expert kernels from HF Hub, there is still a check that prevents them from being used. This grabs the list of registered expert fns instead of hardcoding.

Fixes # (issue) N/A

## Code Agent Policy

AI was used just to write the unit tests.

- [X] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?

## Who can review?

@ArthurZucker @SunMarc

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T13:53:17Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45577). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by SunMarc at 2026-04-22T13:56:12Z ---
@bot /style

--- Comment by SunMarc at 2026-04-22T14:18:01Z ---
@bot /style

--- Comment by IlyasMoutawwakil at 2026-04-23T07:15:33Z ---
thanks for catching this, i left a small suggestion to allow fp8 implementations as well.

--- Comment by IlyasMoutawwakil at 2026-04-23T08:46:28Z ---
@bot /style

--- Comment by IlyasMoutawwakil at 2026-04-23T09:30:12Z ---
@bot /style

--- Comment by github-actions[bot] at 2026-04-23T09:30:54Z ---
Style fix bot fixed some files and pushed the changes.

--- Comment by IlyasMoutawwakil at 2026-04-23T10:10:27Z ---
😭

--- Comment by SunMarc at 2026-04-22T13:55:34Z ---
we can remove `deepgemm` in the `base_experts_fns`. it will be present in `ALL_EXPERTS_FUNCTIONS` if fp8 is used.
```suggestion
        base_experts_fns = ["eager"] + list(ALL_EXPERTS_FUNCTIONS.keys())
```

--- Comment by IlyasMoutawwakil at 2026-04-22T16:03:48Z ---
actually fp8 has its own different registry, maybe we can add both registeries

--- Comment by IlyasMoutawwakil at 2026-04-23T07:13:26Z ---
```suggestion
from .integrations.moe import ALL_EXPERTS_FUNCTIONS
from .intergrations.finegrained_fp8 import ALL_FP8_EXPERTS_FUNCTIONS
```

--- Comment by IlyasMoutawwakil at 2026-04-23T07:15:00Z ---
```suggestion
        base_experts_fns = list(set(["eager"] + list(ALL_EXPERTS_FUNCTIONS.keys()) + list(ALL_FP8_EXPERTS_FUNCTIONS.keys())))
```

--- Comment by IlyasMoutawwakil at 2026-04-23T09:29:27Z ---
```suggestion
from .integrations.finegrained_fp8 import ALL_FP8_EXPERTS_FUNCTIONS
```


PULL REQUEST #45576: docs(pipeline): fix num_workers docstring default from 8 to 0
State: closed  |  Merged: False
Author: Anai-Guo  |  Base: main
Labels: Code agent slop
Created: 2026-04-22T13:14:19Z

## What does this PR do?

Fixes #45557.

The docstring for `num_workers` in `build_pipeline_init_args()` states `defaults to 8`, but the actual runtime fallback in `Pipeline.__call__()` is `0`:

```python
# Line 1210 in base.py
if self._num_workers is None:
    num_workers = 0  # actual default
```

This PR updates the docstring to accurately reflect the runtime behavior (`defaults to 0`), preventing confusion for users relying on the docs for performance tuning.

## Before submitting
- [x] This PR fixes a typo or improves the docs (and does not need the community review, just direct merge by the author)
- [x] Did you read the contributor guideline?

Fixes: #45557

--- Comment by github-actions[bot] at 2026-04-22T13:14:33Z ---
This PR was flagged by our automated quality checks. If you're a genuine
contributor, please reply here and a maintainer will review your PR.

Common reasons for flagging:
- New GitHub account
- Unusually high number of repository forks in a 24-hour window

We appreciate your contribution and apologize if this is a false positive!


PULL REQUEST #45575: fix(generation): remove stale warning for num_return_sequences in paged generate
State: closed  |  Merged: False
Author: CodersAcademy006  |  Base: main
Labels: Code agent slop
Created: 2026-04-22T11:32:16Z

Fixes #45563.

generate(..., cache_implementation=paged) incorrectly warned that num_return_sequences is unsupported for continuous batching. This warning is stale: generate_batch() already uses generation_config.num_return_sequences to expand the number of requests.

Changes: Split the guard in src/transformers/generation/utils.py — keep warning only for num_beams > 1 (beam search is genuinely unsupported), remove the warning for num_return_sequences.

Test plan:
- generate(..., cache_implementation=paged, num_return_sequences=2) no longer emits warning
- generate(..., cache_implementation=paged, num_beams=2) still emits warning

Reference: https://github.com/oleksii-tumanov/transformers/commit/f7a939d95239d26f94195cd6b820e4c720976507


Fix typos

PULL REQUEST #45574: Fix typos
State: closed  |  Merged: True
Author: vasqu  |  Base: main
Labels:
Created: 2026-04-22T11:28:42Z

As per title

--- Comment by github-actions[bot] at 2026-04-22T11:30:43Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, esm, hy_v3

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T11:58:29Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45574). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


PULL REQUEST #45573: fix transformers + torchao nvfp4 serialization
State: closed  |  Merged: True
Author: vkuzo  |  Base: main
Labels:
Created: 2026-04-22T10:50:42Z

Summary:

1. fix torchao NVFP4 serialization with transformers
2. add a test to cover the fix

While i'm here, also did the following bundled into this PR:
3. make the torchao serialization test have human readable names (easier to debug)
4. fix the float8 test (update the expected output)

after this PR the test command for all torchao configs passes on an NVIDIA B200

Co-developed with Claude.

Test Plan:

```
RUN_SLOW=1 pytest tests/quantization/torchao_integration/test_torchao.py -k "Serialization" -s
```

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
**we ask that new users do not submit pure code agent PRs** at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [x] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

--- Comment by vkuzo at 2026-04-22T11:08:47Z ---
test failure looks unrelated

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T19:13:23Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45573). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by jerryzh168 at 2026-04-22T19:43:31Z ---
so torchao tests are actually running in transformers repo's CI and only nvfp4 is broken right?

--- Comment by github-actions[bot] at 2026-04-22T20:01:23Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: torchao_integration

--- Comment by SunMarc at 2026-04-22T20:01:32Z ---
> so torchao tests are actually running in transformers repo's CI and only nvfp4 is broken right?

The tests are working fine, we are just adding coverage for nvfp4 here @jerryzh168. I've reworked the testing suite + the integration a few weeks again to clean a bit.

--- Comment by SunMarc at 2026-04-22T12:42:24Z ---
We are always using the latest version of torchao in our tests so maybe it is fine to not do the try except ?

--- Comment by jerryzh168 at 2026-04-22T17:14:53Z ---
I guess we can use latest stable, and have one PR to fix any breakages and bump version after every release, so not needed if it exists in latest stable


PULL REQUEST #45572: refactor(Dots1): drop Dots1MoE override to `pass` (inherits from DSV3 MoE)
State: closed  |  Merged: True
Author: casinca  |  Base: main
Labels:
Created: 2026-04-22T10:20:01Z

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Hello, Fixes https://github.com/huggingface/transformers/pull/45441#pullrequestreview-4152049510

The `Dots1MoE` method `route_tokens_to_experts` is exactly the same as in `DeepseekV3MoE` (despite the inline comment saying otherwise, probably a remnant) therefore there's no need to override it.

It will also benefit (via inheritance) from the fix in #45441

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
**we ask that new users do not submit pure code agent PRs** at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [x] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->
It was discussed with @ArthurZucker and @vasqu

--- Comment by github-actions[bot] at 2026-04-22T10:21:05Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: dots1

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T11:01:02Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45572). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


PULL REQUEST #45570: Fix whisper long-form generation when eos_token_id is a list
State: open  |  Merged: False
Author: ronansgd  |  Base: main
Labels:
Created: 2026-04-22T08:50:02Z

# What does this PR do?

Fixes: #45584

Fixes a bug in Whisper generation code, happening when `generation_config.eos_token_id` is a `list[int]` and not an `int` (happens for instance after `align_special_tokens` is called in `Trainer.train`).

Fix is to normalize to a list and use membership checks instead of equality.

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
**we ask that new users do not submit pure code agent PRs** at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [x] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

cc @eustlb

--- Comment by github-actions[bot] at 2026-04-22T08:51:30Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: whisper

--- Comment by ronansgd at 2026-04-22T17:24:47Z ---
Hey @eustlb, I've opened an issue #45584 with a minimal reproducer script


Proper nemotron H and 3 and 2

PULL REQUEST #45569: Proper nemotron H and 3 and 2
State: open  |  Merged: False
Author: ArthurZucker  |  Base: main
Labels:
Created: 2026-04-22T07:03:26Z

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
**we ask that new users do not submit pure code agent PRs** at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [ ] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

--- Comment by github-actions[bot] at 2026-04-22T13:09:58Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, nemotron_h, nemotron_h_dense, nemotron_h_sparse

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T13:18:43Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45569). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


Gemma4: fix failed test cases

PULL REQUEST #45568: Gemma4: fix failed test cases
State: closed  |  Merged: True
Author: kaixuanliu  |  Base: main
Labels:
Created: 2026-04-22T06:52:45Z

# What does this PR do?

This PR did several things:
1. Skip some test cases that are not suitbale for gemma4 model
2. Fix bug when `attention_mask` is None(tests/models/gemma4/test_modeling_gemma4.py::Gemma4Audio2TextModelTest::test_eager_matches_fa2_generate)
3. fix some failed test cases related to `test_flash_attn_x_from_config`
4. Add XPU related Expectations

Fixes # (issue)

## Code Agent Policy

- [X] I confirm that this is not a pure code agent PR.

## Who can review?
@ydshieh pls help review

--- Comment by Qodo-Free-For-OSS at 2026-04-30T09:32:17Z ---
Hi, Integration tests add device-specific Expectations entries for XPU without a default fallback, so running these tests on an unsupported accelerator type (or an XPU generation not covered) can select an unintended expectation or raise if no expectation matches. This makes the tests more brittle to new device properties.

**Severity:** informational | **Category:** reliability

**How to fix:** Add default expectation fallback

**Agent prompt to fix** - you can give this to your LLM of choice:

> ### Issue description
> XPU expectations were added without a `(None, None)` default. This can make the test brittle when run on different XPU generations or unexpected device properties.
>
> ### Fix Focus Areas
> - tests/models/gemma4/test_modeling_gemma4.py[534-542]
> - tests/models/gemma4/test_modeling_gemma4.py[575-593]
> - tests/models/gemma4/test_modeling_gemma4.py[621-629]
> - tests/models/gemma4/test_modeling_gemma4.py[675-681]
> - tests/models/gemma4/test_modeling_gemma4.py[745-755]
>
> ### Recommended changes
> - Add a default expectation `(None, None): <existing cuda expectation>` if you want to preserve previous behavior for other devices.
> - Or, add additional XPU keys if multiple gens are expected to run these tests.
> - Ensure the intended device coverage is explicit to avoid accidental matching on future hardware.

---
*Found by [Qodo](https://qodo.ai) code review*

--- Comment by vasqu at 2026-05-05T14:12:10Z ---
run-slow: gemma4

--- Comment by github-actions[bot] at 2026-05-05T14:13:32Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma4

--- Comment by github-actions[bot] at 2026-05-05T14:14:06Z ---
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/25381629130)

This comment contains `run-slow`, running the specified jobs:

models: ["models/gemma4"]
quantizations: []

--- Comment by HuggingFaceDocBuilderDev at 2026-05-05T14:23:40Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45568). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by github-actions[bot] at 2026-05-05T14:50:58Z ---
## CI Results
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/25381629130)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [a3c44435](https://github.com/huggingface/transformers/commit/a3c4443502c7df6cb736e34a2e4da3434c8c2d41) | workflow commit (merge commit) |
| PR | [37b8baa4](https://github.com/kaixuanliu/transformers/commit/37b8baa48d3fe610dbc7070c366454902151ea8e) | branch commit (from PR) |
| main | [a6ccf935](https://github.com/huggingface/transformers/commit/a6ccf9354a4b9f6ec8d9c7482ed5e37734b6dbb2) | base commit (on `main`) |

### Model CI Report

❌ **[1 new failed tests from this PR](https://huggingface.co/datasets/hf-internal-testing/transformers_pr_ci/raw/739d9e255e95db6ac4eda398d999569300508d90/2026-05-05/runs/34191-25381629130/ci_results_run_models_gpu/new_failures_with_bad_commit.json)** 😭

- [gemma4](https://github.com/huggingface/transformers/actions/runs/25381629130/job/74431715944):
    tests/models/gemma4/test_modeling_gemma4.py::Gemma4IntegrationTest::test_export_text_only (❌ ⟹ ❌)

--- Comment by ydshieh at 2026-04-29T12:37:17Z ---
@Cyrilvallez  any opinion.

From PR descriptioin

> Fix bug when attention_mask is None(tests/models/gemma4/test_modeling_gemma4.py::Gemma4Audio2TextModelTest::test_eager_matches_fa2_generate)

--- Comment by ydshieh at 2026-04-29T12:40:29Z ---
OK for me. Just let one of @Cyrilvallez or @vasqu to also valid or comment

--- Comment by ydshieh at 2026-04-29T12:40:45Z ---
They are indeed failing on our CI too

--- Comment by ydshieh at 2026-04-29T12:43:04Z ---
@kaixuanliu I didn't see these 2 failing on our Flash Attn CI job.

Could you share more info / error logs ?

--- Comment by vasqu at 2026-04-29T13:00:24Z ---
Our flash attn ci doesn have FA3 - I think it's hard to install because you need to compile from source and it's much longer than FA2 build from source

Maybe we could add a separate FA4 CI - not sure how stable it is tho since it's still in beta

--- Comment by vasqu at 2026-04-29T13:00:55Z ---
Hmm, iirc the fallback should be compile compatible cc @IlyasMoutawwakil

--- Comment by vasqu at 2026-04-29T13:02:16Z ---
Can we have something like https://github.com/huggingface/transformers/blob/727741f533b1f47120db47ef5934296d07214d7e/tests/models/dia/test_modeling_dia.py#L258-L271

But for FA? Imo it will always be quite a lot to skip these manually like that

--- Comment by vasqu at 2026-04-29T13:02:59Z ---
Which torch version is CI now at btw?

--- Comment by IlyasMoutawwakil at 2026-04-29T13:12:23Z ---
it is torch compileable, just not any mode that uses cuda graphs (like max-autotune), where the torch.grouped_mm also fails on <sm90 (it also uses the fallback path)

--- Comment by kaixuanliu at 2026-04-29T13:13:03Z ---
Well, for FA3 and FA4, on my env they are skipped as well. I can delete these two.

--- Comment by vasqu at 2026-04-29T13:16:51Z ---
Ah no, see my comment below https://github.com/huggingface/transformers/pull/45568#discussion_r3161170061

--- Comment by ydshieh at 2026-04-29T14:26:46Z ---
No, I mean for

test_flash_attn_2_from_config

our CI is `[PASSED]`. So I am not sure why we need this fix, at least for FA2.

Our CI runner don't have FA3 or FA4, so they are skipped. But the question may still valid: do we really this fix?

--- Comment by kaixuanliu at 2026-04-29T14:48:07Z ---
well, the code is added before this PR is merged: https://github.com/huggingface/transformers/pull/45454, it will crash for this case before [here](https://github.com/huggingface/transformers/pull/45454/changes#diff-663099e7a4d592c4a9bcad8f04f555f71d0475b8fa9e7ec399845d29ef5ba18dL2100) is removed. After this PR this case can pass. I will update the code.

--- Comment by kaixuanliu at 2026-04-29T15:16:16Z ---
Thx for the advice. Is it OK now?

--- Comment by vasqu at 2026-04-29T15:23:24Z ---
So it shouldn't fail iiuc?

--- Comment by vasqu at 2026-04-29T15:23:37Z ---
Yep much better thanks

--- Comment by kaixuanliu at 2026-04-30T01:45:03Z ---
In the skipped test case, they use mode="reduce-overhead", which use cuda graphs and will fail at [L238](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/moe.py#L238) with error:
```
E                       torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
E                       GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 0) (unhinted: Eq(u0, 0)).  (Size-like symbols: none)
E
E                       consider using data-dependent friendly APIs such as guard_or_false, guard_or_true and statically_known_true.
E                       Caused by: (oot/upstream/transformers/src/transformers/integrations/moe.py:238 in _grouped_mm_fallback_backward)
E                       For more information, run with TORCH_LOGS="dynamic"
E                       For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0"
E                       If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1
E                       For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing
E
E                       For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1
E
E                       Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"

/opt/venv/lib/python3.12/site-packages/torch/fx/experimental/symbolic_shapes.py:7862: BackendCompilerFailed
```

--- Comment by vasqu at 2026-04-30T12:40:25Z ---
Can we adjust the comment then at least?

--- Comment by kaixuanliu at 2026-04-30T14:06:16Z ---
Done.

--- Comment by vasqu at 2026-04-30T15:30:48Z ---
cc @zucchini-nlp for viz, don't think it's super important but would still be nice to fix at some point ig

--- Comment by zucchini-nlp at 2026-05-01T08:37:40Z ---
` Gemma4 requires mm_token_type_ids in train mode, so we test in eval mode` > was fixed already no?

If not, @kaixuanliu can you open an issue and ping me there. I might forget to come back when bugs are reported under PRs 😅

--- Comment by kaixuanliu at 2026-05-02T13:37:55Z ---
It's already fixed. And I have removed this part.


PULL REQUEST #45567: Move some conversion mappings to PrefixChange
State: closed  |  Merged: True
Author: Cyrilvallez  |  Base: main
Labels:
Created: 2026-04-22T05:49:53Z

# What does this PR do?

As per the title.
cc @vasqu @zucchini-nlp

Confirmed with the following script that it works correctly:

```python
import transformers
from safetensors.torch import load_file
from transformers.utils.hub import cached_file, cached_files
import json
import torch

# model_id = "Qwen/Qwen3.5-0.8B"
# model_id = "google/gemma-3n-E4B"
model_id = "vidore/colqwen2-v1.0-hf"
model_class = transformers.ColQwen2ForRetrieval

target_folder = "/raid/cyril/test_model"

try:
    with open(cached_file(model_id, "model.safetensors.index.json")) as f:
        index = json.load(f)
    model_files = set(index["weight_map"].values())
    model_files = cached_files(model_id, model_files)
except OSError:
    model_files = cached_files(model_id, ["model.safetensors"])

original_state_dict = {}
for file in model_files:
    original_state_dict.update(load_file(file))
original_state_dict = {k: v.to(torch.bfloat16) for k,v in original_state_dict.items()}

model = model_class.from_pretrained(model_id, dtype=torch.bfloat16)
model.save_pretrained(target_folder)

saved_weights = load_file(f"{target_folder}/model.safetensors")

not_in_saved = []
for k, v in original_state_dict.items():
    if k not in saved_weights:
        not_in_saved.append(k)
    else:
        assert (v == saved_weights[k]).all()

not_in_original = []
for k, v in saved_weights.items():
    if k not in original_state_dict:
        not_in_original.append(k)
    else:
        assert (v == original_state_dict[k]).all()

print(f"The following are in original but not in saved: {not_in_saved}")
print(f"The following are saved but not in original: {not_in_original}")

model = model_class.from_pretrained(target_folder)
```

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T06:00:55Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45567). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


PULL REQUEST #45566: fix: raise clear error when tokenizer config uses v5 list format on older versions
State: closed  |  Merged: False
Author: armorbreak001  |  Base: main
Labels: Code agent slop
Created: 2026-04-22T05:27:49Z

## Summary

When loading a model whose `tokenizer_config.json` uses the v5-style list-based `extra_special_tokens` format on transformers < 5.0, `_set_model_specific_special_tokens()` crashes with a misleading:

```
AttributeError: 'list' object has no attribute 'keys'
```

This sends users down the wrong debugging path (patching config files, reinstalling packages) when the real issue is a version mismatch.

## Fix

Add an `isinstance(special_tokens, list)` guard at the top of `_set_model_specific_special_tokens()` that raises a clear `ValueError` telling users to upgrade to transformers >= 5.0.0.

Fixes #45376

--- Comment by github-actions[bot] at 2026-04-22T05:28:02Z ---
This PR was flagged by our automated quality checks. If you're a genuine
contributor, please reply here and a maintainer will review your PR.

Common reasons for flagging:
- New GitHub account
- Unusually high number of repository forks in a 24-hour window

We appreciate your contribution and apologize if this is a false positive!


PULL REQUEST #45565: fix: remove stale num_return_sequences warning in paged generate
State: closed  |  Merged: False
Author: armorbreak001  |  Base: main
Labels: Code agent slop
Created: 2026-04-22T05:24:00Z

## Summary

The `generate(..., cache_implementation="paged")` path emits a misleading warning that `num_return_sequences` is not supported for continuous batching. This is stale — `generate_batch()` already handles `num_return_sequences` by expanding the number of requests internally.

## Changes

- Remove the `num_return_sequences > 1` condition from the warning check
- Keep the warning for `num_beams > 1` (still not supported)
- Read `num_beams` from the prepared generation config instead of raw kwargs (more accurate)
- Cache the prepared generation config to avoid calling `_prepare_generation_config()` twice

## Before

```
num_return_sequences and num_beams are not supported for continuous batching yet. Got num_return_sequences=2 and num_beams=1.
```

## After (when num_beams=1)

No warning emitted.

## After (when num_beams > 1)

```
num_beams is not supported for continuous batching yet. Got num_beams=3.
```

Fixes #45563

--- Comment by github-actions[bot] at 2026-04-22T05:24:15Z ---
This PR was flagged by our automated quality checks. If you're a genuine
contributor, please reply here and a maintainer will review your PR.

Common reasons for flagging:
- New GitHub account
- Unusually high number of repository forks in a 24-hour window

We appreciate your contribution and apologize if this is a false positive!


PULL REQUEST #45564: Gemma3n and Gemma4 cannot use rotary kernel
State: closed  |  Merged: True
Author: Cyrilvallez  |  Base: main
Labels:
Created: 2026-04-22T04:37:56Z

# What does this PR do?

As per the title. Some layers need to apply the rotary only to `q`, so it cannot use the kernel.
cc @vasqu

--- Comment by github-actions[bot] at 2026-04-22T04:38:58Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3n, gemma4

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T04:49:14Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45564). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by vasqu at 2026-04-22T14:45:46Z ---
Shouldn't be too hard to allow maybe :thinking: same for interleaved and partial layouts imo. It just needs to be properly defined


PULL REQUEST #45562: Updated the image cache for Paddle models according to the latest API
State: closed  |  Merged: True
Author: zhang-prog  |  Base: main
Labels:
Created: 2026-04-22T03:56:09Z

(no description)

--- Comment by vasqu at 2026-04-22T11:13:19Z ---
run-slow: paddleocr_vl, pp_chart2table, pp_doclayout_v2, pp_doclayout_v3, pp_lcnet, pp_ocrv5_mobile_det, pp_ocrv5_mobile_rec, pp_ocrv5_server_det, pp_ocrv5_server_rec, slanext, uvdoc

--- Comment by github-actions[bot] at 2026-04-22T11:14:34Z ---
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/24775195147)

This comment contains `run-slow`, running the specified jobs:

models: ["models/paddleocr_vl", "models/pp_chart2table", "models/pp_doclayout_v2", "models/pp_doclayout_v3", "models/pp_lcnet", "models/pp_ocrv5_mobile_det", "models/pp_ocrv5_mobile_rec", "models/pp_ocrv5_server_det", "models/pp_ocrv5_server_rec", "models/slanext", "models/uvdoc"]
quantizations: []

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T11:24:10Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45562). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by github-actions[bot] at 2026-04-22T11:26:35Z ---
## CI Results
[Workflow Run ⚙](https://github.com/huggingface/transformers/actions/runs/24775195147)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [69de3498](https://github.com/huggingface/transformers/commit/69de3498875cea918d9cb9299fd073f0e281a2ad) | workflow commit (merge commit) |
| PR | [5434ad9c](https://github.com/zhang-prog/transformers/commit/5434ad9c816ba2f2f5b3838a4dd40026c2d7037b) | branch commit (from PR) |
| main | [08244b96](https://github.com/huggingface/transformers/commit/08244b96860aa6ebc0b2fa8c0e070bc9bd2bb1a4) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !

--- Comment by github-actions[bot] at 2026-04-22T11:45:03Z ---
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: paddleocr_vl, pp_chart2table, pp_doclayout_v2, pp_doclayout_v3, pp_lcnet, pp_ocrv5_mobile_det, pp_ocrv5_mobile_rec, pp_ocrv5_server_det, pp_ocrv5_server_rec, slanext, uvdoc


Update torchao usage for XPU and CPU

PULL REQUEST #45560: Update torchao usage for XPU and CPU
State: closed  |  Merged: True
Author: jiqing-feng  |  Base: main
Labels:
Created: 2026-04-22T03:11:44Z

Update torchao quantization doc to align with latest torchao API changes.

## Changes

- **XPU int4**: Removed manual `Int4XPULayout` and `ZeroPointDomain` setup — `Int4WeightOnlyConfig` now handles these internally.
- **CPU int4**: Replaced deprecated `Int4WeightOnlyOpaqueTensorConfig` with `PrototypeInt4WeightOnlyConfig`.
- **`.to()` fix**: Split `.to(device, dtype)` into `.to(device).to(dtype)` to avoid incorrect casting.

## Why

torchao updated its APIs: XPU int4 config now auto-handles layout/zero-point internally, CPU int4 config was renamed and moved to a new module path. The `.to()` chaining fix ensures device placement and dtype casting are applied correctly.

--- Comment by jiqing-feng at 2026-04-22T03:13:44Z ---
Hi @stevhliu  . Would you please review the PR? Thanks!

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T16:25:28Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45560). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


PULL REQUEST #45559: Drop noisy generate warnings when do_sample=False (or num_beams=1)
State: closed  |  Merged: False
Author: ArthurZucker  |  Base: main
Labels:
Created: 2026-04-22T03:03:26Z

## Summary

`GenerationConfig.validate()` was warning about sampling-only flags (`temperature`, `top_p`, `top_k`, `min_p`, `top_h`, `typical_p`, `epsilon_cutoff`, `eta_cutoff`) whenever `do_sample` was not `True`, and about beam-only flags (`early_stopping`, `length_penalty`) whenever `num_beams == 1` -- even when those values were inherited from the model's `generation_config.json`. In practice, nearly every popular Hub model ships with a non-default `temperature`/`top_p`, so users got a warning for every `generate(do_sample=False)` call.

This PR threads a `user_set_attributes` set through `GenerationConfig.__init__` and `update()`, so `validate()` can distinguish attributes the caller **explicitly** provided from values inherited from the model's default config. The sampling-only and beam-only warnings now only fire for user-set attributes.

### Behavior

- `generate(do_sample=False)` on a model whose hub config has `temperature=0.6, top_p=0.9`: **silent** (values inherited, not user intent).
- `generate(do_sample=False, top_p=0.8)`: **warns** about `top_p` (user explicitly set both -- almost certainly a mistake).
- `GenerationConfig(do_sample=False, temperature=0.5)`: **warns** (both explicit).
- `generation_config.validate(strict=True)` (e.g. from `save_pretrained`) with no `user_set_attributes` preserves the original "refuse to save bad configs" behavior.

cc @cyrilvallez

--- Comment by HuggingFaceDocBuilderDev at 2026-04-22T03:32:16Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45559). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by Cyrilvallez at 2026-04-24T06:49:40Z ---
Completely messed up this branch by mistake 😅 Opened https://github.com/huggingface/transformers/pull/45619 with the correct diffs

--- Comment by zucchini-nlp at 2026-04-22T09:36:10Z ---
great opportunity to move to hub-dataclass validation 😄

--- Comment by zucchini-nlp at 2026-04-22T09:39:07Z ---
i am not sure about this one. Beginner users might expect this to just work and sample with top-p, while we silently fallback to greedy

--- Comment by zucchini-nlp at 2026-04-22T09:41:01Z ---
so the is-user check is only when greedy decoding, no changes on other params?


PULL REQUEST #45558: feat(trainer): log individual losses from loss_dict
State: closed  |  Merged: False
Author: Abdeltoto  |  Base: main
Labels:
Created: 2026-04-22T02:56:43Z

# What does this PR do?

Fixes #31081.

When a model returns auxiliary losses alongside the main loss (e.g. via a
`loss_dict` field in its `ModelOutput`), the Trainer currently only logs the
combined `loss`. That makes debugging multi-term objectives painful: you can
see total loss going down without knowing which term is actually moving.

This PR teaches the Trainer to also log each scalar term it finds in
`outputs.loss_dict`, plus any top-level `*_loss` scalar attribute on the
output, under namespaced keys like `loss_dict_<name>` and `loss_<name>`.
Behaviour is opt-in by virtue of the model itself: if no extra losses are
returned, nothing changes. The main `loss` value and all existing logs are
unchanged.

## Implementation notes

- New buffer `self._aux_losses_accumulator` on the `Trainer`, mirroring how
  `_total_loss_scalar` already works.
- In `training_step`, after `compute_loss(..., return_outputs=True)`, scalar
  tensor entries from `outputs.loss_dict` and from `outputs.<name>_loss` are
  detached, gradient-accumulation-scaled, and accumulated. Same DP mean and
  same `num_items_in_batch` normalization as the main loss, so the numbers
  are comparable.
- In `_maybe_log_save_evaluate`, accumulators are gathered across processes
  with `nested_gather` (consistent with the main `tr_loss` path), averaged
  over the logging window, and added to `logs`. Then the buffers are reset.
- Non-tensor / non-scalar values are silently ignored, so models that put
  arbitrary metadata in `loss_dict` won't crash the Trainer.

No public API change. No new dependency. The diff is mostly localized to two
methods in `trainer.py`.

## Tests

`tests/trainer/test_trainer.py::TrainerIntegrationTest::test_trainer_logs_auxiliary_losses_from_loss_dict`

A small `RegressionPreTrainedModelWithLossDict` (added to
`tests/trainer/trainer_test_utils.py`) returns `loss`, `loss_dict={'mse',
'l1'}`, and a top-level `extra_loss`. The test runs a tiny `Trainer.train()`
and asserts the resulting log entries contain the expected
`loss_dict_mse`, `loss_dict_l1`, and `loss_extra` keys with finite,
positive values. Ran locally:

```
Ran 1 test in 0.245s
OK
```

## Code Agent Policy

- [x] I confirm that this is not a pure code agent PR.

I used Cursor as a coding assistant (the commit trailer says
`Made-with: Cursor`), but I read every diff, ran the tests locally, wrote
the description myself, and own the change. Happy to iterate on review
feedback.

## Before submitting
- [ ] This PR fixes a typo or improves the docs.
- [x] Did you read the contributor guideline?
- [x] Was this discussed/approved via a GitHub issue? See #31081.
- [ ] Did you make sure to update the documentation with your changes? No
      user-facing docs change — the new keys appear automatically when a
      model already returns `loss_dict`.
- [x] Did you write any new necessary tests?

## Who can review?

@SunMarc — Trainer maintainer per the template.

--- Comment by Rocketknight1 at 2026-04-22T12:58:49Z ---
Sorry, we really don't want code agent PRs on old issues like this!


PULL REQUEST #45556: Add image processors refactor to v5 migration guide
State: closed  |  Merged: True
Author: yonigozlan  |  Base: main
Labels:
Created: 2026-04-21T19:00:48Z

# What does this PR do?

As discussed internally @vasqu
Cc @stevhliu

--- Comment by HuggingFaceDocBuilderDev at 2026-04-21T19:11:45Z ---
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45556). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

--- Comment by stevhliu at 2026-04-21T19:55:05Z ---
```suggestion
The old slow/fast dual-file design has been replaced with a named-backend architecture. Each model previously had a PIL-based `image_processing_<model>.py` and a torchvision-based `image_processing_<model>_fast.py`. The new layout is:
```

--- Comment by stevhliu at 2026-04-21T20:02:09Z ---
```suggestion
Processor classes now inherit from `TorchvisionBackend` or `PilBackend` (defined in `image_processing_backends.py`), which provide ready-made implementations of all standard operations (`resize`, `rescale`, `normalize`, `center_crop`, `pad`) and a default `_preprocess` pipeline. `BaseImageProcessor` (in `image_processing_utils`) handles shared preprocessing boilerplate: kwargs validation, default-filling from class attributes, and input preparation. Model-specific processors contain only what is unique to the model. Most processors inherit from a backend and declare class-attribute defaults. Only those with custom logic (e.g. patch tiling) need to override `_preprocess`.
```

--- Comment by stevhliu at 2026-04-21T20:03:54Z ---
```suggestion
### Image processors
```