ISSUE #44509: Docs still mention text2text-generation / summarization / translation pipeline tasks which were removed in v5
State: closed  |  Labels: bug
Author: math-hiyoko  |  Created: 2026-03-06T23:34:44Z

### System Info

Transformers version: 5.x

The issue is related to the documentation rather than a specific runtime environment.

### Who can help?

@Rocketknight1 @stevhliu

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

This is a documentation inconsistency in Transformers v5.
text2text-generation / summarization / translation pipeline tasks (`pipeline("text2text-generation")`, `pipeline("summarization")` and `pipeline("translation")`) were removed in v5.
Despite this, the official documentation still contains multiple references to these tasks.

- https://huggingface.co/docs/transformers/ar/tasks/summarization#%D8%A7%D9%84%D8%A7%D8%B3%D8%AA%D8%AF%D9%84%D8%A7%D9%84-inference
- https://huggingface.co/docs/transformers/en/pipeline_tutorial?tasks=summarization#tasks
- https://huggingface.co/docs/transformers/en/model_doc/pegasus#pegasus
- https://huggingface.co/docs/transformers/en/model_doc/pegasus_x#pegasus-x
- https://huggingface.co/docs/transformers/en/model_doc/led#led
- https://huggingface.co/docs/transformers/en/model_doc/bartpho#bartpho
- https://huggingface.co/docs/transformers/en/model_doc/encoder-decoder#encoder-decoder-models
- https://huggingface.co/docs/transformers/en/model_doc/bigbird_pegasus#bigbirdpegasus
- https://huggingface.co/docs/transformers/ja/tasks/translation#inference
- https://huggingface.co/docs/transformers/ko/tasks/translation#inference
- https://huggingface.co/docs/transformers/ja/task_summary#translation
- https://huggingface.co/docs/transformers/zh/task_summary#%E7%BF%BB%E8%AF%91
- https://huggingface.co/docs/transformers/es/task_summary#traducci%C3%B3n
- https://huggingface.co/docs/transformers/fr/task_summary#traduction
- https://huggingface.co/docs/transformers/zh/quicktour#pipeline
- https://huggingface.co/docs/transformers/ar/task_summary#%D8%A7%D9%84%D8%AA%D8%B1%D8%AC%D9%85%D8%A9
- https://huggingface.co/docs/transformers/ko/quicktour#pipeline
- https://huggingface.co/docs/transformers/fr/quicktour#pipeline
- https://huggingface.co/docs/transformers/main/en/model_doc/t5#t5
- https://huggingface.co/docs/transformers/main/en/model_doc/t5gemma#t5gemma
- https://huggingface.co/docs/transformers/main/en/model_doc/switch_transformers#switch-transformers
- https://huggingface.co/docs/transformers/main/en/tasks/prompting#best-practices
- https://huggingface.co/docs/transformers/main/en/model_doc/byt5#byt5
- https://huggingface.co/docs/transformers/main/en/model_doc/mt5#mt5
- https://huggingface.co/docs/transformers/main/ko/tasks/prompting#types-of-models
- https://huggingface.co/docs/transformers/main/en/model_doc/bert-generation#bertgeneration
- https://huggingface.co/docs/transformers/main/ja/tasks/prompting#types-of-models

### Expected behavior

References to `text2text-generation`, `summarization` and `translation` as pipeline task names should be removed.

--- Comment by math-hiyoko at 2026-03-07T13:45:37Z ---
text2text-generation task also

--- Comment by stevhliu at 2026-03-09T15:14:21Z ---
nice, thanks! ~would you like to update these examples to use the `text-generation` task instead?~ nvm i see you've already opened a pr!

--- Comment by math-hiyoko at 2026-03-09T15:36:42Z ---
Am I correct that encoder–decoder models like T5 cannot be used with the text-generation task?

--- Comment by stevhliu at 2026-03-09T15:38:52Z ---
yeah thats correct, feel free to ignore my comment above :)


ISSUE #44496: ValueError: Unrecognized model in allenai/Olmo-Hybrid-Instruct-SFT-7B. Should have a `model_type` key in its config.json.
State: closed  |  Labels: bug
Author: xenova  |  Created: 2026-03-06T15:17:24Z

### System Info

- `transformers` version: 5.3.0.dev0 (main)
- Platform: macOS-15.7.4-arm64-arm-64bit-Mach-O
- Python version: 3.13.2
- Huggingface_hub version: 1.3.1
- Safetensors version: 0.5.3
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0 (NA)
- Using distributed or parallel set-up in script?: no

### Who can help?

@CyrilVallez

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```sh
>>> from transformers import AutoConfig
>>> AutoConfig.from_pretrained('allenai/Olmo-Hybrid-Instruct-SFT-7B')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    AutoConfig.from_pretrained('allenai/Olmo-Hybrid-Instruct-SFT-7B')
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../transformers/src/transformers/models/auto/configuration_auto.py", line 1471, in from_pretrained
    raise ValueError(
    ...<2 lines>...
    )
ValueError: Unrecognized model in allenai/Olmo-Hybrid-Instruct-SFT-7B. Should have a `model_type` key in its config.json.
>>>
```

However, model_type is indeed set in config.json https://huggingface.co/allenai/Olmo-Hybrid-Instruct-SFT-7B/blob/main/config.json

### Expected behavior

The config should load as expected

--- Comment by xenova at 2026-03-06T15:21:16Z ---
I added a few debug print lines, and config_dict is resolved as `{}` for some reason.
even tried refreshing cache. no luck.

--- Comment by jasiecky at 2026-03-07T09:13:51Z ---
Hey, I’ll start working on it!

--- Comment by Prachi-kushwaha at 2026-03-07T22:12:58Z ---
> I added a few debug print lines, and config_dict is resolved as `{}` for some reason. even tried refreshing cache. no luck.

I tried reproducing this but the config loads correctly with transformers==5.3.0 on my side. It might be due to using an older development version (5.3.0.dev0). Updating transformers should resolve the issue.

--- Comment by xenova at 2026-03-08T23:07:06Z ---
Hmm how strange... running again (still on same 5.3.0.dev) seems to work 🤷
Will close :)


ISSUE #44493: Many models started showing UNEXPECTED Key with position id
State: closed  |  Labels: bug
Author: weathon  |  Created: 2026-03-06T12:06:44Z

### System Info

- `transformers` version: 5.0.0
- Platform: Linux-6.6.113+-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 1.5.0
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0+cu128 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100-SXM4-40GB

### Who can help?

Many models, including `google/owlvit-base-patch32` and a few LLMs started showing unexpecting keys. For example, I got this

```
import torch
from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from transformers.image_utils import load_image

# Prepare processor and model
# model_id = "google/owlv2-large-patch14-ensemble"
model_id = "google/owlvit-base-patch32"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)
```
```
OwlViTForObjectDetection LOAD REPORT from: google/owlvit-base-patch32
Key                                         | Status     |  |
--------------------------------------------+------------+--+-
owlvit.text_model.embeddings.position_ids   | UNEXPECTED |  |
owlvit.vision_model.embeddings.position_ids | UNEXPECTED |  |

Notes:
- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
```

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Loading some models that could trigger this, example code as below
```
import torch
from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from transformers.image_utils import load_image

# Prepare processor and model
# model_id = "google/owlv2-large-patch14-ensemble"
model_id = "google/owlvit-base-patch32"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)
```

### Expected behavior

It should load correctly with no unexpected keys as they are the same unmodified model

--- Comment by Rocketknight1 at 2026-03-06T13:57:45Z ---
This is annoying, but it thankfully shouldn't affect model performance! I believe what's happening here is that owlvit used to save `self.position_ids` as a buffer, but this was not the position **embeddings**, it was just an integer range tensor from 0 to the max sequence length. Obviously, there's not much point in saving this in the checkpoints, since it can easily be recomputed on the fly.

At some point the model was likely updated to recompute the key, but then we were left with these extra keys in the checkpoints and the loader is complaining about them. The right solution is to add those keys to `keys_to_ignore_on_load_unexpected` for `owlvit` to suppress those warnings - would you like to make a PR?

--- Comment by KartikPawade at 2026-03-06T18:52:15Z ---
Hi @Rocketknight1 , I’ve opened a PR to address this by adding the corresponding patterns to `_keys_to_ignore_on_load_unexpected` in `OwlViTPreTrainedModel`.

PR: https://github.com/huggingface/transformers/pull/44508

Please let me know if any adjustments are needed. Thanks!


Typo in Cache strategies

ISSUE #44492: Typo in Cache strategies
State: closed  |  Labels: bug
Author: aryan221102  |  Created: 2026-03-06T12:00:54Z

### System Info

I’m sorry if I am writing this issue in a wrong subsection but your chatbot let me here. The document Cache Strategies, https://huggingface.co/docs/transformers/kv_cache , says that the JIT maximizes latency while it should “minimizes” the latency .
I basically just greaduated, start of my career so please don’t judge,
Happy to help.

### Who can help?

@stevhliu

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

#it is just the documentation

### Expected behavior

Just fix the typo.

--- Comment by stevhliu at 2026-03-06T19:57:40Z ---
good catch, thanks for raising this issue!


ISSUE #44488: Current version also does not load "cjvt/sleng-bert"
State: closed  |  Labels: bug
Author: AngledLuffa  |  Created: 2026-03-06T08:36:44Z

### System Info

broken config:

Python 3.13.5
tokenizers 0.22.2
transformers 5.2.0
torch 2.7.1+cu118

working config:

Python 3.13.5
tokenizers 0.22.1
transformers 4.57.1
torch 2.8.0+cu129

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
>>> from transformers import AutoTokenizer
>>> bert_tokenizer = AutoTokenizer.from_pretrained("cjvt/sleng-bert")
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    bert_tokenizer = AutoTokenizer.from_pretrained("cjvt/sleng-bert")
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/models/auto/tokenization_auto.py", line 749, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/tokenization_utils_base.py", line 1721, in from_pretrained
    return cls._from_pretrained(
           ~~~~~~~~~~~~~~~~~~~~^
        resolved_vocab_files,
        ^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/tokenization_utils_base.py", line 1910, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/models/camembert/tokenization_camembert.py", line 118, in __init__
    unk_index = next((i for i, (tok, _) in enumerate(self._vocab) if tok == str(unk_token)), 0)
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/models/camembert/tokenization_camembert.py", line 118, in <genexpr>
    unk_index = next((i for i, (tok, _) in enumerate(self._vocab) if tok == str(unk_token)), 0)
                               ^^^^^^^^
ValueError: too many values to unpack (expected 2)
```

### Expected behavior

Loading the model would be great!  The older version of transformers works fine.

--- Comment by Rocketknight1 at 2026-03-06T13:47:45Z ---
Looks like a tokenizer issue so cc @arthurzucker @itazap

--- Comment by itazap at 2026-03-06T15:02:56Z ---
The `tokenizer_class` should be updated here to `PreTrainedTokenizerFast` (v4 and v5 compatible)  https://huggingface.co/cjvt/sleng-bert/blob/main/tokenizer_config.json . The `tokenizer.json` file for this model is a BPE, not Unigram like Camembert, so it breaks when trying to load a Unigram `CamembertTokenzier` in v5. It was passing in v4 because we were not enforcing loading the same tokenizer `type`

--- Comment by AngledLuffa at 2026-03-06T16:55:48Z ---
Are you able to make that kind of edit to the tokenizers saved on HF, or does it have to be the model maintainers to make those edits?

There are a couple others with the exact same problem:

```
cjvt/sloberta-sleng
cjvt/sleng-bert
EMBEDDIA/sloberta
```

Thanks!

--- Comment by KartikPawade at 2026-03-06T21:19:09Z ---
Hi @itazap  and @ArthurZucker  👋

I traced through both files carefully and wanted to share what I found.

**Why this is tricky**

Both a legitimate model like `camembert-base` and the broken models (`cjvt/sleng-bert` etc.) have identical routing — both have `model_type: camembert` and `tokenizer_class: CamembertTokenizer` in their configs. So the existing mismatch guard in `from_pretrained` never fires. The only real difference between them is what's inside `tokenizer.json` — `camembert-base` has similar to this `"model": {"type": "Unigram"}`, while `cjvt/sleng-bert` has `"model": {"type": "BPE"}`.

**Proposed fix**

Would it make sense to peek at `tokenizer.json` before routing to `CamembertTokenizer`, and fall back to `TokenizersBackend` if the model type is not `Unigram`? Something like this in the `elif tokenizer_config_class is not None` branch:

```python
if tokenizer_class is not None and tokenizer_class.__name__ == "CamembertTokenizer":
    tokenizer_json_file = cached_file(pretrained_model_name_or_path, "tokenizer.json", **kwargs)
    if tokenizer_json_file is not None:
        with open(tokenizer_json_file, encoding="utf-8") as f:
            tokenizer_json = json.load(f)
        if tokenizer_json.get("model", {}).get("type") != "Unigram":
            if TokenizersBackend is not None:
                return TokenizersBackend.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
```

This would:
- Leave `camembert-base` and all legitimate Unigram CamemBERT models completely untouched ✓
- Transparently reroute `cjvt/sleng-bert`, `cjvt/sloberta-sleng`, and `EMBEDDIA/sloberta` to `TokenizersBackend` ✓
- Not touch `MODELS_WITH_INCORRECT_HUB_TOKENIZER_CLASS` or any other existing logic ✓

Both `cached_file` and `json` are already imported in the file so no new dependencies needed.

Does this approach look reasonable? Happy to raise a PR if so!

--- Comment by itazap at 2026-03-12T17:07:27Z ---
hey @KartikPawade this would be quite the workaround for `CamembertTokenzier` so ideally we solve the problem at the root which is that we don't want to instantiate a `CamembertTokenzier` here

--- Comment by AngledLuffa at 2026-03-16T06:41:07Z ---
Is there some alternate fix which would make this model viable again?

--- Comment by michaelanderson01826-glitch at 2026-03-17T14:15:33Z ---
Might be worth considering an alternative approach

On Mon, Mar 16, 2026, 1:41 AM John Bauer ***@***.***> wrote:

> *AngledLuffa* left a comment (huggingface/transformers#44488)
> <https://github.com/huggingface/transformers/issues/44488#issuecomment-4065433481>
>
> Is there some alternate fix which would make this model viable again?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/44488#issuecomment-4065433481>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/B7HTEEJ6CE7QHWIXLKQZJH34Q6OZ7AVCNFSM6AAAAACWJHOBZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DANRVGQZTGNBYGE>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>

--- Comment by AngledLuffa at 2026-03-19T22:13:23Z ---
wouldn't the simplest thing to do be to edit the model on HF so that it tries to instantiate the correct transformer?  HF has seemed resistant to that kind of change in the past, but it would completely solve the problem for these four models

of course, if there's some other code change that makes it work instead, that would also be excellent

--- Comment by itazap at 2026-03-20T10:27:50Z ---
```python
AutoTokenizer.from_pretrained("cjvt/sloberta-sleng")
```
This uses the `tokenizer.json` to instantiate a `CamembertTokenizer` class, as explicitly specified in the `tokenizer_class` in the `tokenizer_config.json` [here](https://huggingface.co/cjvt/sloberta-sleng/blob/main/tokenizer_config.json). However,  the tokenizer in `tokenizer.json` is not a `CamembertTokenizer` at all, its a BPE model and Camembert is Unigram model : https://github.com/huggingface/transformers/blob/f5e573080ae0838799c1f9a0ba28be8431120b56/src/transformers/models/camembert/tokenization_camembert.py#L119 That is why it errors out. The simplest thing is to change the `tokenizer_class: PreTrainedTokenizerFast`

Alternatively,

```python
 TokenizersBackend.from_pretrained("cjvt/sloberta-sleng")
```
This loads the `tokenizer.json` to instantiate a generic `TokenizersBackend` class, so it doesn't care about any config tokenizer class and would work for the models listed above!

Let me know if this sounds reasonable on your end and a feasible solution 🙏

--- Comment by AngledLuffa at 2026-03-20T20:24:04Z ---
I can do that.  There have been other buggy models on the hub I've had to special case in different ways.

So in other words, don't use `AutoTokenizer` for these models?  Manually updating the .json on my end seems tough since I was not using the .json at all but was just letting the library do the downloading and loading.

What's wild is the old transformers v4 was successfully loading these models.  If it was tokenizing incorrectly, should I just disregard those results and rerun them?

May I ask why editing the model on the hub so it has the correct tokenizer is not an option?  For all we know, I may very well be the only person to ever try using this model to programmatically get Slovenian word embeddings... but if anyone else ever does try, they'll run into the exact same issue.

Or is there something diagnostic about the json which the transformers library can look at to know to make the switch itself?

--- Comment by ArthurZucker at 2026-03-23T08:52:03Z ---
This is typically an textbook example of a model where we cannot "fix" it, and unfortunately the user has to bear with either loading with `TokenizersBackend` or using older `transformers` version.

I'll close this as its a bug fix, and the best is to open a PR on the hub to update this `tokenizer_config.json` as @itazap said!

--- Comment by AngledLuffa at 2026-03-24T06:38:46Z ---
That is fair...  loading with `TokenizersBackend` seems to work in terms of loading the model.

When I load the transformer and tokenizer with the latest version, it returns float16 by default.  Is there a flag to make it return float32?

For that matter, is there a flag I could pass to AutoTokenizer which replaces the tokenizer type in the config with the correct tokenizer type?  `model_type=...` but I don't know what `model_type` to pass.

Thanks!

--- Comment by itazap at 2026-03-24T09:15:01Z ---
you can pass a dtype when loading models:
```python
model = AutoModel.from_pretrained(path, dtype=torch.float32)

```

you can also pass a `tokenizer_type` when loading tokenizers:
```python
tok = AutoTokenizel.from_pretrained(path, tokenizer_type="bert")
# tok would be a BertTokenizer

```

But as for which `model_type`, it depends on your use case !

--- Comment by AngledLuffa at 2026-03-24T10:05:25Z ---
> you can also pass a `tokenizer_type` when loading tokenizers:
>
> ```
> tok = AutoTokenizel.from_pretrained(path, tokenizer_type="bert")
> # tok would be a BertTokenizer
> ```
>
> But as for which `model_type`, it depends on your use case !

Thanks!

Are you able to figure out which tokenizer_type is appropriate for `cjvt/sloberta-sleng`?


ISSUE #44486: KubeflowCallback: Native progress reporting for Kubernetes-based Kubeflow training
State: closed  |  Labels: Feature request
Author: abhijeet-dhumal  |  Created: 2026-03-06T07:07:19Z

### Feature request

Add a `KubeflowCallback` to enable automatic progress and metrics reporting for training jobs running on [Kubeflow Trainer](https://github.com/kubeflow/trainer), the Kubernetes-native platform for distributed AI/ML training.

 **Context:** This is part of a coordinated effort with the Kubeflow community. The controller-side implementation is available in [kubeflow/trainer#3227](https://github.com/kubeflow/trainer/pull/3227) which adds a status server that receives progress updates from training pods and exposes them via the TrainJob CR status. This HuggingFace callback would be the client-side integration that automatically reports progress from Transformers training loops.

When training runs inside a Kubeflow TrainJob, the callback would automatically:
- Report training progress percentage (0-100%)
- Calculate and report estimated time remaining (ETA)
- Push training metrics (loss, accuracy, etc.) to the TrainJob status

This enables real-time visibility into training progress via standard Kubernetes APIs:
```bash
kubectl get trainjob my-llm-finetune -o jsonpath='{.status.trainerStatus}'
# {"progressPercentage": 45, "estimatedRemainingSeconds": 1200, "metrics": [{"name": "loss", "value": "0.234"}]}
```

### Motivation

**Problem:** AI practitioners running HuggingFace training jobs on Kubernetes have no native way to monitor training progress. They must either:
1. Parse container logs manually
2. Set up external tracking systems (MLflow/W&B) which adds infrastructure overhead
3. Wait blindly for jobs to complete

**Why Kubeflow matters:**
- Kubeflow Trainer is the standard for distributed training on Kubernetes (PyTorch, JAX, MPI, etc.)
- Large organizations (Google, Red Hat, Bloomberg, etc.) use Kubeflow for production ML workloads
- Kubernetes is becoming the default platform for LLM fine-tuning at scale

**User experience improvement:**

Before (no visibility):
```bash
kubectl get trainjob my-job
# NAME     STATUS    AGE
# my-job   Running   2h
# (Is it 10% done? 90% done? No idea.)
```

After (with KubeflowCallback):
```bash
kubectl get trainjob my-job -o jsonpath='{.status.trainerStatus}'
# {"progressPercentage": 67, "estimatedRemainingSeconds": 3600, "metrics": [{"name": "loss", "value": "0.15"}]}
```

**Zero friction for users:**

No code changes required. The callback auto-activates when the Kubeflow controller injects environment variables:

```python
from transformers import Trainer, TrainingArguments

trainer = Trainer(model=model, args=TrainingArguments(...), train_dataset=ds)
trainer.train()  # Progress automatically reported when running in Kubeflow
```

**Synergy with existing integrations:**

This complements (not replaces) MLflow/W&B. Users can run both:
- `KubeflowCallback` for Kubernetes-native observability (kubectl, dashboards, alerting)
- `MLflowCallback` or `WandbCallback` for experiment tracking and artifact management

### Your contribution

**Yes, I am willing to submit a PR for this feature.**

I am a contributor of [Kubeflow Trainer](https://github.com/kubeflow/trainer) and [Kubeflow SDK](https://github.com/kubeflow/sdk) and have been working on the progress tracking feature (KEP-2779) alongside [Rob Bell](https://github.com/robert-bell). The implementation plan is finalized and the controller-side PR is in review here [kubeflow/trainer#3227](https://github.com/kubeflow/trainer/pull/3227)

--- Comment by abhijeet-dhumal at 2026-03-06T07:07:50Z ---
/assign


ISSUE #44485: [Bug/Discusion] GLM-5 RoPE Implementation
State: closed  |  Labels: bug
Author: Jintao-Huang  |  Created: 2026-03-06T06:04:18Z

### System Info

https://huggingface.co/zai-org/GLM-5/blob/main/config.json#L45

hi! I see that the rope setting here is rope_interleave true.

However, looking at the transformers implementation, the logic here is false, refer to these

https://github.com/huggingface/transformers/blob/d5e555a632682555332c3c8e938461efd49d52b9/src/transformers/models/glm_moe_dsa/modular_glm_moe_dsa.py#L46-L74

https://github.com/huggingface/transformers/blob/d5e555a632682555332c3c8e938461efd49d52b9/src/transformers/models/deepseek_v3/modular_deepseek_v3.py#L47-L82

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

-

--- Comment by Jintao-Huang at 2026-03-06T06:23:08Z ---
vllm implementation:

https://github.com/jeejeelee/vllm/blob/2b637c60198c4df068d0f980a978c3fc8853d745/vllm/model_executor/models/deepseek_v2.py#L816-L839

I'm confused...

--- Comment by JaredforReal at 2026-03-10T09:58:04Z ---
@Jintao-Huang Sorry for the confusion! Although the config.yaml says the `rope_interleave` is `true`, we should NOT it practically like what we did in vllm and sglang to make it work correctly.
https://github.com/vllm-project/vllm/blob/ddbb0d230a3592106ac9f5f7f4e9a861863fcbee/vllm/model_executor/models/deepseek_v2.py#L924-L930
https://github.com/sgl-project/sglang/blob/51d9d34977582dc565c784e4435cf23cf3ea0640/python/sglang/srt/models/deepseek_v2.py#L1143-L1162

--- Comment by github-actions[bot] at 2026-04-05T08:08:29Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44484: Why max_shard_size in PreTrainedModel.save_pretrained() is 50GB?
State: closed  |  Labels: bug
Author: Silencezong  |  Created: 2026-03-06T02:51:48Z

### System Info

In old version like 4.57.1, max_shard_size in PreTrainedModel.save_pretrained() is 5GB, but in new version, max_shard_size is '50GB'. Is it normal?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

New version:
def save_pretrained(
        self,
        save_directory: str | os.PathLike,
        is_main_process: bool = True,
        state_dict: dict | None = None,
        push_to_hub: bool = False,
        max_shard_size: int | str = "50GB",
        variant: str | None = None,
        token: str | bool | None = None,
        save_peft_format: bool = True,
        save_original_format: bool = True,
        **kwargs,
    ):

Old version:
    def save_pretrained(
        self,
        save_directory: Union[str, os.PathLike],
        is_main_process: bool = True,
        state_dict: Optional[dict] = None,
        save_function: Callable = torch.save,
        push_to_hub: bool = False,
        max_shard_size: Union[int, str] = "5GB",
        safe_serialization: bool = True,
        variant: Optional[str] = None,
        token: Optional[Union[str, bool]] = None,
        save_peft_format: bool = True,
        **kwargs,
    ):

### Expected behavior

Fix this bug?

--- Comment by Rocketknight1 at 2026-03-06T13:34:27Z ---
It's not a bug! We updated the default to stop large models by default being sharded into hundreds of files.


ISSUE #44483: [Critical  v5.3] /v1/chat/completions would not accept request as usual
State: closed  |  Labels: bug
Author: zhangwei217245  |  Created: 2026-03-06T02:39:54Z

### System Info

Linux

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run `transformers serve` in the terminal, and then issue request:

curl -sS -X POST 'http://localhost:8000/v1/chat/completions' \
  -H 'Content-Type: application/json' \
  -d '{"model":"LiquidAI/LFM2.5-VL-1.6B","stream":true,"messages":[{"role":"user","content":[{"type":"text","text":"What is in this image?"},{"type":"image_url","image_url":{"url":"https://natureconservancy-h.assetsadobe.com/is/image/content/dam/tnc/nature/en/photos/b/l/Blue-jay-Matt-Williams.jpg?crop=0%2C11%2C4000%2C3000&wid=400&hei=300&scl=10.0"}}]}],"max_tokens":300}'

Response:

{"detail":[{"type":"missing","loc":["query","request"],"msg":"Field required","input":null}]}

### Expected behavior

it should provide generated result

--- Comment by zhangwei217245 at 2026-03-06T03:08:28Z ---
The same thing happened to /v1/audio/transcriptions

--- Comment by lang216 at 2026-03-08T03:22:58Z ---
second this! For me it's the Qwen 3.5 series failing.

--- Comment by Rocketknight1 at 2026-03-09T13:20:58Z ---
cc @lysandrejik for `transformers serve`!

--- Comment by LysandreJik at 2026-03-16T05:26:18Z ---
It should work on the latest `main`! I just merged this which should fix it: https://github.com/huggingface/transformers/pull/44620


ISSUE #44479: [`bug`] v5.3.0 video input regression for `qwen2_5_vl`, `qwen3_vl`, `qwen3_5`, and `qwen3_5_moe`
State: closed  |  Labels: bug
Author: tomaarsen  |  Created: 2026-03-05T18:47:53Z

### System Info

- `transformers` version: 5.3.0.dev0
- Platform: Windows-10-10.0.26200-SP0
- Python version: 3.11.6
- Huggingface_hub version: 1.5.0
- Safetensors version: 0.6.2
- Accelerate version: 1.13.0.dev0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0+cu128 (CUDA)
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No (issue persists with GPU and CPU)
- GPU type: NVIDIA GeForce RTX 3090

### Who can help?

@zucchini-nlp

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Here is a simple, tiny reproducer using a `qwen3_vl` tiny-random model:

```python
from transformers import AutoModel, AutoProcessor

model_name = "tiny-random/qwen3-vl"
model = AutoModel.from_pretrained(model_name)
processor = AutoProcessor.from_pretrained(model_name)

messages = [
    [
        {
            "role": "user",
            "content": [
                {
                    "type": "video",
                    "video": "https://huggingface.co/datasets/tomaarsen/tiny-test/resolve/main/tiny_test_video_0.mp4",
                }
            ],
        }
    ],
    [
        {
            "role": "user",
            "content": [
                {
                    "type": "video",
                    "video": "https://huggingface.co/datasets/tomaarsen/tiny-test/resolve/main/tiny_test_video_1.mp4",
                }
            ],
        }
    ],
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
)
outputs = model(**inputs)
print(outputs)
```
```
Traceback (most recent call last):
  File "[sic]/sentence-transformers/demo_qwen_transformers.py", line 38, in <module>
    outputs = model(**inputs)
              ^^^^^^^^^^^^^^^
  File "[sic]/sentence-transformers/Lib/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/sentence-transformers/Lib/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/src/transformers/utils/generic.py", line 843, in wrapper
    output = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 1348, in forward
    position_ids = self.compute_3d_position_ids(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 1242, in compute_3d_position_ids
    position_ids, rope_deltas = self.get_rope_index(
                                ^^^^^^^^^^^^^^^^^^^^
  File "[sic]/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 1134, in get_rope_index
    position_ids[:, batch_idx, attention_mask[batch_idx].bool()] = llm_positions.to(position_ids.device)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: shape mismatch: value tensor of shape [3, 25] cannot be broadcast to indexing result of shape [3, 23]
```

These are the relevant lines where it goes wrong (for `qwen3_vl`): https://github.com/huggingface/transformers/blob/e498b5bd273e638990cfc82d8c2177a9c5b67858/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py#L1121-L1122

The `attention_mask` and `position_ids` are correctly shaped (e.g. `torch.Size([2, 23])` and `torch.Size([3, 2, 23])`), but `llm_positions` is consistently 2 tokens too large.

### Expected behavior

I would expect this to work as it did in v5.2.0. This is currently preventing inference on the VL-capable Qwen model with video inputs.

- Tom Aarsen

--- Comment by zucchini-nlp at 2026-03-06T08:27:28Z ---
Fixed in https://github.com/huggingface/transformers/pull/44474

--- Comment by tomaarsen at 2026-03-06T08:33:50Z ---
Damn, I only searched issues, not PRs 😅

- Tom Aarsen

--- Comment by zucchini-nlp at 2026-03-10T09:57:36Z ---
Merged the fix, closing as resolved!


ISSUE #44466: [v5] Inconsistent serialization of `lm_head.weight` (tied weights?) depending on model device in v5/`main`, while v4.57 behaves correctly
State: closed  |  Labels: bug
Author: fxmarty-amd  |  Created: 2026-03-05T13:26:59Z

### System Info

```
- `transformers` version: 5.3.0.dev0
- Platform: Linux-6.8.0-100-generic-x86_64-with-glibc2.39
- Python version: 3.12.12
- Huggingface_hub version: 1.5.0
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1+rocm6.4 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: AMD Instinct MI300X
```

and `4.57.6` for testing.

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi,

As mentioned in the title, it looks to me something is broken in the serialization of tied weights in Transformers v5 (tried both v5.2 and current `main` @ 81b53436bf97b8cb02040839a902e0b799be3159).

When moving a model to cuda device, `lm_head.weight` gets serialized while it does not when staying on CPU. This is inconsistent with v4.57 behavior (consistent serialization on all devices) and looks strange to me.

What do you think?

```python
from transformers import AutoModelForCausalLM
from safetensors import safe_open
from pathlib import Path
from packaging import version
import transformers

model = AutoModelForCausalLM.from_pretrained("facebook/opt-125m")

move_to_device = False  # toggle this flag.
if move_to_device:
    print("model to cuda")
    model.eval()
    model = model.to("cuda")

model.save_pretrained("opt-125m-saved")

with safe_open(Path("opt-125m-saved", "model.safetensors"), framework="pt") as f:
    checkpoint_keys = f.keys()

    for name in checkpoint_keys:
        print("checkpoint key", name)

    if move_to_device:
        if version.parse(transformers.__version__) >= version.parse("5.0"):
            # NOTE: this looks wrong / to be a regression to me.
            assert "lm_head.weight" in checkpoint_keys
        else:
            # NOTE: transformers<5 behavior looks to be the correct one to me.
            assert "lm_head.weight" not in checkpoint_keys
    else:
        assert "lm_head.weight" not in checkpoint_keys
```

### Expected behavior

`model.safetensors` after `save_pretrained` should contain the same tensor keys independently of the device the `PretrainedModel` is placed on.

It used to be the case with v4.57.

It is not anymore the case with v5.

--- Comment by Rocketknight1 at 2026-03-05T13:45:22Z ---
cc @cyrilvallez for this one I think!

--- Comment by Cyrilvallez at 2026-03-06T16:14:12Z ---
Hey @fxmarty-amd! Thanks for the report! This one is quite a peculiar edge case 🙂
What's happening here is that since recently, when we encounter both weights in the checkpoints, we do NOT re-tie them, and assume the config was wrongly set. That's why you get the following warning when loading:

```
The tied weights mapping and config for this model specifies to tie model.decoder.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
```

So the weights are not actually tied, i.e.

```python
model.lm_head.weight is model.model.decoder.embed_tokens.weight
>>> False
```

but since they were loaded from torch checkpoints (and not safetensors), the checkpoints contains both keys mapping to the same storage, i.e.

```python
model.lm_head.weight.data_ptr() == model.model.decoder.embed_tokens.weight.data_ptr()
>>> True
```

and when we try to save them, since we check the pointers, they are considered tied. When moved to a device, they loose their similar pointer as they are not supposed to be tied -> thus the inconsistency.

I've been considering adding an equality check to still tie weights if both keys are present in the checkpoints and weights are the same, so I'll do it now! It's true that we overlooked that with older torch `.bin` checkpoints, both keys will ALWAYS be present!

--- Comment by fxmarty-amd at 2026-03-06T17:42:30Z ---
@Cyrilvallez thanks a lot! Any of the two behavior (serialize/not serialize lm_head in checkpoint) is fine to me, but it is great if this is independent of the device! Thanks a lot for the fix!


ISSUE #44464: Chunked generation produces inconsistent outputs when using compiled forward
State: closed  |  Labels: bug
Author: mrTsjolder  |  Created: 2026-03-05T12:50:02Z

### System Info

- `transformers` version: 4.57.6
- Platform: Linux-6.18.13-arch1-1-x86_64-with-glibc2.43
- Python version: 3.12.12
- Huggingface_hub version: 0.36.0
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA GeForce RTX 2050

### Who can help?

@Cyrilvallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Change `disable_compile` to `False` to make the second assert fail.

```python
import torch
from transformers.models import LlamaForCausalLM, LlamaConfig
from transformers.generation.configuration_utils import GenerationConfig
from transformers.cache_utils import StaticCache

torch.manual_seed(2026)
model = LlamaForCausalLM(LlamaConfig(num_hidden_layers=1)).to(device="cuda", dtype=torch.bfloat16)
cache = StaticCache(model.config, max_cache_len=192)
tokens = torch.randint(model.vocab_size, size=(4, 64), device=model.device)

gen_kwargs = {
    "eos_token_id": 128001,
    "bos_token_id": 128000,
    "do_sample": False,
    "disable_compile": True,
}

gen_config = GenerationConfig(max_new_tokens=128, min_new_tokens=128, **gen_kwargs)
cache.reset()
out_ref = model.generate(tokens, past_key_values=cache, generation_config=gen_config)

gen_config2 = GenerationConfig(max_new_tokens=64, min_new_tokens=64, **gen_kwargs)
cache.reset()
out1 = model.generate(tokens, past_key_values=cache, generation_config=gen_config2)
out2 = model.generate(
    out1[:, -1:],
    past_key_values=cache,
    cache_position=torch.tensor([cache.get_seq_length()], device=model.device),
    attention_mask=torch.ones_like(out1),  # necessary for correct pos_ids
    generation_config=gen_config2,
)

L = out1.shape[1]
torch.testing.assert_close(out1, out_ref[:, :L])
torch.testing.assert_close(out2[:, 1:], out_ref[:, L:])
```

### Expected behavior

I expect results from a single call to `generate` to produce the same results as repeated `generate` calls (with appropriate arguments) when doing greedy decoding, independent of whether model calls are being compiled or not.

--- Comment by Rocketknight1 at 2026-03-05T13:49:04Z ---
Hi @mrTsjolder, when you're using a randomly initialized model with `bfloat16` precision, it's likely the output logits for multiple tokens are very close, and small differences in the order of operations, or switching between eager and compiled execution, could change which one is the maximum. Can you check the logit values, or try a pretrained checkpoint rather than a randomly-initialized one?

--- Comment by Cyrilvallez at 2026-03-06T16:49:58Z ---
Well, this should work and it does! I run your snippet and all checks pass 🤗 If you still have difficulties, please upgrade to latest, I've recently worked on `generate` which may have solved a few old bugs on restarting generation from existing cache!


ISSUE #44462: AutoTokenizer ignores tokenizer.json from the repository
State: closed  |  Labels: bug
Author: apaniukov  |  Created: 2026-03-05T12:04:31Z

### System Info

- `transformers` version: 5.3.0
- Python version: 3.10.12
- Huggingface_hub version: 1.5.0

### Who can help?

@ArthurZucker and @itazap

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`AutoTokenizer` doesn't load tokenizer based on `tokenizer.json` from the repository. Saving this tokenizer produces different `tokenizer.json` file.

```python
from transformers import AutoTokenizer

hf_tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-coder-6.7b-instruct")
hf_tokenizer.save_pretrained("hf_deepseek_tokenizer/")
```
[tokenizer_original.json](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct/raw/main/tokenizer.json)
[tokenizer_saved.json](https://github.com/user-attachments/files/25766810/tokenizer_saved.json)

Original normalizer/pre-tokenizer:
```json
  "normalizer": {
    "type": "Sequence",
    "normalizers": []
  },
  "pre_tokenizer": {
    "type": "Sequence",
    "pretokenizers": [
      {
        "type": "Split",
        "pattern": {
          "Regex": "[\r\n]"
        },
        "behavior": "Isolated",
        "invert": false
      },
      {
        "type": "Split",
        "pattern": {
          "Regex": "\\s?\\p{L}+"
        },
        "behavior": "Isolated",
        "invert": false
      },
      {
        "type": "Split",
        "pattern": {
          "Regex": "\\s?\\p{P}+"
        },
        "behavior": "Isolated",
        "invert": false
      },
      {
        "type": "Split",
        "pattern": {
          "Regex": "[一-龥ࠀ-一가-퟿]+"
        },
        "behavior": "Isolated",
        "invert": false
      },
      {
        "type": "Digits",
        "individual_digits": true
      },
      {
        "type": "ByteLevel",
        "add_prefix_space": false,
        "trim_offsets": true,
        "use_regex": false
      }
    ]
  }
```
Saved normalizer/pre-tokenizer:
```json
  "normalizer": null,
  "pre_tokenizer": {
    "type": "Metaspace",
    "replacement": "▁",
    "prepend_scheme": "always",
    "split": false
  },
```

### Expected behavior

Original `tokenizer.json` should be used to instantiate the tokenizer.

--- Comment by apaniukov at 2026-03-06T10:19:03Z ---
I am also wonder how does this tokenizer even work, tokenizer's vocab is byte-level encoded, but the instantiated tokenizer lacks `ByteLevel` pre-tokenizer step.

--- Comment by itazap at 2026-03-12T17:20:20Z ---
hey! Thanks for raising, we recently made some fixes for this and I can't reproduce this issue on latest. Please let me know if you still have issues on main - latest! 😊

--- Comment by apaniukov at 2026-03-20T12:37:12Z ---
Hi @itazap,
Yes, the issue is fixed on the main branch, thanks. Looking forward to the next release!


ISSUE #44458: Mllama compile failed after new attn mask
State: closed  |  Labels: bug
Author: jiqing-feng  |  Created: 2026-03-05T07:33:29Z

### System Info

torch                     2.10.0+cpu

regression PR: #42848

### Who can help?

@vasqu

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import requests
import torch
from PIL import Image
from transformers import MllamaForConditionalGeneration, AutoProcessor

model_id = "meta-llama/Llama-3.2-11B-Vision-Instruct"

model = MllamaForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="cpu",
)
processor = AutoProcessor.from_pretrained(model_id)

# apply torch compile
model.forward = torch.compile(model.forward)

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
image = Image.open(requests.get(url, stream=True).raw)

messages = [
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": "If I had to write a haiku for this one, it would be: "}
    ]}
]
input_text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(
    image,
    input_text,
    add_special_tokens=False,
    return_tensors="pt"
).to(model.device)

output = model.generate(**inputs, max_new_tokens=30)
print(processor.decode(output[0]))
```

### Expected behavior

output before regression:
```
Loading weights: 100%|█| 906/906 [00:00<00:00, 1756.16it/s, Materializing param=model.vision_model.transformer.layers.31.self_attn.v_
The image processor of type `MllamaImageProcessor` is now loaded as a fast processor by default, even if the model checkpoint was sav
ed with a slow processor. This is a breaking change and may produce slightly different outputs. To continue using the slow processor,
 instantiate this class with `use_fast=False`.
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8] torch._dynamo hit config.recompile_limit (8)
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8]    function: '__call__' (/home/jiqing/transformers/src/trans
formers/modeling_layers.py:59)
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8]    last reason: 8/3: ___check_type_id(self, 787516432), type
=<class 'transformers.models.mllama.modeling_mllama.MllamaCrossAttentionDecoderLayer'>  # if self.gradient_checkpointing and self.tra
ining:  # ome/jiqing/transformers/src/transformers/modeling_layers.py:60 in __call__ (HINT: type MllamaCrossAttentionDecoderLayer)
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8] User stack trace:
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8]   File "/home/jiqing/transformers/src/transformers/modeling_
layers.py", line 60, in __call__
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8]     if self.gradient_checkpointing and self.training:
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8] To log all recompilation reasons, use TORCH_LOGS="recompiles
".
W0305 07:24:38.959000 1936280 torch/_dynamo/convert_frame.py:1767] [8/8] To diagnose recompilation issues, see https://docs.pytorch.o
rg/docs/main/user_guide/torch_compiler/compile/programming_model.recompilation.html                                                  <|begin_of_text|><|start_header_id|>user<|end_header_id|>

<|image|>If I had to write a haiku for this one, it would be: <|eot_id|><|start_header_id|>assistant<|end_header_id|>
                                                                                Here is a haiku for the image:

A rabbit in a coat
Stands on a dirt path, smiling
Springtime's gentle charm<|eot_id|>
```

output after regression:
```
W0305 06:53:48.493000 1917780 torch/_dynamo/convert_frame.py:1676] [8/8] torch._dynamo hit config.recompile_limit (8)
W0305 06:53:48.493000 1917780 torch/_dynamo/convert_frame.py:1676] [8/8]    function: '__call__' (/home/jiqing/transformers/src/trans
formers/modeling_layers.py:59)
W0305 06:53:48.493000 1917780 torch/_dynamo/convert_frame.py:1676] [8/8]    last reason: 8/3: ___check_type_id(self, 1250368528), typ
e=<class 'transformers.models.mllama.modeling_mllama.MllamaCrossAttentionDecoderLayer'>  # if self.gradient_checkpointing and self.tr
aining:  # ome/jiqing/transformers/src/transformers/modeling_layers.py:60 in __call__
W0305 06:53:48.493000 1917780 torch/_dynamo/convert_frame.py:1676] [8/8] To log all recompilation reasons, use TORCH_LOGS="recompiles
".
W0305 06:53:48.493000 1917780 torch/_dynamo/convert_frame.py:1676] [8/8] To diagnose recompilation issues, see https://pytorch.org/do
cs/main/compile/programming_model.recompilation.html
Traceback (most recent call last):
  File "/home/jiqing/test_llama_vision.py", line 35, in <module>
    output = model.generate(**inputs, max_new_tokens=30)                                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiqing/transformers/src/transformers/generation/utils.py", line 2555, in generate
    result = decoding_method(
             ^^^^^^^^^^^^^^^^
  File "/home/jiqing/transformers/src/transformers/generation/utils.py", line 2762, in _sample
    outputs = model_forward(**model_inputs, return_dict=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/_dynamo/eval_frame.py", line 967, in compile_wrapper
    raise e.remove_dynamo_frames() from None  # see TORCHDYNAMO_VERBOSE=1
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1019, in _compile_fx_inner
    raise InductorError(e, currentframe()).with_traceback(
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1003, in _compile_fx_inner
    mb_compiled_graph = fx_codegen_and_compile(
                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1766, in fx_codegen_and_compile
    return scheme.codegen_and_compile(gm, example_inputs, inputs_to_check, graph_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1537, in codegen_and_compile
    compiled_module = graph.compile_to_module()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/graph.py", line 2416, in compile_to_module
    return self._compile_to_module()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/graph.py", line 2426, in _compile_to_module
    mod = self._compile_to_module_lines(wrapper_code)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/graph.py", line 2501, in _compile_to_module_lines
.......
......
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 2966, in _worker_compile_cpp
    builder.build()
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/cpp_builder.py", line 2144, in build
    run_compile_cmd(build_cmd, cwd=_build_tmp_dir)
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/cpp_builder.py", line 636, in run_compile_cmd
    _run_compile_cmd(cmd_line, cwd)
  File "/opt/.venv/lib/python3.12/site-packages/torch/_inductor/cpp_builder.py", line 631, in _run_compile_cmd
    raise exc.CppCompileError(cmd, output) from e
torch._inductor.exc.InductorError: CppCompileError: C++ compile error

Command:
g++ /tmp/torchinductor_root/6q/c6qt5khkam3fycdonzojkruxp6xbm67j4hobbgxub2kjtsoqojma.main.cpp -D TORCH_INDUCTOR_CPP_WRAPPER -D STANDAL
ONE_TORCH_HEADER -D C10_USING_CUSTOM_GENERATED_MACROS -D CPU_CAPABILITY_AVX512 -O3 -DNDEBUG -fno-trapping-math -funsafe-math-optimiza
tions -ffinite-math-only -fno-signed-zeros -fno-math-errno -fno-finite-math-only -fno-unsafe-math-optimizations -ffp-contract=off -fe
xcess-precision=fast -fno-tree-loop-vectorize -march=native -shared -fPIC -Wall -std=c++17 -Wno-unused-variable -Wno-unknown-pragmas
-pedantic -fopenmp -include /tmp/torchinductor_root/precompiled_headers/cimseuvkhk6u5tg72hhkbkur6zutyynuqzqik7rx7nziiylz223c.h -I/roo
t/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/include/python3.12 -I/opt/.venv/lib/python3.12/site-packages/torch/include
-I/opt/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -mavx512f -mavx512dq -mavx512vl -mavx512bw -mfma -mavx
512vnni -mavx512vl -mamx-tile -mamx-bf16 -mamx-int8 -mavx512bf16 -mamx-fp16 -o /tmp/torchinductor_root/6q/c6qt5khkam3fycdonzojkruxp6x
bm67j4hobbgxub2kjtsoqojma.main.so -ltorch -ltorch_cpu -ltorch_python -lgomp -L/root/.local/share/uv/python/cpython-3.12.12-linux-x86_
64-gnu/lib -L/opt/.venv/lib/python3.12/site-packages/torch/lib

...
/tmp/torchinductor_root/6q/c6qt5khkam3fycdonzojkruxp6xbm67j4hobbgxub2kjtsoqojma.main.cpp: In function ‘void kernel(const int64_t*, bo
ol*, int64_t, int64_t)’:
/tmp/torchinductor_root/6q/c6qt5khkam3fycdonzojkruxp6xbm67j4hobbgxub2kjtsoqojma.main.cpp:14:62: error: ‘tmp2’ was not declared in thi
s scope; did you mean ‘tm’?
   14 |                     TORCH_CHECK((at::vec::VecMask<int64_t,2>(tmp2 < at::vec::VectorizedN<int64_t,2>(ks1))).all_masked(), "ind
ex out of bounds: tmp2 < ks1");
      |                                                              ^~~~
/opt/.venv/lib/python3.12/site-packages/torch/include/torch/headeronly/macros/Macros.h:202:64: note: in definition of macro ‘C10_UNLI
KELY’
  202 | #define C10_UNLIKELY(expr) (__builtin_expect(static_cast<bool>(expr), 0))
      |                                                                ^~~~
/opt/.venv/lib/python3.12/site-packages/torch/include/c10/util/Exception.h:566:7: note: in expansion of macro ‘C10_UNLIKELY_OR_CONST’
  566 |   if (C10_UNLIKELY_OR_CONST(!(cond))) {       \
      |       ^~~~~~~~~~~~~~~~~~~~~
/tmp/torchinductor_root/6q/c6qt5khkam3fycdonzojkruxp6xbm67j4hobbgxub2kjtsoqojma.main.cpp:14:21: note: in expansion of macro ‘TORCH_CH
ECK’
   14 |                     TORCH_CHECK((at::vec::VecMask<int64_t,2>(tmp2 < at::vec::VectorizedN<int64_t,2>(ks1))).all_masked(), "ind
ex out of bounds: tmp2 < ks1");
      |                     ^~~~~~~~~~~
/tmp/torchinductor_root/6q/c6qt5khkam3fycdonzojkruxp6xbm67j4hobbgxub2kjtsoqojma.main.cpp:36:134: error: ‘tmp2’ was not declared in th
is scope; did you mean ‘tm’?
...
...
/opt/.venv/lib/python3.12/site-packages/torch/include/ATen/cpu/vec/vec512/vec512_int.h:1866:7: warning: overflow in conversion from ‘
int’ to ‘char’ changes value from ‘128’ to ‘-128’ [-Woverflow]
 1866 |       0x80,
      |       ^~~~
/opt/.venv/lib/python3.12/site-packages/torch/include/ATen/cpu/vec/vec512/vec512_int.h:1868:7: warning: overflow in conversion from ‘
int’ to ‘char’ changes value from ‘128’ to ‘-128’ [-Woverflow]
 1868 |       0x80,
      |       ^~~~
/opt/.venv/lib/python3.12/site-packages/torch/include/ATen/cpu/vec/vec512/vec512_int.h:1870:7: warning: overflow in conversion from ‘
int’ to ‘char’ changes value from ‘128’ to ‘-128’ [-Woverflow]
 1870 |       0x80,
      |       ^~~~
...
...
```

--- Comment by jiqing-feng at 2026-03-13T02:23:21Z ---
Hi @SunMarc . Would you please check this issue? The regression change is too big; I am afraid I cannot fix it properly.

--- Comment by michaelanderson01826-glitch at 2026-03-13T11:57:32Z ---
Yes alright I will check it for you

On Fri, Mar 13, 2026, 2:23 AM jiqing-feng ***@***.***> wrote:

> *jiqing-feng* left a comment (huggingface/transformers#44458)
> <https://github.com/huggingface/transformers/issues/44458#issuecomment-4051895223>
>
> Hi @SunMarc <https://github.com/SunMarc> . Would you please check this
> issue? The regression change is too big; I am afraid I cannot fix it
> properly.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/44458#issuecomment-4051895223>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/B7HTEEPK56LRLYTMRYXMS7T4QNWLLAVCNFSM6AAAAACWHUVT3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DANJRHA4TKMRSGM>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>

--- Comment by jiqing-feng at 2026-03-19T05:37:51Z ---
Hi @michaelanderson01826-glitch  Please let me know if there is any progress. Thanks!
cc @SunMarc

--- Comment by github-actions[bot] at 2026-04-12T08:13:57Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


LORA权重合并保存在本地之后重新加载出来，两个模型输出结果不一致

ISSUE #44457: LORA权重合并保存在本地之后重新加载出来，两个模型输出结果不一致
State: closed  |  Labels: bug
Author: fish-kong  |  Created: 2026-03-05T06:40:39Z

### System Info

Debian GNU/Linux 12 (bookworm)+5090-32G

```
absl-py                           2.3.1
accelerate                        1.12.0
aiohappyeyeballs                  2.6.1
aiohttp                           3.13.2
aiosignal                         1.4.0
altgraph                          0.17.5
annotated-doc                     0.0.4
annotated-types                   0.7.0
anthropic                         0.71.0
anyio                             4.12.0
apache-tvm-ffi                    0.1.6
astor                             0.8.1
attrs                             25.4.0
auto-round                        0.9.2
blake3                            1.0.8
boto3                             1.42.7
botocore                          1.42.7
cachetools                        6.2.4
cbor2                             5.7.1
certifi                           2025.11.12
cffi                              2.0.0
chardet                           5.2.0
charset-normalizer                3.4.4
click                             8.3.1
cloudpickle                       3.1.2
colorama                          0.4.6
coloredlogs                       15.0.1
compressed-tensors                0.12.2
cryptography                      46.0.3
cuda-bindings                     13.1.1
cuda-pathfinder                   1.3.3
cuda-python                       13.1.1
cupy-cuda12x                      13.6.0
dataproperty                      1.1.0
datasets                          4.4.1
depyf                             0.20.0
dill                              0.4.0
diskcache                         5.6.3
distro                            1.9.0
dnspython                         2.8.0
docstring-parser                  0.17.0
einops                            0.8.1
email-validator                   2.3.0
evaluate                          0.4.6
fastapi                           0.127.0
fastapi-cli                       0.0.16
fastapi-cloud-cli                 0.7.0
fastar                            0.8.0
fastrlock                         0.8.3
filelock                          3.20.0
flashinfer-python                 0.5.3
flatbuffers                       25.9.23
frozenlist                        1.8.0
fschat                            0.2.36
fsspec                            2025.10.0
gguf                              0.17.1
h11                               0.16.0
hf-xet                            1.2.0
httpcore                          1.0.9
httptools                         0.7.1
httpx                             0.28.1
httpx-sse                         0.4.3
huggingface-hub                   0.36.0
humanfriendly                     10.0
idna                              3.11
ijson                             3.4.0.post0
interegular                       0.3.3
jinja2                            3.1.6
jiter                             0.12.0
jmespath                          1.0.1
joblib                            1.5.3
jsonlines                         4.0.0
jsonschema                        4.25.1
jsonschema-specifications         2025.9.1
lark                              1.2.2
latex2mathml                      3.78.1
llguidance                        1.3.0
llmcompressor                     0.9.0
llvmlite                          0.44.0
lm-eval                           0.4.9.2
lm-format-enforcer                0.11.3
loguru                            0.7.3
lxml                              6.0.2
markdown-it-py                    4.0.0
markdown2                         2.5.4
markupsafe                        3.0.3
mbstrdecoder                      1.1.4
mcp                               1.25.0
mdurl                             0.1.2
mistral-common                    1.8.6
model-hosting-container-standards 0.1.12
modelscope                        1.34.0
more-itertools                    10.8.0
mpmath                            1.3.0
msgpack                           1.1.2
msgspec                           0.20.0
multidict                         6.7.0
multiprocess                      0.70.18
networkx                          3.6.1
nh3                               0.3.2
ninja                             1.13.0
nltk                              3.9.2
numba                             0.61.2
numexpr                           2.14.1
numpy                             2.2.6
nvidia-cublas-cu12                12.8.4.1
nvidia-cuda-cupti-cu12            12.8.90
nvidia-cuda-nvrtc-cu12            12.8.93
nvidia-cuda-runtime-cu12          12.8.90
nvidia-cudnn-cu12                 9.10.2.21
nvidia-cudnn-frontend             1.17.0
nvidia-cufft-cu12                 11.3.3.83
nvidia-cufile-cu12                1.13.1.3
nvidia-curand-cu12                10.3.9.90
nvidia-cusolver-cu12              11.7.3.90
nvidia-cusparse-cu12              12.5.8.93
nvidia-cusparselt-cu12            0.7.1
nvidia-cutlass-dsl                4.3.4
nvidia-ml-py                      13.590.44
nvidia-nccl-cu12                  2.27.5
nvidia-nvjitlink-cu12             12.8.93
nvidia-nvshmem-cu12               3.3.20
nvidia-nvtx-cu12                  12.8.90
onnxruntime-gpu                   1.23.2
openai                            2.14.0
openai-harmony                    0.0.8
opencv-python                     4.11.0.86
opencv-python-headless            4.12.0.88
optimum                           2.1.0
outlines-core                     0.2.11
packaging                         25.0
pandas                            2.3.3
partial-json-parser               0.2.1.1.post7
pathvalidate                      3.3.1
peft                              0.18.0
pillow                            12.0.0
platformdirs                      4.5.1
portalocker                       3.2.0
prettytable                       3.17.0
prometheus-client                 0.23.1
prometheus-fastapi-instrumentator 7.1.0
prompt-toolkit                    3.0.52
propcache                         0.4.1
protobuf                          6.33.2
psutil                            7.1.3
py-cpuinfo                        9.0.0
pyarrow                           22.0.0
pybase64                          1.4.3
pybind11                          3.0.1
pyclipper                         1.4.0
pycountry                         24.6.1
pycparser                         2.23
pydantic                          2.12.5
pydantic-core                     2.41.5
pydantic-extra-types              2.10.6
pydantic-settings                 2.12.0
pyecharts                         2.0.9
pygments                          2.19.2
pyinstaller                       6.17.0
pyinstaller-hooks-contrib         2025.10
pyjwt                             2.10.1
pynvml                            13.0.1
pytablewriter                     1.2.1
python-dateutil                   2.9.0.post0
python-dotenv                     1.2.1
python-json-logger                4.0.0
python-multipart                  0.0.21
pytz                              2025.2
pyyaml                            6.0.3
pyzmq                             27.1.0
ray                               2.53.0
referencing                       0.37.0
regex                             2025.11.3
requests                          2.32.5
rich                              13.9.4
rich-toolkit                      0.17.1
rignore                           0.7.6
rouge-score                       0.1.2
rpds-py                           0.30.0
ruff                              0.14.8
s3transfer                        0.16.0
sacrebleu                         2.5.1
safetensors                       0.7.0
scikit-learn                      1.8.0
scipy                             1.16.3
sentencepiece                     0.2.1
sentry-sdk                        2.48.0
setproctitle                      1.3.7
setuptools                        79.0.1
shapely                           2.1.2
shellingham                       1.5.4
shortuuid                         1.0.13
simplejson                        3.20.2
six                               1.17.0
sniffio                           1.3.1
sqlitedict                        2.1.0
sse-starlette                     3.0.4
starlette                         0.50.0
supervisor                        4.3.0
svgwrite                          1.4.3
swanlab                           0.7.3
sympy                             1.14.0
tabledata                         1.3.4
tabulate                          0.9.0
tcolorpy                          0.1.7
threadpoolctl                     3.6.0
tiktoken                          0.12.0
tokenizers                        0.22.1
torch                             2.9.0+cu128
torchaudio                        2.9.0+cu128
torchvision                       0.24.0+cu128
tornado                           6.5.2
tqdm                              4.67.1
tqdm-multiprocess                 0.0.11
transformers                      4.57.3
triton                            3.5.0
typepy                            1.3.4
typer                             0.20.1
typing-extensions                 4.15.0
typing-inspection                 0.4.2
tzdata                            2025.2
urllib3                           2.6.1
uvicorn                           0.40.0
uvloop                            0.22.1
vllm                              0.13.0
watchfiles                        1.1.1
wavedrom                          2.0.3.post3
wcwidth                           0.2.14
websockets                        15.0.1
word2number                       1.1
wrapt                             2.0.1
xformers                          0.0.33.post1
xgrammar                          0.1.27
xxhash                            3.6.0
yarl                              1.22.0
zhconv                            1.4.3
zstandard                         0.25.0
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

对qwen3-4b大模型采用peft+transformers进行lora微调后得到lora权重，将base模型与lora模型合并后的模型为model1，

当model1进行save_pretrained之后，再重新加载模型得到model2

然后，后续利用相同的数据、推理参数与程序进行推理，并在推理程序中do_sample设置为False。

model1与model2的模型加载、合并、保存，重新加载的代码如下：

```
model_path = 'Qwen3-4B path'

checkpoint_path = 'lorapath'

tokenizer = AutoTokenizer.from_pretrained(model_path,padding_side='left',fix_mistral_regex=True)

base_model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16)

model = PeftModel.from_pretrained(base_model, checkpoint_path)

model1 = model.merge_and_unload()

model1 = model1.to(torch.bfloat16)

# 保存合并之后的模型再重新加载出来推理
model1.save_pretrained(save_path, safe_serialization=True)

tokenizer.save_pretrained(save_path)

tokenizer = AutoTokenizer.from_pretrained(save_path,padding_side='left',fix_mistral_regex=True)

model2 = AutoModelForCausalLM.from_pretrained(save_path, torch_dtype=torch.bfloat16)

model = model1 或者model2

model.eval()

#推理的代码里面设置了do_sample=False,且top_k=1

outputs = model.generate(
        **inputs,
        max_new_tokens=10,
        do_sample=False,
        top_k=1)
```

两次运行，同样的数据、同样的推理程序、同样的推理参数，model第一次是model1、第二次是model2，但是输出的结果不一样。

我按照下面的程序打印model1和model2的模型参数差，所有的打印都是：tensor(0., dtype=torch.bfloat16, grad_fn=<MaxBackward1>)，说明两个模型的参数是一致的。

```
for (n1,p1),(n2,p2) in zip(merged_model.named_parameters(), model2.named_parameters()):

	diff = (p1 - p2).abs().max()

	print(diff)
```

请问如何才能保证两个模型的结果完全一样?

### Expected behavior

transformers save_pretrained bug

--- Comment by github-actions[bot] at 2026-04-04T08:08:58Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


推荐一个基于 Transformers 的 AI 效率工具库

ISSUE #44453: 推荐一个基于 Transformers 的 AI 效率工具库
State: closed  |  Labels:
Author: zhuxunyu  |  Created: 2026-03-05T03:39:45Z

作者好！

感谢维护这个优秀的项目！我在使用过程中开发了一个基于 Transformers 的效率工具库，感觉可以互相学习参考：

🔗 https://github.com/zhuxunyu/ai-productivity-toolkit

这个工具库包含：
- Prompt 优化器（自动优化 AI 提示词）
- Excel 自动化工具（批量处理/转换）
- 数据爬取框架（合法合规采集）
- AI 工作流（一键自动化复杂任务）

都是 Python 实现，完全开源免费。

感觉可以和 transformers 生态互相参考，也许能整合一些功能？

再次感谢作者的辛勤付出！🙏

--- Comment by github-actions[bot] at 2026-04-04T08:08:59Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.


ISSUE #44451: Latest version cannot load "vesteinn/ScandiBERT"
State: closed  |  Labels: bug
Author: AngledLuffa  |  Created: 2026-03-05T03:02:13Z

### System Info

broken config:

Python 3.13.5
tokenizers               0.22.2
transformers             5.2.0
torch                    2.7.1+cu118

working config:

Python 3.13.5
tokenizers               0.22.1
transformers             4.57.1
torch                    2.8.0+cu129

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
>>> from transformers import AutoTokenizer
>>> bert_tokenizer = AutoTokenizer.from_pretrained("vesteinn/ScandiBERT")

Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    bert_tokenizer = AutoTokenizer.from_pretrained("vesteinn/ScandiBERT")
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/models/auto/tokenization_auto.py", line 712, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/tokenization_utils_base.py", line 1712, in from_pretrained
    return cls._from_pretrained(
           ~~~~~~~~~~~~~~~~~~~~^
        resolved_vocab_files,
        ^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/tokenization_utils_base.py", line 1897, in _from_pretrained
    init_kwargs = cls.convert_to_native_format(**init_kwargs)
  File "/nlp/scr/horatio/miniconda3/lib/python3.13/site-packages/transformers/tokenization_utils_tokenizers.py", line 127, in convert_to_native_format
    if vocab and isinstance(vocab[0], (list, tuple)):
                            ~~~~~^^^
KeyError: 0
```

### Expected behavior

Loading without an exception would be better!

--- Comment by AngledLuffa at 2026-03-05T03:10:33Z ---
This worked as of transformers 4.57.6, but not starting with transformers 5.0.0

--- Comment by Cyrilvallez at 2026-03-09T11:46:04Z ---
cc @ArthurZucker, another instance of the tokenizer issue!

--- Comment by AngledLuffa at 2026-03-09T17:59:10Z ---
The linked PR looks promising.  Sounds like it fixes the problem for this model

--- Comment by AngledLuffa at 2026-03-19T17:45:28Z ---
Thank you!  That's excellent.  Swedish users of Stanza everywhere will rejoice

(I have no idea how many there actually are, but HF analytics indicate there are some downloads, and not all of them were me testing out this particular bug!)


ISSUE #44450: Support argumentless loading from Trainer checkpoints
State: open  |  Labels: Feature request
Author: adosar  |  Created: 2026-03-05T02:08:23Z

### Feature request

`Trainer` checkpoints don't include the `config.json` necessary to instantiate the model.

This means that if we want to use a specific checkpoint (e.g. use it for fine-tuning, evaluate on test set etc.) we need to know the `init_args` when calling `.from_pretrained(ckpt_path, **init_args)`.

By saving the `config.json` under the checkpoint, `.from_pretrained(ckpt_path)` suffices.

```bash
├── model.safetensors
├── optimizer.pt
├── rng_state.pth
├── scheduler.pt
├── special_tokens_map.json
├── tokenizer.json
├── tokenizer_config.json
├── trainer_state.json
├── training_args.json
└── config.json  # Store this also
```

### Motivation

It comes handy when a model needs to be loaded back for further fine-tuning/testing/debugging etc.

### Your contribution

Not familiar with the HuggingFace code base, but I would be happy to do it as my first PR.

--- Comment by Rocketknight1 at 2026-03-05T13:19:35Z ---
cc @sunmarc


ISSUE #44448: [BUG] Different output for google/pegasus-cnn_dailymail between Transformers v4 and v5
State: closed  |  Labels: bug
Author: math-hiyoko  |  Created: 2026-03-04T22:37:37Z

### System Info

- `transformers` version: 4.57.6 (working) / 5.0.0 (incorrect output)
- Platform: Linux-6.6.113+-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 1.5.0
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.10.0+cpu (NA)
- Using distributed or parallel set-up in script?: distributed

### Who can help?

@ArthurZucker @Cyrilvallez

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When executing the example from the Pegasus documentation:
https://huggingface.co/docs/transformers/main/en/model_doc/pegasus?usage=AutoModel

Transformers v4 produces a reasonable summary, while v5 produces an incoherent / meaningless output.
```python
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "google/pegasus-cnn_dailymail"
)
model = AutoModelForSeq2SeqLM.from_pretrained(
    "google/pegasus-cnn_dailymail",
    dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)

input_text = """Plants are remarkable organisms that produce their own food using a method called photosynthesis.
This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
Plants play a crucial role in sustaining life on Earth by generating oxygen and serving as the foundation of most ecosystems."""
input_ids = tokenizer(input_text, return_tensors="pt").to(model.device)

output = model.generate(**input_ids, cache_implementation="static")
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

output (v4.*)
```
Plants are remarkable organisms that produce their own food using a method called photosynthesis.<n>This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
```

output (v5.*)
```
And want seek help Jen first gavepist brought well magical am high place go her three Brit Meadows building – event masterre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows building masterre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows buildingpre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows buildingpre – variety source me well only healthy am many high place go her three Brit Meadows buildingpre – variety source me well only healthy am many high place go her three Brit Meadows buildingpre –
```

### Expected behavior

The model generates a coherent summary of the article.
Example output:
```
Plants are remarkable organisms that produce their own food using a method called photosynthesis.<n>This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
```

--- Comment by Cyrilvallez at 2026-03-09T09:34:33Z ---
I `git`-bisected, and the issue appears after https://github.com/huggingface/transformers/pull/42563, cc @ArthurZucker @itazap it's tokenization-related!

--- Comment by ArthurZucker at 2026-03-10T09:55:14Z ---
The tokens are the same in v5, the issue is the added tokens, checking but most probably mapping related.

--- Comment by ArthurZucker at 2026-03-10T14:32:04Z ---
Okay we just were missing a convert from_spm for this model


ISSUE #44442: [BUG] AutoTokenizer fails to load FastSpeech2ConformerTokenizer
State: closed  |  Labels: bug
Author: harshaljanjani  |  Created: 2026-03-04T19:58:11Z

### System Info

* `transformers` version: `5.0.0.dev0`
* Platform: `Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.39`
* Python version: `3.12.3`
* `huggingface_hub` version: `1.3.2`
* `safetensors` version: `0.7.0`
* `accelerate` version: `1.12.0`
* Accelerate config: `not installed`
* DeepSpeed version: `not installed`
* PyTorch version (accelerator?): `2.9.1+cu128 (CUDA)`
* GPU type: `NVIDIA L4`
* NVIDIA driver version: `550.90.07`
* CUDA version: `12.4`

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer

try:
    tokenizer = AutoTokenizer.from_pretrained("espnet/fastspeech2_conformer")
    print("Loaded tokenizer:", type(tokenizer))
except Exception as e:
    print(e)
```

Trying to load `espnet/fastspeech2_conformer` tokenizer using `AutoTokenizer` crashes with a `ValueError`; the output I've attached below (feel free to reproduce) shows that `AutoTokenizer` is failing to instantiate a fast tokenizer backend.

**Current Output:**

<img alt="Image" src="https://github.com/user-attachments/assets/532044d0-a936-4acc-ba22-e6b0525dd793" />

### Expected behavior

→ `AutoTokenizer.from_pretrained("espnet/fastspeech2_conformer")` should correctly instantiate and return a `FastSpeech2ConformerTokenizer` obj.


ISSUE #44423: [Bug] `transformers serve --continuous-batching` crashes with multimodal models (Qwen3.5) — AttributeError: 'str' object has no attribute 'to'
State: closed  |  Labels:
Author: jw9603  |  Created: 2026-03-04T00:51:26Z

### System Info

- `transformers` main branch (5.3.0.dev0, commit 5c1c72be)
- Python 3.11.14
- PyTorch 2.5.1+cu121
- OS: Ubuntu Linux

### Who can help?

@Lysandre @ArthurZucker @joaogante

### Reproduction

1. Install latest transformers from main:
```bash
pip install "transformers[serving] @ git+https://github.com/huggingface/transformers.git@main"
```

2. Launch the server with continuous batching:
```
transformers serve --force-model Qwen/Qwen3.5-9B --port 8000 --continuous-batching
```

3. Send any chat completion request:
```
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3.5-9B", "messages": [{"role": "user", "content": "Hello"}]}'
```

### Error
```
File "transformers/cli/serve.py", line 831, in continuous_batching_chat_completion
    ).to(model.device)["input_ids"][0]
      ^^
AttributeError: 'str' object has no attribute 'to'
```

### Root Cause
In [serve.py line 829-831](https://github.com/huggingface/transformers/blob/main/src/transformers/cli/serve.py#L829-L831):
```
inputs = processor.apply_chat_template(
    req["messages"], return_tensors="pt", add_generation_prompt=True, return_dict=True
).to(model.device)["input_ids"][0]
```

For multimodal models like Qwen3.5, `processor` is a multimodal processor (not a plain tokenizer). Its `apply_chat_template()` returns a plain string instead of a `BatchEncoding`, so calling `.to(model.device)` raises `AttributeError.`

Note: Without `--continuous-batching`, the server works fine with Qwen3.5.

### Expected behavior

`transformers serve --continuous-batching` should handle multimodal models whose processor returns a string from `apply_chat_template`.

--- Comment by Rocketknight1 at 2026-03-04T13:56:29Z ---
cc @molbap @mcpatate @remi-or for CB

--- Comment by remi-or at 2026-03-04T14:24:43Z ---
IMO this is a legitimate issue that should be addressed on the processors side, check https://github.com/huggingface/transformers/pull/44424 for more info. Don't know who to tag for this, maybe @zucchini-nlp as it concerns multi-modal models?
Also, this looks like code agent slop.

--- Comment by zucchini-nlp at 2026-03-04T14:29:38Z ---
We just need to pass `processor.apply_chat_template(tokenize=True)` so that it returns a processed dict. IIRC the tokenizer has a different default, so it worked for simple LLMs but not MLLMs

Anyone, pls feel free to open a PR and add a test, check that it works! (no code agents pls)


ISSUE #44419: Urge DOJ & FBI to Protect AI Innovation Even as the U.S. Focuses on the Iran War
State: closed  |  Labels:
Author: NobleResearch  |  Created: 2026-03-03T22:27:17Z

Due to U.S.-Israel war on Iran, as we all can see, the People's Republic of China (PRC) and the Russian Federation do not appear to help Ayatollah Iran on the war. Why? Well, even though it might not be proven yet, one reasonable reason might be that they are using this to distract the U.S. federal law enforcement, military, and intelligence communities to only thinking about Iran but not other geopolitical threats like PRC stealing U.S. AI research, which is actually important itself to fight in Iran.

This is why I am creating a petition to remind the FBI and DOJ to allocate more resources to continuously watch whether U.S. AI research has been stolen.

Here is the link below:
https://www.change.org/p/enhance-protection-for-american-ai-intellectual-property

Also, please kindly note that your colleagues, peers, etc. might also receive this message, so please kindly discuss this message with the ones that you know but also work in related or similar industry.

Thank you very much for your attention.


📋 Documentation Enhancement Suggestion

ISSUE #44418: 📋 Documentation Enhancement Suggestion
State: closed  |  Labels:
Author: croviatrust  |  Created: 2026-03-03T21:58:34Z

<spam>


ISSUE #44410: qwen3next: layer 0 missing attn_qkv/attn_gate projections
State: closed  |  Labels: bug
Author: fcorneli  |  Created: 2026-03-03T10:49:24Z

### System Info

Since Ollama 0.17.5 I get the following error:
```
ollama run qwen3-next:80b-a3b-instruct-q4_K_M
Error: 500 Internal Server Error: failed to initialize model: qwen3next: layer 0 missing attn_qkv/attn_gate projections
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Since Ollama 0.17.5 I get the following error:
```
ollama run qwen3-next:80b-a3b-instruct-q4_K_M
Error: 500 Internal Server Error: failed to initialize model: qwen3next: layer 0 missing attn_qkv/attn_gate projections
```

### Expected behavior

Expected behavior is that qwen3-next:80b-a3b-instruct-q4_K_M loads as before.

--- Comment by fcorneli at 2026-03-03T11:04:21Z ---
Wrong project.


ISSUE #44409: Suggestion: optional external troubleshooting reference for retrieval-heavy and RAG-style workflows
State: closed  |  Labels:
Author: onestardao  |  Created: 2026-03-03T09:34:55Z

Hi, I know this may be slightly downstream from the core scope of Transformers itself, so I wanted to frame this as an optional external reference rather than a request to change any core documentation direction.

I’d like to suggest the WFGY RAG 16 Problem Map as a practical, framework-agnostic troubleshooting resource for users building retrieval-heavy or RAG-style workflows on top of transformer stacks.

It is a compact failure checklist designed to help classify common issues such as:
- retrieval misses
- stale or mismatched context
- chunking / embedding mismatch
- citation instability
- answer inconsistency
- multi-step reasoning breakdowns

Reference:
https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

The WFGY repository is currently around 1.6k stars, and the Problem Map has already been referenced or integrated by multiple public RAG / LLM ecosystem projects, including:
- RAGFlow
- LlamaIndex
- ToolUniverse (Harvard MIMS Lab)
- Rankify
- Multimodal RAG Survey (QCRI LLM Lab)

I am not suggesting this as a Transformers-specific standard, only as a practical external debugging reference that some downstream users may find useful when troubleshooting retrieval and context assembly failures.

If this feels appropriate anywhere in your broader docs / resources ecosystem, I’d be happy to prepare a minimal docs-only PR or adapt the wording to match the repo’s style.

Thanks for your time.

--- Comment by Rocketknight1 at 2026-03-03T14:08:07Z ---
No thank you!

--- Comment by Rocketknight1 at 2026-03-03T14:52:39Z ---
Also @onestardao if I'm honest I should flag that I see several signs of AI psychosis here. Most of the code in your repo doesn't actually do anything. The core engine `wfgy_sdk/wfgy_engine.py` just scales logits by `0.55` and the rest is just prompts to pass to ChatGPT. It's all just a chaotic mess of folders and links, generally AI generated, with lots of "quantum mysticism" and allusions to fancy physics (including naming a random constant `mc²` that has nothing to do with rest mass or the speed of light)

To the extent that you are developing this project in conversation with an AI, please be warned that it might be too "eager-to-please" and is essentially leading you down a nonsensical path. Discovering this after you invest a lot of time and personal reputation into this might be harmful, so I'd encourage you to take a critical look sooner rather than later! If you're not confident enough with the code or math to review it without AI help, I'd suggest using a strong code agent harness (e.g. Claude Code + Opus 4.6) and asking it to critically review the project from an external perspective, without prompting it in a way that makes it too "eager-to-please" first.

--- Comment by onestardao at 2026-03-03T14:59:36Z ---
Thanks for the candid feedback — I appreciate you taking the time to look through the repository.

You're right that the early `wfgy_sdk/wfgy_engine.py` file does not contain meaningful standalone functionality. That was part of my very first GitHub experiment and was never intended as a production-ready execution engine. I should have documented that more clearly.

Since then, the project direction has shifted significantly toward TXT-style reasoning artifacts rather than executable SDK components. The early folder structure remains for historical reasons, but I agree it can create confusion.

I’ll review the repository structure and documentation to make the boundaries and intent clearer. Thanks again for the honest input.


ISSUE #44405: Add AutoModelForSequenceClassification support for Qwen3.5 (Qwen3_5Config)
State: closed  |  Labels: Feature request
Author: medhakimbedhief  |  Created: 2026-03-03T03:19:25Z

### Feature request

### What happens

When trying to load a Qwen3.5 model for sequence classification:
```
from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained(
    "Qwen/Qwen3.5-0.8B",
    num_labels=2,
    trust_remote_code=True,
)
```

Transformers raises:
```
ValueError: Unrecognized configuration class <class 'transformers.models.qwen3_5.configuration_qwen3_5.Qwen3_5Config'>
for this kind of AutoModel: AutoModelForSequenceClassification.
```
This error occurs because Qwen3_5Config is not registered in the internal mapping of configuration classes to sequence-classification model classes. Sequence classification mappings currently include Qwen3Config and related, but not Qwen3_5Config. This prevents classification from loading via the auto class.

### Expected Behavior
```AutoModelForSequenceClassification.from_pretrained("Qwen/Qwen3.5-0.8B", ...)``` should:
- Recognize ```Qwen3_5Config```
- Instantiate a ```Qwen3_5ForSequenceClassification``` class (or equivalent)
- Allow fine-tuning and inference on classification tasks without manual patching

### Metadata
**Libraries / Versions**
- transformers >= 5.2.0
- torch >= 2.x
- Qwen3.5 model on HF Hub

**Task**
- sequence classification

### Motivation

Many users are adopting Qwen3.5 for fine-tuning tasks beyond generation (e.g., classification, routing), and the auto model infrastructure currently does not support classification out of the box. Addressing this will improve usability for downstream tasks.

### Your contribution

I am working on implementing it and will open a PR shortly.


ISSUE #44404: Bring Loss classes back to modeling file and leverage modular
State: closed  |  Labels:
Author: sbucaille  |  Created: 2026-03-03T03:10:54Z

As discussed while implementing #36895 , I think losses that are currently present in the loss folder should be back to the modeling file like it is the case for many existing models (MaskFormer for example).
I think it makes sense because the loss, when custom like the LwDetr or RfDetr one are part of the model definition. They are released by authors as part of it. And per the tenets, users should not have to browse across different files to know how the loss got computed for their training.
Additionally, we have modular now so it could be a great opportunity to refactor them and leverage that.
I'd like to volunteer for that.

EDIT : at least for the custom one that are defined in the `LOSS_MAPPING` :
https://github.com/huggingface/transformers/blob/28d02a31386a4dfd85e2a91f86b7dfcebd44596d/src/transformers/loss/loss_utils.py#L157-L167

cc @yonigozlan @molbap @ArthurZucker

--- Comment by github-actions[bot] at 2026-04-02T08:17:50Z ---
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.