transformers/pr_48314/en/pr_checks.md

# Pull request checks

When you open a pull request, the Hugging Face CI runs several checks that must pass before your PR can be merged.

- [Fixing the CI](#fixing-the-ci) lists the commands to run locally to pass the checks.
- The sections after that describe what each check validates, such as [code quality](#code-quality) and [repository consistency](#repository-consistency).
- [Tests](#tests) describe which tests are run, test categories, and slow tests.

## Fixing the CI

In most cases, `make style` is enough to clear the code quality check, which is the most common failure.

```bash
make style
```

For failures in repository consistency, copies, or auto-generated files, run `make fix-repo`. It fixes style, copies, docstrings, and auto-generated files in one pass.

```bash
make fix-repo
```

For a larger change, the three-command sequence below covers every check. It is heavier, but catches everything before you push.

```bash
make fix-repo   # auto-fix everything that can be auto-fixed
make typing     # check types and model structure, fix any errors manually
make check-repo # verify all checks pass, fix anything that remains
```

`make typing` catches type errors and model structure violations that you fix manually. `make check-repo` does a final read-only pass so you can confirm everything is ready.

> [!NOTE]
> The CI is occasionally flaky. If a check fails on something unrelated to your change, ping a maintainer to re-run it.

## Code quality

The code quality check covers formatting, imports, type checking, and model structure rules. It corresponds to `make fix-repo` and `make typing`.

`make style` (included in `make fix-repo`) auto-fixes [Ruff](https://docs.astral.sh/ruff/) linting and formatting, `__init__.py` import sort order, and auto-mapping consistency.

`make typing` performs type checking with [ty](https://docs.astral.sh/ty/) and validates TRansFormers (TRF) rules, which cover config class naming conventions and `forward()` signatures. Type errors and TRF violations report a specific rule number and must be fixed manually. The rules live in the [mlinter](https://github.com/huggingface/transformers-mlinter) repository. Run `python -m utils.mlinter --list-rules` to see every TRF rule, or `python -m utils.mlinter --rule TRFXXX` to view the full documentation for a specific rule.

If a TRF rule needs an exception, choose one of these options (see [Suppressing violations](./modeling_rules#suppressing-violations) for more details).

- Add your model name to the `allowlist_models` list for the relevant rule in `utils/mlinter/rules.toml`. Use this when the whole model file needs an exception.
- Add `# trf-ignore: TRFXXX` on the same line as the flagged construct, or on the line immediately above it. Use this when only one flagged construct needs an exception.

## Repository consistency

The repository consistency check is similar to `make check-repo`, except it stops on the first failure. It keeps the repository internally consistent across the categories below: public objects stay importable, copied code stays in sync with its source, and auto-generated files (dummies, doctests, metadata) reflect the current state of the code. For new models, it also verifies every new model class is registered in the auto-mappings.

| Category | What it validates | Auto-fixed? |
|---|---|---|
| Init files | `src/transformers/models/__init__.py` matches the import structure on disk, so the `if TYPE_CHECKING` block (type checker imports) exposes the same models as the lazy runtime half | `make fix-repo` |
| Copies and modular | `# Copied from` blocks match their source and modular-generated files are up to date | `make fix-repo` |
| Docstrings and docs | Argument docstrings match function signatures and documentation table of contents | `make fix-repo` |
| Auto-generated files | Dummies, pipeline typing, doctest list, metadata, dependency table | `make fix-repo` |
| Config validation | Config classes have valid checkpoints in docstrings and config attributes match modeling file | Manual |
| Reviewer assignment | Every model directory, and every rule in the reviewer file, still reaches a reviewer | Manual |

## Reviewer assignment

When a pull request is marked ready for review, the `Assign PR Reviewers` workflow requests up to two reviewers, ranked by how many lines the pull request changes in the files they own. A draft pull request gets no reviewers until it leaves draft. The workflow itself lives in [huggingface/transformers-ci](https://github.com/huggingface/transformers-ci) and this repository calls it; what stays here is the ownership data it reads.

Ownership is resolved per file, most specific first.

| Where | Looks like | Use it for |
|---|---|---|
| The model file | `# Reviewers: @login` in the leading comment block of `modular_<model>.py` or `modeling_<model>.py` | a model that needs someone other than its modality owner. The modular converter copies the header into the generated modeling file, so the tag survives regeneration |
| A path rule | `/src/transformers/<area>/ @login` in `.github/scripts/codeowners_for_review_action` | anything that is not a model |
| The modality table | `@@modality/vision @login`, in the same file | every model whose doc page sits in that section of `docs/source/en/_toctree.yml` |
| The catch-all | `* @Rocketknight1 @ArthurZucker` | whatever nothing else claims |

A new model needs no entry anywhere: adding its documentation page to `_toctree.yml`, which the documentation table of contents check already requires, places it in a modality, and the modality table covers it from there.

A review can only be requested from a repository collaborator. The workflow requests each reviewer in a separate call and skips anyone it cannot ask, so an entry naming someone who has left costs one reviewer instead of all of them, and every skip is reported as a warning on the workflow run.

A pull request cannot re-route its own review. `.github/scripts/codeowners_for_review_action` is read from the base branch, never from the pull request's head, and the PR CI security gate blocks a pull request that is not from a collaborator from touching anything outside `src/`, `tests/`, `docs/` and `utils/` — which is why that file lives where it does. The `# Reviewers:` tag is read from the head, since a new model arrives with it, so every login it names still goes through the collaborator check before anyone is asked.

`make check-repository-consistency` runs `utils/check_reviewers.py`, which fails when a model reaches nobody but the catch-all, when a rule matches no file any more, or when the modality table disagrees with the table of contents. It shares its resolver with the assignment workflow, so the two cannot disagree. That resolver is not a dependency of `transformers`; it is listed in `utils/checkers-requirements.txt`, which the checker runner installs on its first run in an environment. If the install cannot happen — offline, say — the check reports it and passes rather than blocking you.
 Paths outside `src/transformers/models` that have no owner are reported as a note; list them with:

```bash
python utils/check_reviewers.py --strict
```

## Tests

CI runs a targeted subset of tests based on what your PR changes. The CI runs tests in a slightly randomized order with [pytest-random-order](https://github.com/jbasko/pytest-random-order) to catch coupled tests. The run prints the random seed at the start so you can replay the same order with `--random-order-seed=<seed>`.

If a test passes locally on GPU but fails in CI, set `TRANSFORMERS_TEST_DEVICE="cpu"` to check whether you can reproduce the failure on CPU.

```bash
TRANSFORMERS_TEST_DEVICE="cpu" pytest tests/models/my_model/ -v
```

The sections below explain how test selection works, which jobs run, and how to handle slow tests.

### Test selection

CI doesn't run the full test suite on every PR. CI skips tests decorated with `@slow` on every PR, and a maintainer triggers them on GPU once the PR is under review.

`utils/tests_fetcher.py` traces import dependencies from your changed files to identify affected tests, and only runs those. It also catches regressions in other models when you touch shared utilities. The fetcher prints which files changed and which tests are impacted, then writes the list to `tests_torch_test_list.txt`.

Use the fetcher to replicate exactly what CI runs.

```bash
python utils/tests_fetcher.py
python -m pytest -n 8 --dist=loadfile -rA -s $(cat test_preparation/tests_torch_test_list.txt)
```

> [!TIP]
> Changing core files like `modeling_utils.py` or `generation/utils.py` triggers all model tests, not just the affected subset.

You can also run every test for your model unconditionally. A direct run is a faster local sanity check, but it won't catch regressions in other models caused by shared code you touched.

```bash
pytest tests/models/my_model/ -v
```

### Test job categories

Tests are split across parallel CI jobs, and each job picks up files by path pattern. The relevant jobs for a model PR are:

- `tests_torch`: modeling tests (`tests/models/*/test_modeling_*.py`)
- `tests_tokenization`: tokenizer tests (`tests/models/*/test_tokenization_*.py`)
- `tests_processors`: processor and feature extractor tests (`tests/models/*/test_(processing|image_processing|feature_extractor)_*.py`)
- `tests_generate`: generation tests
- `pipelines_torch`: pipeline tests
- `tests_training_ci`: training loop tests
- `tests_tensor_parallel_ci`: tensor parallel tests
- `tests_fsdp_ci`: FSDP tests

### Slow tests

Regular CI runs skip tests decorated with `@slow`. They download real checkpoints or need significant compute, so they run on GPU instances, and maintainers trigger them once your PR is under review.

Slow tests run on an NVIDIA A10, and numerical results can vary slightly between the CI hardware and your local machine. Maintainers usually adjusts those values on tests if needed when adding a new model.

Run slow tests locally with the command below.

```bash
RUN_SLOW=1 python -m pytest tests/models/my_model/ -v
```

### Documentation build

A `build_pr_documentation` job builds and generates a preview of the documentation. A bot posts a preview link in your PR, and the check must pass before merging. Most failures are a missing entry in the `toctree`. To build the documentation locally, see the [README.md](https://github.com/huggingface/transformers/tree/main/docs) in the docs folder.

## `# Copied from` syntax

> [!WARNING]
> For new models, always prefer the [modular workflow](modular_transformers) (`modular_*.py`) over `# Copied from`. Avoid `# Copied from` whenever possible.

The `# Copied from` mechanism keeps copied code in sync with its source. When `make fix-repo` runs, it checks every `# Copied from` block and updates it to match the original, so edits inside a `# Copied from` block are overwritten. Edit the source instead and let `make fix-repo` propagate the change.

The basic forms of `# Copied from` include the following.

```py
# Copied from transformers.models.bert.modeling_bert.BertSelfOutput

# Copied from transformers.models.bert.modeling_bert.BertAttention with Bert->Roberta

# Copied from transformers.models.bert.modeling_bert.BertForSequenceClassification with Bert->MobileBert all-casing
```

The `with model->newModel` syntax applies string replacements after copying. Separate multiple replacements with commas, which are applied from left to right. The `all-casing` option replaces every casing variant at once (`Bert`, `bert`, `BERT` become `MobileBert`, `mobilebert`, `MOBILEBERT`).