docs/source/en/internal/modeling_utils.md

Custom Layers and Utilities

This page lists all the custom layers used by the library, as well as the utility functions and classes it provides for modeling.

Most of those are only useful if you are studying the code of the models in the library.

WeightConverter

[[autodoc]] WeightConverter

Conversion operations

[[autodoc]] ConversionOps

[[autodoc]] Chunk

[[autodoc]] Concatenate

[[autodoc]] MergeModulelist

[[autodoc]] SplitModulelist

[[autodoc]] PermuteForRope

Layers

[[autodoc]] GradientCheckpointingLayer

Attention Functions

[[autodoc]] AttentionInterface - register

Attention Mask Functions

[[autodoc]] AttentionMaskInterface - register

Rotary Position Embedding Functions

[[autodoc]] dynamic_rope_update

Pytorch custom modules

[[autodoc]] pytorch_utils.Conv1D

PyTorch Helper Functions

[[autodoc]] pytorch_utils.apply_chunking_to_forward

[[autodoc]] pytorch_utils.prune_linear_layer