accelerate/pr_4021/en/package_reference/fp8.md

FP8

Below are functions and classes relative to the underlying FP8 implementation

FP8RecipeKwargs[[accelerate.utils.FP8RecipeKwargs]]

accelerate.utils.FP8RecipeKwargs[[accelerate.utils.FP8RecipeKwargs]]

Source

Deprecated. Please use one of the proper FP8 recipe kwargs classes such as TERecipeKwargs or MSAMPRecipeKwargs instead.

convert_model[[accelerate.utils.convert_model]]

accelerate.utils.convert_model[[accelerate.utils.convert_model]]

Source

Recursively converts the linear and layernorm layers of a model to their transformers_engine counterpart.

has_transformer_engine_layers[[accelerate.utils.has_transformer_engine_layers]]

accelerate.utils.has_transformer_engine_layers[[accelerate.utils.has_transformer_engine_layers]]

Source

Returns whether a given model has some transformer_engine layer or not.

contextual_fp8_autocast[[accelerate.utils.contextual_fp8_autocast]]

accelerate.utils.contextual_fp8_autocast[[accelerate.utils.contextual_fp8_autocast]]

Source

Wrapper for a model's forward method to apply FP8 autocast. Is context aware, meaning that by default it will disable FP8 autocast during eval mode, which is generally better for more accurate metrics.

apply_fp8_autowrap[[accelerate.utils.apply_fp8_autowrap]]

accelerate.utils.apply_fp8_autowrap[[accelerate.utils.apply_fp8_autowrap]]

Source

Applies FP8 context manager to the model's forward method