transformers/pr_39895/en/main_classes/kernels.md

Kernels

This page documents the kernels configuration utilities.

kernelize[[transformers.kernelize]]

transformers.kernelize[[transformers.kernelize]]

Source

Temporarily register hidden kernel wrappers so kernelize can discover and replace them.

KernelConfig[[transformers.KernelConfig]]

transformers.KernelConfig[[transformers.KernelConfig]]

Source

Kernel configuration class. This class is used to configure the kernel mapping for a model.

create_compatible_mappingtransformers.KernelConfig.create_compatible_mappinghttps://github.com/huggingface/transformers/blob/vr_39895/src/transformers/utils/kernel_config.py#L250[{"name": "model", "val": ""}, {"name": "compile", "val": " = False"}]

Transforms a simple kernel_mapping of the form: { "RMSNorm": ("kernels-community/layer_norm:LlamaRMSNorm", {"version": 1, "trust_remote_code": True}), ... },

or for local path:

{ "RMSNorm": "/home/user/liger_kernels:LigerRMSNorm", ... },

into a nested mapping:

{ "RMSNorm": { "cuda": { Mode.INFERENCE: LayerRepository( repo_id="kernels-community/layer_norm", layer_name="LlamaRMSNorm", version=1, trust_remote_code=True, ) } } }

or for local path:

{ "RMSNorm": { "cuda": { Mode.INFERENCE: LocalLayerRepository( repo_path=Path("/home/user/liger_kernels"), package_name="liger_kernels", layer_name="LigerRMSNorm", ) } } }

that's compatible with the kernels library.

The device is inferred from the model's parameters if not provided. The Mode is inferred from the model's training state.

sanitize_kernel_mapping[[transformers.KernelConfig.sanitize_kernel_mapping]]

Source

Validates the kernel_mapping to ensure that:

1. Each layer_name in the mapping is registered in the model (i.e., the model contains a module with a matching kernel_layer_name).
2. Each kernel value is
   • either a string of the form 'org/repo:layer_name' or a tuple with the same as string and a dict of {"revision"/"version/trust_remote_code": ...},
   • or a dict mapping device types ("cuda", "rocm", "xpu", "npu") to such values as above.
3. Each device key in a dict is one of "cuda", "rocm", "xpu", or "npu".
4. Each trust remote code key must be a bool.
5. Each revision or version key must exist mutually exclusive if it has been passed explicitly.
6. Each repo_name is a valid repository and layer name in the format 'org/repo:layer_name' (i.e., a string containing both a slash and a colon).
7. If a local path is detected, it should be in the format '/abs/path:layer_name'. The absolute path must include the package_name, like "/home/user/layer_norm".

Parameters:

model : The model instance whose modules are checked for registered kernel_layer_name attributes.