File size: 2,483 Bytes
1fa3c6c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | # Liger Kernel Integration
[Liger Kernel](https://github.com/linkedin/Liger-Kernel) is a collection of Triton kernels designed specifically for LLM training. It can effectively increase multi-GPU training throughput by 20% and reduce memory usage by 60%. That way, we can **4x** our context length, as described in the benchmark below. They have implemented Hugging Face compatible `RMSNorm`, `RoPE`, `SwiGLU`, `CrossEntropy`, `FusedLinearCrossEntropy`, with more to come. The kernel works out of the box with [FlashAttention](https://github.com/Dao-AILab/flash-attention), [PyTorch FSDP](https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html), and [Microsoft DeepSpeed](https://github.com/microsoft/DeepSpeed).
With this memory reduction, you can potentially turn off `cpu_offloading` or gradient checkpointing to further boost the performance.
| Speed Up | Memory Reduction |
| --- | --- |
|  |  |
## Supported Trainers
Liger Kernel is supported in the following TRL trainers:
- **SFT** (Supervised Fine-Tuning)
- **DPO** (Direct Preference Optimization)
- **GRPO** (Group Relative Policy Optimization)
- **KTO** (Kahneman-Tversky Optimization)
- **GKD** (Generalized Knowledge Distillation)
## Usage
1. First, install Liger Kernel:
```bash
pip install liger-kernel
```
2. Once installed, set `use_liger_kernel=True` in your trainer config. No other changes are needed!
<hfoptions id="liger">
<hfoption id="SFT">
```python
from trl import SFTConfig
training_args = SFTConfig(..., use_liger_kernel=True)
```
</hfoption>
<hfoption id="DPO">
```python
from trl import DPOConfig
training_args = DPOConfig(..., use_liger_kernel=True)
```
</hfoption>
<hfoption id="GRPO">
```python
from trl import GRPOConfig
training_args = GRPOConfig(..., use_liger_kernel=True)
```
</hfoption>
<hfoption id="KTO">
```python
from trl import KTOConfig
training_args = KTOConfig(..., use_liger_kernel=True)
```
</hfoption>
<hfoption id="GKD">
```python
from trl.experimental.gkd import GKDConfig
training_args = GKDConfig(..., use_liger_kernel=True)
```
</hfoption>
</hfoptions>
To learn more about Liger-Kernel, visit their [official repository](https://github.com/linkedin/Liger-Kernel/).
|