File size: 24,768 Bytes

1fa3c6c

# PEFT Integration

TRL supports [PEFT](https://github.com/huggingface/peft) (Parameter-Efficient Fine-Tuning) methods for memory-efficient model training. PEFT enables fine-tuning large language models by training only a small number of additional parameters while keeping the base model frozen, significantly reducing computational costs and memory requirements.

This guide covers how to use PEFT with different TRL trainers, including LoRA, QLoRA, and prompt tuning techniques.

For a complete working example, see the [SFT with LoRA/QLoRA notebook](https://github.com/huggingface/trl/blob/main/examples/notebooks/sft_trl_lora_qlora.ipynb).

## Installation

To use PEFT with TRL, install the required dependencies:

```bash

pip install trl[peft]

```

For QLoRA support (4-bit and 8-bit quantization), also install:

```bash

pip install bitsandbytes

```

## Quick Start

All TRL trainers support PEFT through the `peft_config` argument. The simplest way to enable PEFT is by using the command-line interface with the `--use_peft` flag:

```bash

python trl/scripts/sft.py \

    --model_name_or_path Qwen/Qwen2-0.5B \

    --dataset_name trl-lib/Capybara \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16 \

    --output_dir Qwen2-0.5B-SFT-LoRA

```

Alternatively, you can pass a PEFT config directly in your Python code:

```python

from peft import LoraConfig

from trl import SFTTrainer



# Configure LoRA

peft_config = LoraConfig(

    r=32,

    lora_alpha=16,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

)



# Configure training - note the higher learning rate for LoRA (10x base rate)

training_args = SFTConfig(

    learning_rate=2.0e-4,  # 10x the base rate (2.0e-5) for LoRA

    ...

)



# Create trainer with PEFT

trainer = SFTTrainer(

    model=model,

    train_dataset=dataset,

    peft_config=peft_config,

)

```

## Three Ways to Configure PEFT

TRL provides three different methods to configure PEFT, each suited for different use cases:

### 1. Using CLI Flags (Simplest)

The easiest way to enable PEFT is to use the `--use_peft` flag with the command-line interface. This method is ideal for quick experiments and standard configurations:

```bash

python trl/scripts/sft.py \

    --model_name_or_path Qwen/Qwen2-0.5B \

    --dataset_name trl-lib/Capybara \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16 \

    --lora_dropout 0.05 \

    --output_dir Qwen2-0.5B-SFT-LoRA

```

**Pros**: Quick setup, no code required

**Cons**: Limited to LoRA, fewer customization options

### 2. Passing peft_config to Trainer (Recommended)



For more control, pass a PEFT configuration directly to the trainer. This is the recommended approach for most use cases:



```python

from peft import LoraConfig

from trl import SFTConfig, SFTTrainer



peft_config = LoraConfig(
    r=32,

    lora_alpha=16,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],

)


trainer = SFTTrainer(
    model=model,

    args=training_args,

    train_dataset=dataset,

    peft_config=peft_config,  # Pass config here

)

```


**Pros**: Full control, supports all PEFT methods (LoRA, Prompt Tuning, etc.)

**Cons**: Requires Python code

### 3. Applying PEFT to Model Directly (Advanced)

For maximum flexibility, you can apply PEFT to your model before passing it to the trainer:

```python

from peft import LoraConfig, get_peft_model

from transformers import AutoModelForCausalLM

from trl import SFTConfig, SFTTrainer



# Load base model

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B")



# Apply PEFT configuration

peft_config = LoraConfig(

    r=32,

    lora_alpha=16,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

)

model = get_peft_model(model, peft_config)



# Pass PEFT-wrapped model to trainer

trainer = SFTTrainer(

    model=model,  # Already has PEFT applied

    args=training_args,

    train_dataset=dataset,

    # Note: no peft_config needed here

)

```

**Pros**: Maximum control, useful for custom model architectures or complex setups

**Cons**: More verbose, requires understanding of PEFT internals

## Learning Rate Considerations

When using LoRA or other PEFT methods, you typically need to use a **higher learning rate** (approximately 10x) compared to full fine-tuning. This is because PEFT methods train only a small fraction of parameters, requiring a larger learning rate to achieve similar parameter updates.

**Recommended learning rates:**

| Trainer | Full Fine-Tuning | With LoRA (10x) |
|---------|------------------|-----------------|
| **SFT** | `2.0e-5` | `2.0e-4` |
| **DPO** | `5.0e-7` | `5.0e-6` |
| **GRPO** | `1.0e-6` | `1.0e-5` |
| **Prompt Tuning** | N/A | `1.0e-2` to `3.0e-2` |

> **Why 10x?** LoRA adapters have significantly fewer trainable parameters than the full model. A higher learning rate compensates for this reduced parameter count, ensuring effective training. For detailed explanation, see [this blog post](https://thinkingmachines.ai/blog/lora/).

For additional best practices on using LoRA effectively, refer to the [LoRA Without Regret](lora_without_regret) documentation.

## PEFT with Different Trainers

TRL's trainers support PEFT configurations for various training paradigms. Below are detailed examples for each major trainer.

<hfoptions id="trainer-type">
<hfoption id="sft">

### Supervised Fine-Tuning (SFT)

The `SFTTrainer` is used for supervised fine-tuning on instruction datasets.

#### With LoRA

```bash

python trl/scripts/sft.py \

    --model_name_or_path Qwen/Qwen2-0.5B \

    --dataset_name trl-lib/Capybara \

    --learning_rate 2.0e-4 \

    --num_train_epochs 1 \

    --per_device_train_batch_size 2 \

    --gradient_accumulation_steps 8 \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16 \

    --output_dir Qwen2-0.5B-SFT-LoRA

```

#### Python Example

```python

from peft import LoraConfig

from trl import SFTConfig, SFTTrainer



# Configure LoRA

peft_config = LoraConfig(

    r=32,

    lora_alpha=16,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

    target_modules=["q_proj", "v_proj"],  # optional: specify target modules

)



# Configure training with higher learning rate for LoRA

training_args = SFTConfig(

    learning_rate=2.0e-4,  # 10x the base rate for LoRA

    ...

)



# Create trainer with PEFT config

trainer = SFTTrainer(

    model="Qwen/Qwen2-0.5B",  # can pass model name or loaded model

    args=training_args,

    train_dataset=dataset,

    peft_config=peft_config,  # pass PEFT config here

)

trainer.train()

```

</hfoption>
<hfoption id="dpo">

### Direct Preference Optimization (DPO)

The [`DPOTrainer`] implements preference learning from human feedback.

#### With LoRA

```bash

python trl/scripts/dpo.py \

    --model_name_or_path Qwen/Qwen2-0.5B-Instruct \

    --dataset_name trl-lib/ultrafeedback_binarized \

    --learning_rate 5.0e-6 \

    --per_device_train_batch_size 2 \

    --gradient_accumulation_steps 8 \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16 \

    --output_dir Qwen2-0.5B-DPO-LoRA

```

#### Python Example

```python

from peft import LoraConfig

from trl import DPOConfig, DPOTrainer



# Configure LoRA

peft_config = LoraConfig(

    r=32,

    lora_alpha=16,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

)



# Configure training with higher learning rate for LoRA

training_args = DPOConfig(

    learning_rate=5.0e-6,  # 10x the base rate for DPO with LoRA

    ...

)



# Create trainer with PEFT config

trainer = DPOTrainer(

    model="Qwen/Qwen2-0.5B",  # can pass model name or loaded model

    args=training_args,

    train_dataset=dataset,

    peft_config=peft_config,  # pass PEFT config here

)

trainer.train()

```

**Note:** When using PEFT with DPO, you don't need to provide a separate reference model (`ref_model`). The trainer automatically uses the frozen base model as the reference.

</hfoption>
<hfoption id="grpo">

### Group Relative Policy Optimization (GRPO)

The `GRPOTrainer` optimizes policies using group-based rewards.

#### With LoRA

```bash

python trl/scripts/grpo.py \

    --model_name_or_path Qwen/Qwen2-0.5B \

    --dataset_name trl-lib/math-reasoning \

    --learning_rate 1.0e-5 \

    --per_device_train_batch_size 2 \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16 \

    --output_dir Qwen2-0.5B-GRPO-LoRA

```

#### Python Example

```python

from peft import LoraConfig

from trl import GRPOConfig, GRPOTrainer



# Configure LoRA

peft_config = LoraConfig(

    r=32,

    lora_alpha=16,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

)



# Configure training with higher learning rate for LoRA

training_args = GRPOConfig(

    learning_rate=1.0e-5,  # 10x the base rate for GRPO with LoRA

    ...

)



# Create trainer with PEFT config

trainer = GRPOTrainer(

    model="Qwen/Qwen2-0.5B",  # can pass model name or loaded model

    args=training_args,

    train_dataset=dataset,

    peft_config=peft_config,  # pass PEFT config here

)

trainer.train()

```

</hfoption>
</hfoptions>

### Proximal Policy Optimization (PPO)

#### Multi-Adapter RL Training

You can use a single base model with multiple PEFT adapters for the entire PPO algorithm - including retrieving reference logits, computing active logits, and calculating rewards. This approach is useful for memory-efficient RL training.

> [!WARNING]
> This feature is experimental and convergence has not been extensively tested. We encourage the community to share feedback and report any issues.

**Requirements**

Install PEFT and optionally bitsandbytes for 8-bit models:

```bash

pip install peft bitsandbytes

```

**Training Workflow**

The multi-adapter approach requires three stages:

1. **Supervised Fine-Tuning (SFT)**: Train a base model on your target domain (e.g., IMDB dataset) using `SFTTrainer`
2. **Reward Model Training**: Train a reward model adapter using PEFT and `RewardTrainer` (see [reward modeling example](https://github.com/huggingface/trl/tree/main/examples/scripts/reward_modeling.py))
3. **PPO Training**: Fine-tune new adapters using PPO with the reward adapter

> [!IMPORTANT]
> Use the same base model (architecture and weights) for stages 2 & 3.

**Basic Usage**

After training your reward adapter and pushing it to the Hub:

```python

from peft import LoraConfig

from trl.experimental.ppo import PPOTrainer, AutoModelForCausalLMWithValueHead



model_name = "huggyllama/llama-7b"

rm_adapter_id = "trl-lib/llama-7b-hh-rm-adapter"



# Configure PPO adapter

lora_config = LoraConfig(

    r=16,

    lora_alpha=32,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

)



# Load model with reward adapter

model = AutoModelForCausalLMWithValueHead.from_pretrained(

    model_name,

    peft_config=lora_config,

    reward_adapter=rm_adapter_id,

)



trainer = PPOTrainer(model=model, ...)

```

In your training loop, compute rewards using:

```python

rewards = trainer.model.compute_reward_score(**inputs)

```

**Advanced Features**

**Quantized Base Models**

For memory-efficient training, load the base model in 8-bit or 4-bit while keeping adapters in float32:

```python

from transformers import BitsAndBytesConfig



model = AutoModelForCausalLMWithValueHead.from_pretrained(

    model_name,

    peft_config=lora_config,

    reward_adapter=rm_adapter_id,

    quantization_config=BitsAndBytesConfig(load_in_8bit=True),

)

```

## QLoRA: Quantized Low-Rank Adaptation

QLoRA combines 4-bit quantization with LoRA to enable fine-tuning of very large models on consumer hardware. This technique can reduce memory requirements by up to 4x compared to standard LoRA.

### How QLoRA Works

1. **4-bit Quantization**: The base model is loaded in 4-bit precision using `bitsandbytes`
2. **Frozen Weights**: The quantized model weights remain frozen during training
3. **LoRA Adapters**: Only the LoRA adapter parameters are trained in higher precision
4. **Memory Efficiency**: Enables fine-tuning of models like Llama-70B on a single consumer GPU

### Using QLoRA with TRL

Simply combine `load_in_4bit=True` with PEFT configuration:

#### Command Line

```bash

python trl/scripts/sft.py \

    --model_name_or_path meta-llama/Llama-2-7b-hf \

    --dataset_name trl-lib/Capybara \

    --load_in_4bit \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16 \

    --per_device_train_batch_size 1 \

    --gradient_accumulation_steps 16 \

    --output_dir Llama-2-7b-QLoRA

```

#### Python Example

```python

import torch



from peft import LoraConfig

from transformers import AutoModelForCausalLM, BitsAndBytesConfig

from trl import SFTConfig, SFTTrainer



# Configure 4-bit quantization

bnb_config = BitsAndBytesConfig(

    load_in_4bit=True,

    bnb_4bit_quant_type="nf4",

    bnb_4bit_compute_dtype=torch.bfloat16,

    bnb_4bit_use_double_quant=True,

)



# Load model with quantization

model = AutoModelForCausalLM.from_pretrained(

    "meta-llama/Llama-2-7b-hf",

    quantization_config=bnb_config,

    device_map="auto",

)



# Configure LoRA

peft_config = LoraConfig(

    r=32,

    lora_alpha=16,

    lora_dropout=0.05,

    bias="none",

    task_type="CAUSAL_LM",

)



# Configure training with higher learning rate for LoRA

training_args = SFTConfig(

    learning_rate=2.0e-4,  # 10x the base rate for QLoRA

    ...

)



# Create trainer with PEFT config

trainer = SFTTrainer(

    model=model,

    args=training_args,

    train_dataset=dataset,

    peft_config=peft_config,

)



trainer.train()

```

### QLoRA Configuration Options

The `BitsAndBytesConfig` provides several options to optimize memory and performance:

```python

import torch



from transformers import BitsAndBytesConfig



bnb_config = BitsAndBytesConfig(

    load_in_4bit=True,

    bnb_4bit_quant_type="nf4",  # or "fp4"

    bnb_4bit_compute_dtype=torch.bfloat16,  # Compute dtype for 4-bit base models

    bnb_4bit_use_double_quant=True,  # Nested quantization for additional memory savings

)

```

**Configuration Parameters:**
- `bnb_4bit_quant_type`: Quantization data type (`"nf4"` or `"fp4"`). NF4 is recommended.
- `bnb_4bit_compute_dtype`: The dtype used for computation. Use `bfloat16` for better training stability.
- `bnb_4bit_use_double_quant`: Enable nested quantization to save additional ~0.4 bits per parameter.

### 8-bit Quantization

For slightly higher precision with reduced memory savings, you can use 8-bit quantization:

```python

from transformers import BitsAndBytesConfig, AutoModelForCausalLM



bnb_config = BitsAndBytesConfig(load_in_8bit=True)



model = AutoModelForCausalLM.from_pretrained(

    "meta-llama/Llama-2-7b-hf",

    quantization_config=bnb_config,

    device_map="auto",

)

```

Or via command line:

```bash

python trl/scripts/sft.py \

    --model_name_or_path meta-llama/Llama-2-7b-hf \

    --load_in_8bit \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16

```

## Prompt Tuning

Prompt tuning is another PEFT technique that learns soft prompts (continuous embeddings) prepended to the input, while keeping the entire model frozen. This is particularly effective for large models.

### How Prompt Tuning Works

1. **Virtual Tokens**: Adds learnable continuous embeddings (virtual tokens) to the input
2. **Frozen Model**: The entire base model remains frozen
3. **Task-Specific Prompts**: Each task learns its own prompt embeddings
4. **Extreme Efficiency**: Only the prompt embeddings are trained (typically 8-20 tokens)

### Using Prompt Tuning with TRL

```python

from peft import PromptTuningConfig, PromptTuningInit, TaskType

from trl import SFTConfig, SFTTrainer



# Configure Prompt Tuning

peft_config = PromptTuningConfig(

    task_type=TaskType.CAUSAL_LM,

    prompt_tuning_init=PromptTuningInit.TEXT,

    num_virtual_tokens=8,

    prompt_tuning_init_text="Classify if the tweet is a complaint or not:",

    tokenizer_name_or_path="Qwen/Qwen2-0.5B",

)



# Configure training with higher learning rate for Prompt Tuning

training_args = SFTConfig(

    learning_rate=2.0e-2,  # Prompt Tuning typically uses 1e-2 to 3e-2

    ...

)



# Create trainer with PEFT config

trainer = SFTTrainer(

    model=model,

    args=training_args,

    train_dataset=dataset,

    peft_config=peft_config,  # pass PEFT config here

)



trainer.train()

```

### Prompt Tuning Configuration

```python

from peft import PromptTuningConfig, PromptTuningInit, TaskType



peft_config = PromptTuningConfig(

    task_type=TaskType.CAUSAL_LM,  # Task type

    prompt_tuning_init=PromptTuningInit.TEXT,  # Initialize from text

    num_virtual_tokens=8,  # Number of virtual tokens

    prompt_tuning_init_text="Your initialization text here",

    tokenizer_name_or_path="model_name",

)

```

**Configuration Parameters:**
- `task_type`: The task type (`TaskType.CAUSAL_LM` for language modeling)
- `prompt_tuning_init`: Initialization method (`TEXT`, `RANDOM`)
- `num_virtual_tokens`: Number of virtual tokens to prepend (typically 8-20)
- `prompt_tuning_init_text`: Text to initialize the virtual tokens (when using `TEXT` init)
- `tokenizer_name_or_path`: Tokenizer for initializing from text

### Prompt Tuning vs LoRA

| Feature | Prompt Tuning | LoRA |
|---------|---------------|------|
| **Parameters Trained** | ~0.001% | ~0.1-1% |
| **Memory Usage** | Minimal | Low |
| **Training Speed** | Fastest | Fast |
| **Model Modification** | None | Adapter layers |
| **Best For** | Large models, many tasks | General fine-tuning |
| **Learning Rate** | Higher (1e-2 to 3e-2) | Standard (1e-4 to 3e-4) |

## Advanced PEFT Configurations

### LoRA Configuration Parameters

```python

from peft import LoraConfig



peft_config = LoraConfig(

    r=16,  # LoRA rank

    lora_alpha=32,  # LoRA scaling factor

    lora_dropout=0.05,  # Dropout probability

    bias="none",  # Bias training strategy

    task_type="CAUSAL_LM",  # Task type

    target_modules=["q_proj", "v_proj"],  # Modules to apply LoRA

    modules_to_save=None,  # Additional modules to train

)

```

**Key Parameters:**
- `r`: LoRA rank (typical values: 8, 16, 32, 64). Higher rank = more parameters but potentially better performance.
- `lora_alpha`: Scaling factor (typically 2x the rank). Controls the magnitude of LoRA updates.
- `lora_dropout`: Dropout probability for LoRA layers (typical: 0.05-0.1).
- `target_modules`: Which modules to apply LoRA to. Common choices:
  - `["q_proj", "v_proj"]`: Attention query and value (memory efficient)
  - `["q_proj", "k_proj", "v_proj", "o_proj"]`: All attention projections
  - `["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]`: All linear layers
- `modules_to_save`: Additional modules to fully train (e.g., `["embed_tokens", "lm_head"]`)

### Target Module Selection

You can specify which modules to apply LoRA to. Common patterns:

```python

# Minimal (most memory efficient)

target_modules=["q_proj", "v_proj"]



# Attention only

target_modules=["q_proj", "k_proj", "v_proj", "o_proj"]



# All linear layers (best performance, more memory)

target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

```

### Using Command-Line Arguments

TRL scripts accept PEFT parameters via command line:

```bash

python trl/scripts/sft.py \

    --model_name_or_path Qwen/Qwen2-0.5B \

    --dataset_name trl-lib/Capybara \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16 \

    --lora_dropout 0.05 \

    --lora_target_modules q_proj v_proj \

    --output_dir output

```

Available flags:
- `--use_peft`: Enable PEFT
- `--lora_r`: LoRA rank (default: 16)
- `--lora_alpha`: LoRA alpha (default: 32)
- `--lora_dropout`: LoRA dropout (default: 0.05)
- `--lora_target_modules`: Target modules (space-separated)
- `--lora_modules_to_save`: Additional modules to train
- `--use_rslora`: Enable Rank-Stabilized LoRA
- `--use_dora`: Enable Weight-Decomposed LoRA (DoRA)
- `--load_in_4bit`: Enable 4-bit quantization (QLoRA)
- `--load_in_8bit`: Enable 8-bit quantization

## Saving and Loading PEFT Models

### Saving

After training, save your PEFT adapters:

```python

# Save the adapters

trainer.save_model("path/to/adapters")



# Or manually

model.save_pretrained("path/to/adapters")

```

This saves only the adapter weights (~few MB) rather than the full model (~several GB).

### Loading

Load a PEFT model for inference:

```python

from transformers import AutoModelForCausalLM

from peft import PeftModel



# Load base model

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B")



# Load PEFT adapters

model = PeftModel.from_pretrained(base_model, "path/to/adapters")



# Optionally merge adapters into base model for faster inference

model = model.merge_and_unload()

```

### Pushing to Hub

You can easily share your PEFT adapters on the Hugging Face Hub:

```python

# Push adapters to Hub

model.push_to_hub("username/model-name-lora")



# Load from Hub

from peft import PeftModel

model = PeftModel.from_pretrained(base_model, "username/model-name-lora")

```

## Multi-GPU Training

PEFT works seamlessly with TRL's multi-GPU support through `accelerate`:

```bash

# Configure accelerate

accelerate config



# Launch training

accelerate launch trl/scripts/sft.py \

    --model_name_or_path Qwen/Qwen2-0.5B \

    --dataset_name trl-lib/Capybara \

    --use_peft \

    --lora_r 32 \

    --lora_alpha 16

```

For QLoRA with multiple GPUs, the base model is automatically sharded:

```bash

accelerate launch trl/scripts/sft.py \

    --model_name_or_path meta-llama/Llama-2-70b-hf \

    --load_in_4bit \

    --use_peft \

    --lora_r 32

```

### Naive Pipeline Parallelism (NPP) for Large Models

For very large models (>60B parameters), TRL supports Naive Pipeline Parallelism (NPP), which distributes the model and adapters across multiple GPUs. The activations and gradients are communicated across GPUs, supporting both `int8` and other data types.

![NPP](https://huggingface.co/datasets/trl-lib/documentation-images/resolve/main/trl-npp.png)

**How to Use NPP**

Load your model with a custom `device_map` to split it across multiple devices:

```python

from transformers import AutoModelForCausalLM

from peft import LoraConfig



# Create custom device map (see accelerate documentation)

device_map = {

    "model.embed_tokens": 0,

    "model.layers.0": 0,

    # ... distribute layers across GPUs

    "lm_head": 0,  # Must be on GPU 0

}



model = AutoModelForCausalLM.from_pretrained(

    "meta-llama/Llama-2-70b-hf",

    device_map=device_map,

    peft_config=lora_config,

)

```

> [!IMPORTANT]
> - Keep the `lm_head` module on the first GPU (device 0) to avoid errors

> - See this [tutorial on device maps](https://github.com/huggingface/blog/blob/main/accelerate-large-models.md) for proper configuration

> - Run training scripts directly (not with `accelerate launch`): `python script.py`

> - Data Parallelism is not yet supported with NPP



## Resources



### TRL Examples and Notebooks



- **[SFT with LoRA/QLoRA Notebook](https://github.com/huggingface/trl/blob/main/examples/notebooks/sft_trl_lora_qlora.ipynb)** - Complete working example showing both LoRA and QLoRA implementations

- **[TRL Examples Directory](https://github.com/huggingface/trl/tree/main/examples)** - Collection of training scripts demonstrating PEFT with different trainers

- **[TRL Cookbook Recipes](https://github.com/huggingface/cookbook/tree/main/notebooks/transformers)** - Step-by-step guides for common PEFT training scenarios



### Documentation



- [PEFT Documentation](https://huggingface.co/docs/peft) - Official PEFT library documentation

- [TRL Documentation](https://huggingface.co/docs/trl) - Complete TRL documentation with trainer guides

- [LoRA Without Regret](lora_without_regret) - Best practices for using LoRA effectively



### Research Papers



- [LoRA Paper](https://huggingface.co/papers/2106.09685) - Original LoRA methodology and results

- [QLoRA Paper](https://huggingface.co/papers/2305.14314) - Efficient finetuning with 4-bit quantization

- [Prompt Tuning Paper](https://huggingface.co/papers/2104.08691) - The Power of Scale for Parameter-Efficient Prompt Tuning