docs/extra_packages.md

Extra Packages Required

This document lists additional packages needed for new features compared to the base v2 model that is already running.

For LoRA Support (legrconv4x_llava_lora.py)

Required Package:

pip install peft>=0.5.0

Purpose:

• PEFT (Parameter-Efficient Fine-Tuning) library from HuggingFace
• Provides LoRA implementation for efficient model adaptation
• Used in model/legrconv4x_llava_lora.py

Features Enabled:

• Low-rank adaptation of LLaVA language model
• Efficient fine-tuning with minimal trainable parameters
• Save/load LoRA weights separately
• Proper integration with LLaVA's causal LM architecture

Fixes Applied:

• Changed TaskType to CAUSAL_LM (matches LLaVA's base class)
• Removed redundant requires_grad loop (PEFT handles this)
• Removed eval() call to allow proper LoRA dropout during training
• Fixed load_adapter to use proper PEFT loading method

---

For Warmup Scheduler (already implemented)

No additional packages required

• Uses only PyTorch's built-in _LRScheduler class
• Implemented in utils/warmup_scheduler.py

---

For AdamW Optimizer (already implemented)

No additional packages required

• AdamW is included in PyTorch (torch.optim.AdamW)
• Available in PyTorch >= 1.2.0

---

For WandB Logging (already implemented)

Already installed in v2

• wandb package is already in the environment
• No additional installation needed

---

Installation Command for All Extra Packages:

# Activate your conda environment first
conda activate llmf2

# Install only the new requirements
pip install peft>=0.5.0