FINETUNING.md

Fine-tuning Guide for Reply Generation Model

Overview

This guide explains how to fine-tune a language model to generate conversational replies based on conversation context, trigger, and move patterns.

Prerequisites

1. Dataset: new_data_selected.csv with columns:

   • user_text: Male's message
   • partner_text: Female's response
   • trigger_*: Binary columns for triggers
   • move_*: Binary columns for moves

2. Hardware: GPU recommended (CUDA-compatible) for faster training

3. Dependencies: All required packages are in requirements.txt

Step 1: Prepare Dataset

Place your dataset CSV file in the project root:

cp /path/to/new_data_selected.csv /Users/paul/Documents/paul/huggingface/lovebird25/

Step 2: Run Fine-tuning

Basic Usage

cd /Users/paul/Documents/paul/huggingface/lovebird25
python finetune_model.py --data_path new_data_selected.csv --output_dir ./finetuned_reply_model

Advanced Options

python finetune_model.py \
    --data_path new_data_selected.csv \
    --output_dir ./finetuned_reply_model \
    --model_name vinai/PhoGPT-4B-Chat \
    --num_epochs 3 \
    --batch_size 2 \
    --learning_rate 2e-4 \
    --use_history

Parameters

• --data_path: Path to training data CSV (default: new_data_selected.csv)
• --output_dir: Output directory for fine-tuned model (default: ./finetuned_reply_model)
• --model_name: Base model name (default: vinai/PhoGPT-4B-Chat)
• --num_epochs: Number of training epochs (default: 3)
• --batch_size: Training batch size (default: 2, adjust based on GPU memory)
• --learning_rate: Learning rate (default: 2e-4)
• --use_history: Use conversation history in training

Step 3: Use Fine-tuned Model

After fine-tuning, update app.py or reply_service.py to use the fine-tuned model:

from finetuned_reply_service import get_finetuned_reply_service

# Get service
service = get_finetuned_reply_service(
    base_model_name="vinai/PhoGPT-4B-Chat",
    finetuned_model_path="./finetuned_reply_model"
)

# Generate reply
reply = service.generate_reply(
    user_text="Tối nay anh có lịch đột xuất",
    partner_text="Thế mai được không?",
    trigger="invite_propose",
    move="invite"
)

Step 4: Deploy to Hugging Face Spaces

Option 1: Upload Fine-tuned Model to Hugging Face Hub

1. Login to Hugging Face:

huggingface-cli login

2. Upload model:

cd finetuned_reply_model
huggingface-cli upload your-username/your-model-name .

3. Update code to load from Hub:

service = get_finetuned_reply_service(
    base_model_name="vinai/PhoGPT-4B-Chat",
    finetuned_model_path="your-username/your-model-name"  # Load from Hub
)

Option 2: Include Model in Space

1. Add model files to Space:

   • Upload finetuned_reply_model/ folder to your Space
   • Or use Git LFS for large files

2. Update app.py to use fine-tuned model:

from finetuned_reply_service import get_finetuned_reply_service

# In predict_reply function, replace:
# reply_service = get_reply_service()
# with:
finetuned_service = get_finetuned_reply_service()
suggestion = finetuned_service.generate_reply(
    male=male,
    female=female,
    tone=top_tone,
    intent=top_intent  # Use as trigger/move
)

Training Tips

1. GPU Memory: If you run out of memory, reduce --batch_size or use gradient checkpointing

2. Training Time:

   • Small dataset (< 1000 examples): 1-2 epochs may be enough
   • Large dataset (> 5000 examples): 3-5 epochs recommended

3. Learning Rate:

   • Start with 2e-4
   • If loss doesn't decrease, try 1e-4 or 5e-4

4. Validation: Monitor eval_loss to avoid overfitting

Troubleshooting

Out of Memory Error

• Reduce --batch_size to 1
• Reduce max_length in tokenization (currently 512)

Model Not Found

• Ensure base model name is correct
• Check Hugging Face token is set: export HF_TOKEN=your_token

Training Too Slow

• Use GPU if available
• Reduce dataset size for testing
• Use smaller base model

File Structure After Fine-tuning

lovebird25/
├── finetuned_reply_model/
│   ├── adapter_config.json
│   ├── adapter_model.bin
│   └── ...
├── finetune_model.py
├── finetuned_reply_service.py
└── ...

Next Steps

1. Evaluate model on test set
2. Fine-tune hyperparameters if needed
3. Deploy to Hugging Face Spaces
4. Integrate into production app