Delete ReadMe.md

ReadMe.md

@@ -1,133 +0,0 @@
-# Lab 2 - Fine-Tuning a Large Language Model
-
-## Overview
-
-In this lab we fine-tuned the open-source base model `unsloth/Llama-3.2-3B-Instruct` on Maxime Labonne's FineTome-100k instruction dataset using LoRA (Low-Rank Adaptation) with QLoRA quantization via Unsloth and `trl.SFTTrainer`. The training used the following hyperparameters:
-
-| Hyperparameter | Value |
-|----------------|-------|
-| `per_device_train_batch_size` | 2 |
-| `gradient_accumulation_steps` | 4 |
-| `num_train_epochs` | 1 |
-| `learning_rate` | 2e-4 |
-| `warmup_steps` | 5 |
-| `weight_decay` | 0.01 |
-| `optimizer` | adamw_8bit |
-
----
-
-## Evaluation Methodology
-
-We evaluated both the base model and the fine-tuned model on **100 held-out examples** from a 15% test split of FineTome-100k. For each example we used the conversation history as input and treated the last assistant turn as the reference answer.
-
-### Metrics Used
-
-We used **ROUGE scores** (Recall-Oriented Understudy for Gisting Evaluation) to measure the overlap between model-generated responses and reference answers:
-
-- **ROUGE-1**: Measures unigram (single word) overlap between the generated text and reference. Higher scores indicate better word-level similarity.
-- **ROUGE-2**: Measures bigram (two consecutive words) overlap. This captures phrase-level similarity and is more sensitive to word order.
-- **ROUGE-L**: Measures the longest common subsequence between generated and reference text. This captures sentence-level structure and fluency.
-
-All scores use the F-measure (harmonic mean of precision and recall) and range from 0 to 1, where higher is better.
-
-### Generation Settings
-
-We used **deterministic generation** (greedy decoding with `do_sample=False`) for both models to ensure reproducible and comparable results. Each response was limited to 256 new tokens.
-
----
-
-## Results
-
-The fine-tuned model clearly outperformed the base model across all ROUGE metrics:
-
-| Metric | Base Model | Fine-Tuned Model | Improvement |
-|--------|------------|------------------|-------------|
-| ROUGE-1 | 0.4732 | 0.5323 | **+12.5%** |
-| ROUGE-2 | 0.2255 | 0.2849 | **+26.4%** |
-| ROUGE-L | 0.2856 | 0.3521 | **+23.3%** |
-
-These results demonstrate that even a single epoch of LoRA fine-tuning on FineTome-100k produces substantial improvements in response quality. The largest gain was in ROUGE-2 (+26.4%), indicating that the fine-tuned model better captures phrase-level patterns from the training data.
-
-### Response Length Analysis
-
-| Metric | Base Model | Fine-Tuned Model | Reference |
-|--------|------------|------------------|-----------|
-| Mean length (words) | 165.0 | 155.7 | 216.3 |
-| Median length (words) | 176.0 | 165.5 | 199.0 |
-
-The fine-tuned model produces slightly more concise responses while achieving higher ROUGE scores, suggesting improved information density.
-
----
-
-## Improving Model Performance
-
-### (a) Model-Centric Approach
-
-A model-centric approach keeps the data fixed and focuses on changing the model architecture, training configuration, or optimization procedure. Below are concrete strategies for further improvement:
-
-#### Hyperparameter Tuning
-
-- **Learning rate**: Sweep over values such as `1e-4`, `2e-4`, `5e-4` to find the optimal learning rate. Our current setting of `2e-4` is a reasonable default but may not be optimal.
-- **Learning rate schedule**: Experiment with cosine decay or cosine with warm restarts instead of constant learning rate.
-- **Training epochs**: Train for 2–3 epochs with early stopping based on validation loss to potentially improve convergence.
-- **Batch size**: Increase effective batch size (via `gradient_accumulation_steps`) within memory constraints for more stable gradients.
-- **Warmup steps**: Adjust warmup duration (e.g., 10–100 steps) to improve training stability.
-- **Weight decay**: Test different regularization strengths (e.g., `0.001`, `0.01`, `0.1`) to control overfitting.
-
-#### LoRA Configuration
-
-- **Rank (r)**: Increase LoRA rank (e.g., from 16 to 32 or 64) to allow more expressive adapter updates, at the cost of increased memory.
-- **Alpha scaling**: Adjust the LoRA alpha parameter to control the magnitude of adapter contributions.
-- **Target modules**: Experiment with applying LoRA to different layer types (attention only, MLP layers, or both) and different layer ranges.
-
-#### Model Architecture
-
-- **Base model selection**: Compare different foundation models such as `Llama-3.2-1B-Instruct` (faster inference) or `Llama-3.1-8B-Instruct` (potentially higher quality but slower).
-- **Quantization**: Compare 4-bit (QLoRA) vs 8-bit quantization to understand the quality-speed tradeoff.
-
-#### Training Procedure
-
-- **Gradient clipping**: Add gradient clipping to prevent exploding gradients and improve training stability.
-- **Mixed precision**: Ensure optimal use of mixed precision training for faster iteration.
-
-### (b) Data-Centric Approach
-
-A data-centric approach keeps the model and training loop mostly fixed and focuses on improving or extending the training data. Below are concrete strategies:
-
-#### Data Quality Improvements
-
-- **Filter low-quality examples**: Remove very short, unclear, or noisy instruction-response pairs from FineTome-100k to increase average signal per batch.
-- **Deduplicate**: Remove near-duplicate examples that may cause the model to overfit to specific patterns.
-- **Balance task types**: If the target application focuses on specific capabilities (e.g., reasoning, coding, explanation), up-sample those categories and down-sample less relevant ones.
-
-#### Additional Data Sources
-
-Augment FineTome-100k with other high-quality open-source instruction datasets:
-
-| Dataset | Focus Area | Potential Benefit |
-|---------|------------|-------------------|
-| OpenAssistant Conversations | Multi-turn dialogue | Improved conversational ability |
-| GSM8K / MetaMath | Math reasoning | Better mathematical problem-solving |
-| CodeAlpaca / Code-Feedback | Programming tasks | Improved code generation |
-| FLAN Collection | Diverse NLP tasks | Broader task coverage |
-| UltraChat | Long-form dialogue | Better handling of extended conversations |
-
-#### Domain-Specific Fine-Tuning
-
-- **Curriculum learning**: Start training on general instructions, then gradually shift to more specialized or difficult examples.
-- **Task-specific adapters**: Train separate LoRA adapters for different domains (math, code, creative writing) and select the appropriate adapter at inference time.
-
-#### Data Alignment
-
-- **Match UI format**: If the final application expects specific output formats (e.g., step-by-step reasoning, JSON responses), construct or filter training examples that demonstrate these formats.
-- **User feedback loop**: In production, log anonymized user interactions (if permitted) to create a fine-tuning set that reflects real usage patterns.
-
----
-
-## Conclusion
-
-Our fine-tuning pipeline demonstrates measurable improvements over the base model, with ROUGE scores increasing by 12–26% on a held-out test set. The model-centric and data-centric strategies outlined above provide clear directions for further performance gains. The most promising next steps would be:
-
-1. **Hyperparameter sweep** on learning rate and number of epochs
-2. **Increase LoRA rank** to allow more expressive updates
-3. **Mix in domain-specific datasets** (e.g., math reasoning or code) to improve performance on specialized tasks