docs/learning_roadmap.md

# CampusGPT Uganda — Learning Roadmap & One-Day Mastery Checklist
## Complete Unsloth Fine-Tuning Ecosystem in One Project

---

## Your Learning Journey

By building CampusGPT Uganda, you have touched every major concept in the
Unsloth ecosystem. This document is your personal mastery checklist and
study guide.

---

## Phase 1: Foundations (Morning — 2 hours)

### ✅ Concept 1: What is Unsloth?

**You learned:**
- Unsloth is a Python library for faster LLM fine-tuning
- It achieves 2× speed + 80% memory reduction through:
  - Custom CUDA kernels (C-level code for attention + FFN)
  - Manual backpropagation (no storing redundant activations)
  - Smart memory layout (packed data, minimal fragmentation)
- Same accuracy as HuggingFace — just faster

**Key file:** `training/finetune.py` — top docstring + `load_model_and_tokenizer()`

**Self-test:** Can you explain why Unsloth is faster to a classmate?

---

### ✅ Concept 2: LoRA — Low-Rank Adaptation

**You learned:**
- Full fine-tuning modifies all 8 billion parameters → impractical
- LoRA adds two small matrices A (m×r) and B (r×n) to each weight
- Only A and B are trained (1–5% of total parameters)
- The adapted weight: W' = W + (A × B) × (alpha/r)
- r=16 is the recommended default for instruction tuning

**Key file:** `training/finetune.py` — `add_lora_adapters()`

**Formula to remember:**
```
W' = W_frozen + (A_trainable × B_trainable) × scaling_factor
```

**Self-test:** Why does LoRA use TWO matrices (A and B) instead of one?

*Answer: A single low-rank update matrix is the product of two smaller matrices.
The product A×B has rank ≤ r, achieving compression. A single matrix of rank r
cannot be parameterised as efficiently.*

---

### ✅ Concept 3: QLoRA — Quantised LoRA

**You learned:**
- Quantisation stores weights in fewer bits: float32 (4 bytes) → int4 (0.5 bytes)
- QLoRA = 4-bit quantised base model + full-precision LoRA adapters
- Llama-3 8B: 32 GB (fp32) → ~5 GB (QLoRA) — fits on a T4 GPU!
- NF4 (NormalFloat4) is used — better distribution for neural network weights
- ~1-2% quality loss vs full fine-tuning — totally acceptable for most tasks

**Key file:** `training/finetune.py` — `load_in_4bit=True` parameter

**Memory calculation:**
```
Model size (GB) ≈ (parameters × bits) / (8 × 1024³)
Llama 8B in 4-bit ≈ (8 × 10⁹ × 4) / (8 × 1024³) ≈ 4 GB
```

---

### ✅ Concept 4: PEFT — Parameter-Efficient Fine-Tuning

**You learned:**
- PEFT is the HuggingFace library managing adapters
- `get_peft_model()` attaches LoRA to the base model
- After calling it: base weights FROZEN, only adapter weights trainable
- Adapter weights are ~10–100 MB (vs full model ~6–16 GB)

**Key file:** `training/finetune.py` — `add_lora_adapters()`

---

## Phase 2: Data Engineering (Mid-Morning — 2 hours)

### ✅ Concept 5: Dataset Formats

**You learned the 4 major formats:**

| Format | Fields | Use Case |
|--------|--------|----------|
| JSONL | question, answer | Simple storage |
| Alpaca | instruction, input, output | General instruction tuning |
| OpenAI Messages | [{role, content}] | Chat models (MOST COMMON) |
| ShareGPT | conversations:[{from, value}] | Multi-turn chat |

**Key file:** `data/generate_dataset.py` — all `to_*_format()` functions

**Self-test:** Convert this Q&A to all 4 formats by hand:
- Q: "What is the attendance policy?"
- A: "Students must attend 75% of lectures."

---

### ✅ Concept 6: Tokenisation

**You learned:**
- Text → Tokens → Input IDs (integers)
- "MUST university" → ["MUST", " university"] → [123, 456]
- Every model has its OWN vocabulary — ALWAYS use the model's tokeniser
- Chat templates format conversations with model-specific special tokens
- `apply_chat_template()` handles this automatically

**Key file:** `training/finetune.py` — `load_and_prepare_dataset()`

**Qwen chat template example:**
```
<|im_start|>system
You are CampusGPT...<|im_end|>
<|im_start|>user
What are the fees?<|im_end|>
<|im_start|>assistant
The fees are...<|im_end|>
```

---

### ✅ Concept 7: Synthetic Data Generation

**You learned:**
- Read PDFs → Extract text → Chunk → LLM generates QA pairs
- Deduplication with Jaccard similarity (n-gram overlap)
- Quality filtering (minimum length, must be a question, no non-answers)
- Key insight: LLMs can generate their own training data!

**Key file:** `data/synthetic_generator.py`

**Pipeline:**
```
PDF → pypdf → chunks of 500 words → LLM prompt → JSON output → dedup → filter → JSONL
```

---

## Phase 3: RAG System (Late Morning — 2 hours)

### ✅ Concept 8: Vector Embeddings

**You learned:**
- Text → dense vector of floats (768–1024 dimensions)
- Similar meaning → similar vector (close in vector space)
- Cosine similarity measures angle between vectors (1.0 = identical)
- Models: BGE-M3 (multilingual, best for Uganda), MiniLM (fast), Qwen (highest quality)

**Key file:** `embeddings/embedding_pipeline.py`

**Key formula:**
```
cos_similarity(A, B) = (A · B) / (|A| × |B|)
Range: 0 (different) to 1 (identical meaning)
```

---

### ✅ Concept 9: RAG — Retrieval-Augmented Generation

**You learned the complete pipeline:**
```
Student Question
       ↓
  Embed Query → [0.23, -0.11, ...]
       ↓
  ChromaDB (vector similarity search)
       ↓
  Top-3 relevant document passages
       ↓
  Prompt = System + Retrieved Context + Question
       ↓
  LLM generates grounded answer
       ↓
  Response + Source Citations
```

**Why RAG is better than pure fine-tuning for some tasks:**
- Can handle new documents without retraining
- Answers are grounded in retrieved text (less hallucination)
- Can cite sources (important for academic contexts)

**Key file:** `rag/rag_pipeline.py`

---

### ✅ Concept 10: ChromaDB

**You learned:**
- ChromaDB = vector database (stores text + embeddings + metadata)
- HNSW index: approximate nearest-neighbour in O(log n) time
- Cosine distance space: closer = more similar
- Persistent: data survives between sessions

---

## Phase 4: Fine-Tuning (Afternoon — 3 hours)

### ✅ Concept 11: SFT — Supervised Fine-Tuning

**You learned:**
- Show the model thousands of (question, answer) pairs
- Train it to predict the correct answer given the question
- Loss computed ONLY on assistant tokens (not system/user tokens)
- SFTTrainer from TRL handles everything automatically

**Key file:** `training/trainer.py` and `training/finetune.py`

---

### ✅ Concept 12: Key Hyperparameters

| Parameter | Recommended | Effect |
|-----------|-------------|--------|
| r | 16 | LoRA capacity |
| lora_alpha | 32 | Update strength |
| learning_rate | 2e-4 | Step size |
| batch_size | 2 | Examples per step |
| grad_accum | 4 | Effective batch = 8 |
| epochs | 3 | Dataset passes |
| warmup_steps | 20 | Gentle start |
| lr_scheduler | cosine | Decay shape |

**Key file:** `training/hyperparams.py`

**Diagnostic rules:**
- Loss explodes → halve learning rate
- Loss not decreasing → double learning rate
- Overfitting → add dropout, reduce epochs
- OOM → reduce batch size or r

---

## Phase 5: Model Export (Late Afternoon — 1 hour)

### ✅ Concept 13: Three Export Options

| Format | Size | Use Case |
|--------|------|----------|
| LoRA adapters | ~50 MB | Development, sharing adapters |
| Merged 16-bit | ~6-16 GB | Production, vLLM |
| GGUF (Q4_K_M) | ~2-4 GB | Ollama, local inference |

**Key file:** `training/export.py`

**Merge operation:** `W' = W + (A × B) × (alpha/r)`

---

## Phase 6: Deployment (Evening — 2 hours)

### ✅ Concept 14: Ollama Deployment

**You learned:**
- Ollama wraps GGUF with a clean REST API
- Modelfile = LLM configuration (like a Dockerfile)
- OpenAI-compatible API: change base_url, same code works
- `ollama run campusgpt` to chat locally

**Key files:** `deployment/ollama/`

---

### ✅ Concept 15: vLLM Deployment

**You learned:**
- PagedAttention: allocates KV cache in pages on demand
- Continuous batching: new requests join mid-generation
- 10-100× higher throughput than Ollama for multiple concurrent users
- For university servers serving hundreds of students

**Key file:** `deployment/vllm/deploy_vllm.py`

---

### ✅ Concept 16: HuggingFace Hub

**You learned:**
- Upload LoRA adapter, merged model, or GGUF
- Model Card = documentation for your model
- HuggingFace Spaces = free Gradio demo hosting
- Ollama can pull directly from HuggingFace: `ollama pull username/campusgpt`

**Key file:** `deployment/huggingface/push_to_hub.py`

---

### ✅ Concept 17: Evaluation

**You learned four metrics:**

| Metric | What it measures | Range |
|--------|-----------------|-------|
| ROUGE-L | Word overlap (recall) | 0–1 |
| BLEU | N-gram precision | 0–1 |
| BERTScore | Semantic similarity | 0–1 |
| Hallucination Rate | Key fact recall | 0–1 (lower=better) |

**Key file:** `evaluation/evaluate_model.py`

**The key experiment:** Base model vs Fine-tuned model comparison.
If fine-tuning worked, you should see:
- Higher ROUGE-L (+0.05 to +0.2)
- Higher BERTScore (+0.02 to +0.1)
- Lower hallucination rate (-0.05 to -0.2)

---

## ONE-DAY UNSLOTH MASTERY CHECKLIST

Check off each item as you complete it:

### 🌅 Morning (Hours 1–3)
- [ ] Read `README.md` and understand the full architecture
- [ ] Run `python data/generate_dataset.py` — generate 3100 training examples
- [ ] Run `python data/synthetic_generator.py` — understand PDF-to-QA pipeline
- [ ] Open `data/processed/combined_all_openai_messages.jsonl` — inspect the data
- [ ] Open `data/processed/combined_all_alpaca.jsonl` — compare formats

### 🌤️ Late Morning (Hours 3–5)
- [ ] Read `training/finetune.py` top docstring — understand LoRA vs QLoRA
- [ ] Study `LORA_CONFIG` in `finetune.py` — understand each parameter
- [ ] Study `TRAINING_CONFIG` — understand each training argument
- [ ] Run `python training/hyperparams.py` — get hardware recommendation
- [ ] (Optional) Run `python embeddings/embedding_pipeline.py` — see semantic search

### ☀️ Afternoon (Hours 5–8) — REQUIRES GPU
- [ ] Open Google Colab (colab.research.google.com) with T4 GPU
- [ ] Install Unsloth: `!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"`
- [ ] Copy `training/finetune.py` to Colab
- [ ] Run training — watch the loss decrease
- [ ] Note peak GPU memory usage
- [ ] Run `training/export.py` — export LoRA adapters

### 🌆 Late Afternoon (Hours 8–10)
- [ ] Run `python rag/rag_pipeline.py --build` — build knowledge base
- [ ] Run `python rag/rag_pipeline.py` — test RAG queries
- [ ] Run `python inference/chat.py` — chat with your model
- [ ] Compare base model vs fine-tuned model responses manually

### 🌙 Evening (Hours 10–12)
- [ ] Run `python evaluation/evaluate_model.py` — compute metrics
- [ ] Run `python frontend/app.py` — launch the Gradio UI
- [ ] Try uploading a PDF in the UI
- [ ] Run `bash deployment/ollama/deploy_ollama.sh` — deploy locally
- [ ] (Optional) Push to HuggingFace Hub

---

## Concepts Mastered After This Project

After completing CampusGPT Uganda, you understand:

1. ✅ **Unsloth** — what it is and why it's faster
2. ✅ **LoRA** — low-rank adaptation mathematics
3. ✅ **QLoRA** — quantisation + LoRA for memory efficiency
4. ✅ **PEFT** — parameter-efficient fine-tuning library
5. ✅ **Dataset formats** — JSONL, Alpaca, OpenAI Messages, ShareGPT
6. ✅ **Tokenisation** — text to tokens to input IDs
7. ✅ **Chat templates** — model-specific conversation formatting
8. ✅ **SFT** — supervised fine-tuning with SFTTrainer
9. ✅ **Evaluation** — ROUGE, BERTScore, hallucination metrics
10. ✅ **Inference** — streaming, temperature, top-p sampling
11. ✅ **RAG** — retrieval-augmented generation end-to-end
12. ✅ **Vector embeddings** — semantic search with BGE/Qwen
13. ✅ **ChromaDB** — vector store, HNSW, cosine similarity
14. ✅ **Model export** — LoRA adapters, merged model, GGUF
15. ✅ **Ollama** — local deployment with Modelfile
16. ✅ **vLLM** — production serving with PagedAttention
17. ✅ **HuggingFace Hub** — model sharing and demo hosting
18. ✅ **Gradio** — building ML web UIs
19. ✅ **Hyperparameter tuning** — what each parameter does and how to tune it
20. ✅ **Synthetic data generation** — PDF → QA pairs with LLMs

---

## What to Build Next

Now that you understand the full stack, here are next project ideas:

1. **Luganda-English Translation Fine-tune** — Use your NLLB/Qwen pipeline + QLoRA
2. **Medical QA for Uganda** — Fine-tune on Ugandan health guidelines
3. **Legal AI for Uganda** — Fine-tune on Ugandan law documents
4. **AgriBot** — Combine with your PotatoGuard project + LLM
5. **Multi-university CampusGPT** — Expand to Makerere, KIU, UCU
6. **Voice CampusGPT** — Add speech-to-text (Whisper) + text-to-speech

---

## Resources

- **Unsloth docs:** https://docs.unsloth.ai
- **Unsloth GitHub:** https://github.com/unslothai/unsloth
- **TRL docs:** https://huggingface.co/docs/trl
- **ChromaDB docs:** https://docs.trychroma.com
- **Gradio docs:** https://www.gradio.app/docs
- **vLLM docs:** https://docs.vllm.ai
- **BGE-M3 paper:** https://arxiv.org/abs/2402.03216
- **QLoRA paper:** https://arxiv.org/abs/2305.14314

---

*Built by Joseph Ssemuli, Computer Science student at MUST, during Sunbird AI internship.*
*This project is your evidence of mastering the complete Unsloth ecosystem.* 🇺🇬