docs/learning_roadmap.md

CampusGPT Uganda — Learning Roadmap & One-Day Mastery Checklist

Complete Unsloth Fine-Tuning Ecosystem in One Project

---

Your Learning Journey

By building CampusGPT Uganda, you have touched every major concept in the Unsloth ecosystem. This document is your personal mastery checklist and study guide.

---

Phase 1: Foundations (Morning — 2 hours)

✅ Concept 1: What is Unsloth?

You learned:

• Unsloth is a Python library for faster LLM fine-tuning
• It achieves 2× speed + 80% memory reduction through:
  • Custom CUDA kernels (C-level code for attention + FFN)
  • Manual backpropagation (no storing redundant activations)
  • Smart memory layout (packed data, minimal fragmentation)
• Same accuracy as HuggingFace — just faster

Key file: training/finetune.py — top docstring + load_model_and_tokenizer()

Self-test: Can you explain why Unsloth is faster to a classmate?

---

✅ Concept 2: LoRA — Low-Rank Adaptation

You learned:

• Full fine-tuning modifies all 8 billion parameters → impractical
• LoRA adds two small matrices A (m×r) and B (r×n) to each weight
• Only A and B are trained (1–5% of total parameters)
• The adapted weight: W' = W + (A × B) × (alpha/r)
• r=16 is the recommended default for instruction tuning

Key file: training/finetune.py — add_lora_adapters()

Formula to remember:

W' = W_frozen + (A_trainable × B_trainable) × scaling_factor

Self-test: Why does LoRA use TWO matrices (A and B) instead of one?

Answer: A single low-rank update matrix is the product of two smaller matrices. The product A×B has rank ≤ r, achieving compression. A single matrix of rank r cannot be parameterised as efficiently.

---

✅ Concept 3: QLoRA — Quantised LoRA

You learned:

• Quantisation stores weights in fewer bits: float32 (4 bytes) → int4 (0.5 bytes)
• QLoRA = 4-bit quantised base model + full-precision LoRA adapters
• Llama-3 8B: 32 GB (fp32) → ~5 GB (QLoRA) — fits on a T4 GPU!
• NF4 (NormalFloat4) is used — better distribution for neural network weights
• ~1-2% quality loss vs full fine-tuning — totally acceptable for most tasks

Key file: training/finetune.py — load_in_4bit=True parameter

Memory calculation:

Model size (GB) ≈ (parameters × bits) / (8 × 1024³)
Llama 8B in 4-bit ≈ (8 × 10⁹ × 4) / (8 × 1024³) ≈ 4 GB

---

✅ Concept 4: PEFT — Parameter-Efficient Fine-Tuning

You learned:

• PEFT is the HuggingFace library managing adapters
• get_peft_model() attaches LoRA to the base model
• After calling it: base weights FROZEN, only adapter weights trainable
• Adapter weights are ~10–100 MB (vs full model ~6–16 GB)

Key file: training/finetune.py — add_lora_adapters()

---

Phase 2: Data Engineering (Mid-Morning — 2 hours)

✅ Concept 5: Dataset Formats

You learned the 4 major formats:

Format	Fields	Use Case
JSONL	question, answer	Simple storage
Alpaca	instruction, input, output	General instruction tuning
OpenAI Messages	[{role, content}]	Chat models (MOST COMMON)
ShareGPT	conversations:[{from, value}]	Multi-turn chat

Key file: data/generate_dataset.py — all to_*_format() functions

Self-test: Convert this Q&A to all 4 formats by hand:

• Q: "What is the attendance policy?"
• A: "Students must attend 75% of lectures."

---

✅ Concept 6: Tokenisation

You learned:

• Text → Tokens → Input IDs (integers)
• "MUST university" → ["MUST", " university"] → [123, 456]
• Every model has its OWN vocabulary — ALWAYS use the model's tokeniser
• Chat templates format conversations with model-specific special tokens
• apply_chat_template() handles this automatically

Key file: training/finetune.py — load_and_prepare_dataset()

Qwen chat template example:

<|im_start|>system
You are CampusGPT...<|im_end|>
<|im_start|>user
What are the fees?<|im_end|>
<|im_start|>assistant
The fees are...<|im_end|>

---

✅ Concept 7: Synthetic Data Generation

You learned:

• Read PDFs → Extract text → Chunk → LLM generates QA pairs
• Deduplication with Jaccard similarity (n-gram overlap)
• Quality filtering (minimum length, must be a question, no non-answers)
• Key insight: LLMs can generate their own training data!

Key file: data/synthetic_generator.py

Pipeline:

PDF → pypdf → chunks of 500 words → LLM prompt → JSON output → dedup → filter → JSONL

---

Phase 3: RAG System (Late Morning — 2 hours)

✅ Concept 8: Vector Embeddings

You learned:

• Text → dense vector of floats (768–1024 dimensions)
• Similar meaning → similar vector (close in vector space)
• Cosine similarity measures angle between vectors (1.0 = identical)
• Models: BGE-M3 (multilingual, best for Uganda), MiniLM (fast), Qwen (highest quality)

Key file: embeddings/embedding_pipeline.py

Key formula:

cos_similarity(A, B) = (A · B) / (|A| × |B|)
Range: 0 (different) to 1 (identical meaning)

---

✅ Concept 9: RAG — Retrieval-Augmented Generation

You learned the complete pipeline:

Student Question
       ↓
  Embed Query → [0.23, -0.11, ...]
       ↓
  ChromaDB (vector similarity search)
       ↓
  Top-3 relevant document passages
       ↓
  Prompt = System + Retrieved Context + Question
       ↓
  LLM generates grounded answer
       ↓
  Response + Source Citations

Why RAG is better than pure fine-tuning for some tasks:

• Can handle new documents without retraining
• Answers are grounded in retrieved text (less hallucination)
• Can cite sources (important for academic contexts)

Key file: rag/rag_pipeline.py

---

✅ Concept 10: ChromaDB

You learned:

• ChromaDB = vector database (stores text + embeddings + metadata)
• HNSW index: approximate nearest-neighbour in O(log n) time
• Cosine distance space: closer = more similar
• Persistent: data survives between sessions

---

Phase 4: Fine-Tuning (Afternoon — 3 hours)

✅ Concept 11: SFT — Supervised Fine-Tuning

You learned:

• Show the model thousands of (question, answer) pairs
• Train it to predict the correct answer given the question
• Loss computed ONLY on assistant tokens (not system/user tokens)
• SFTTrainer from TRL handles everything automatically

Key file: training/trainer.py and training/finetune.py

---

✅ Concept 12: Key Hyperparameters

Parameter	Recommended	Effect
r	16	LoRA capacity
lora_alpha	32	Update strength
learning_rate	2e-4	Step size
batch_size	2	Examples per step
grad_accum	4	Effective batch = 8
epochs	3	Dataset passes
warmup_steps	20	Gentle start
lr_scheduler	cosine	Decay shape

Key file: training/hyperparams.py

Diagnostic rules:

• Loss explodes → halve learning rate
• Loss not decreasing → double learning rate
• Overfitting → add dropout, reduce epochs
• OOM → reduce batch size or r

---

Phase 5: Model Export (Late Afternoon — 1 hour)

✅ Concept 13: Three Export Options

Format	Size	Use Case
LoRA adapters	~50 MB	Development, sharing adapters
Merged 16-bit	~6-16 GB	Production, vLLM
GGUF (Q4_K_M)	~2-4 GB	Ollama, local inference

Key file: training/export.py

Merge operation: W' = W + (A × B) × (alpha/r)

---

Phase 6: Deployment (Evening — 2 hours)

✅ Concept 14: Ollama Deployment

You learned:

• Ollama wraps GGUF with a clean REST API
• Modelfile = LLM configuration (like a Dockerfile)
• OpenAI-compatible API: change base_url, same code works
• ollama run campusgpt to chat locally

Key files: deployment/ollama/

---

✅ Concept 15: vLLM Deployment

You learned:

• PagedAttention: allocates KV cache in pages on demand
• Continuous batching: new requests join mid-generation
• 10-100× higher throughput than Ollama for multiple concurrent users
• For university servers serving hundreds of students

Key file: deployment/vllm/deploy_vllm.py

---

✅ Concept 16: HuggingFace Hub

You learned:

• Upload LoRA adapter, merged model, or GGUF
• Model Card = documentation for your model
• HuggingFace Spaces = free Gradio demo hosting
• Ollama can pull directly from HuggingFace: ollama pull username/campusgpt

Key file: deployment/huggingface/push_to_hub.py

---

✅ Concept 17: Evaluation

You learned four metrics:

Metric	What it measures	Range
ROUGE-L	Word overlap (recall)	0–1
BLEU	N-gram precision	0–1
BERTScore	Semantic similarity	0–1
Hallucination Rate	Key fact recall	0–1 (lower=better)

Key file: evaluation/evaluate_model.py

The key experiment: Base model vs Fine-tuned model comparison. If fine-tuning worked, you should see:

• Higher ROUGE-L (+0.05 to +0.2)
• Higher BERTScore (+0.02 to +0.1)
• Lower hallucination rate (-0.05 to -0.2)

---

ONE-DAY UNSLOTH MASTERY CHECKLIST

Check off each item as you complete it:

🌅 Morning (Hours 1–3)

• Read README.md and understand the full architecture
• Run python data/generate_dataset.py — generate 3100 training examples
• Run python data/synthetic_generator.py — understand PDF-to-QA pipeline
• Open data/processed/combined_all_openai_messages.jsonl — inspect the data
• Open data/processed/combined_all_alpaca.jsonl — compare formats

🌤️ Late Morning (Hours 3–5)

• Read training/finetune.py top docstring — understand LoRA vs QLoRA
• Study LORA_CONFIG in finetune.py — understand each parameter
• Study TRAINING_CONFIG — understand each training argument
• Run python training/hyperparams.py — get hardware recommendation
• (Optional) Run python embeddings/embedding_pipeline.py — see semantic search

☀️ Afternoon (Hours 5–8) — REQUIRES GPU

• Open Google Colab (colab.research.google.com) with T4 GPU
• Install Unsloth: !pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
• Copy training/finetune.py to Colab
• Run training — watch the loss decrease
• Note peak GPU memory usage
• Run training/export.py — export LoRA adapters

🌆 Late Afternoon (Hours 8–10)

• Run python rag/rag_pipeline.py --build — build knowledge base
• Run python rag/rag_pipeline.py — test RAG queries
• Run python inference/chat.py — chat with your model
• Compare base model vs fine-tuned model responses manually

🌙 Evening (Hours 10–12)

• Run python evaluation/evaluate_model.py — compute metrics
• Run python frontend/app.py — launch the Gradio UI
• Try uploading a PDF in the UI
• Run bash deployment/ollama/deploy_ollama.sh — deploy locally
• (Optional) Push to HuggingFace Hub

---

Concepts Mastered After This Project

After completing CampusGPT Uganda, you understand:

1. ✅ Unsloth — what it is and why it's faster
2. ✅ LoRA — low-rank adaptation mathematics
3. ✅ QLoRA — quantisation + LoRA for memory efficiency
4. ✅ PEFT — parameter-efficient fine-tuning library
5. ✅ Dataset formats — JSONL, Alpaca, OpenAI Messages, ShareGPT
6. ✅ Tokenisation — text to tokens to input IDs
7. ✅ Chat templates — model-specific conversation formatting
8. ✅ SFT — supervised fine-tuning with SFTTrainer
9. ✅ Evaluation — ROUGE, BERTScore, hallucination metrics
10. ✅ Inference — streaming, temperature, top-p sampling
11. ✅ RAG — retrieval-augmented generation end-to-end
12. ✅ Vector embeddings — semantic search with BGE/Qwen
13. ✅ ChromaDB — vector store, HNSW, cosine similarity
14. ✅ Model export — LoRA adapters, merged model, GGUF
15. ✅ Ollama — local deployment with Modelfile
16. ✅ vLLM — production serving with PagedAttention
17. ✅ HuggingFace Hub — model sharing and demo hosting
18. ✅ Gradio — building ML web UIs
19. ✅ Hyperparameter tuning — what each parameter does and how to tune it
20. ✅ Synthetic data generation — PDF → QA pairs with LLMs

---

What to Build Next

Now that you understand the full stack, here are next project ideas:

1. Luganda-English Translation Fine-tune — Use your NLLB/Qwen pipeline + QLoRA
2. Medical QA for Uganda — Fine-tune on Ugandan health guidelines
3. Legal AI for Uganda — Fine-tune on Ugandan law documents
4. AgriBot — Combine with your PotatoGuard project + LLM
5. Multi-university CampusGPT — Expand to Makerere, KIU, UCU
6. Voice CampusGPT — Add speech-to-text (Whisper) + text-to-speech

---

Resources

• Unsloth docs: https://docs.unsloth.ai
• Unsloth GitHub: https://github.com/unslothai/unsloth
• TRL docs: https://huggingface.co/docs/trl
• ChromaDB docs: https://docs.trychroma.com
• Gradio docs: https://www.gradio.app/docs
• vLLM docs: https://docs.vllm.ai
• BGE-M3 paper: https://arxiv.org/abs/2402.03216
• QLoRA paper: https://arxiv.org/abs/2305.14314

---

Built by Joseph Ssemuli, Computer Science student at MUST, during Sunbird AI internship. This project is your evidence of mastering the complete Unsloth ecosystem. 🇺🇬