feat: complete Phase 0 - project setup

- Add requirements.txt with pinned versions (PyTorch, MLflow, Gradio, etc.)
- Configure .gitignore for Python, notebooks, models, and MLflow
- Initialize experiment_log.md template
- Create scripts/README.md documentation
- Verify all dependencies install correctly in ai_engg environment
- Confirm CUDA available (PyTorch 2.0.1+cu117)
- Data files present and verified

Tasks completed: 0.1, 0.2, 0.3, 0.4

.github/copilot-instructions.md

@@ -0,0 +1,67 @@
+# AI Coding Agent Instructions
+
+## Project Overview
+MNIST handwritten digit recognition project focusing on CNN-based image classification with emphasis on data quality, SE best practices, and Hugging Face deployment.
+
+## Directory Structure
+```
+data/raw/          # Original MNIST binary files (.idx*-ubyte)
+data/processed/    # Cleaned/augmented data (to be populated)
+notebooks/         # Jupyter notebooks for exploration & development
+scripts/           # Reusable Python modules
+docs/              # Project documentation
+```
+
+## Development Environment
+- **Python**: 3.10 (conda environment: `ai_engg`)
+- **Activate**: `conda activate ai_engg`
+- **Linting**: `ruff check . --fix`
+- Install dependencies: `pip install numpy matplotlib torch torchvision`
+
+## Workflow Protocol
+1. **Pre-Flight**: Verify `ai_engg` env is active, `git status` is clean
+2. **Spec-First**: Read `planning.md` and relevant spec files before coding
+3. **Anti-Redundancy**: Search `scripts/` for existing utilities before building new ones
+4. **Atomic Commits**: One task = one logic block = one commit
+5. **Commit Format**: `type: brief summary` (e.g., `feat: add data augmentation`)
+
+## Data Handling
+- MNIST data is in raw binary IDX format (not standard image files)
+- Use `MnistDataloader` class from [data/raw/read-mnist-dataset.ipynb](data/raw/read-mnist-dataset.ipynb) as reference for loading
+- Images: 28x28 grayscale, Labels: 0-9 digits
+- Training: 60,000 samples, Test: 10,000 samples
+
+## Code Conventions
+- Place reusable data loading/preprocessing code in `scripts/`
+- Exploratory work and model training in `notebooks/`
+- Use type hints and docstrings for all functions in scripts
+- Follow modular design: separate data, model, training, and evaluation logic
+
+## Key Patterns
+```python
+# Example: Loading MNIST data
+from scripts.data_loader import MnistDataloader
+loader = MnistDataloader(train_images, train_labels, test_images, test_labels)
+(x_train, y_train), (x_test, y_test) = loader.load_data()
+```
+
+## Deployment Target
+- Final model deploys to **Hugging Face Spaces**
+- Prepare inference pipeline compatible with Gradio or Streamlit interface
+
+## Documentation Requirements
+- Document data quality issues and preprocessing steps
+- Include visualizations for data exploration
+- Track experiments with clear metrics (accuracy, precision, recall)
+- Update `planning.md` with task status (mark ✅ when complete)
+
+## Code Quality Checklist
+- No unused imports
+- Proper docstrings on all functions
+- Run `ruff check . --fix` before committing
+- Implementation matches spec exactly
+
+## Pending Setup
+- [ ] Create `requirements.txt` with pinned versions
+- [ ] Populate `docs/DEVELOPMENT_WORKFLOW.md` with build/test commands
+- [ ] Set up data augmentation pipeline in `data/processed/`

.gitignore

@@ -0,0 +1,57 @@
+# Python
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+.Python
+*.so
+*.egg
+*.egg-info/
+dist/
+build/
+
+# Jupyter Notebook
+.ipynb_checkpoints/
+*.ipynb_checkpoints
+
+# Data (keep raw, ignore processed)
+data/processed/*.parquet
+data/processed/*.npy
+data/processed/*.npz
+
+# Models
+*.pt
+*.pth
+models/*.pt
+models/*.pth
+!models/.gitkeep
+
+# MLflow
+mlruns/
+mlartifacts/
+
+# Experiments
+experiments/
+runs/
+!experiments/.gitkeep
+
+# Environment
+.env
+.venv/
+venv/
+env/
+ENV/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Docker
+*.log

docs/experiment_log.md

@@ -0,0 +1,65 @@
+# Experiment Log
+
+## Overview
+This document tracks all experiments conducted during the MNIST digit classification project. Each experiment includes configuration, results, and insights.
+
+---
+
+## Experiment Template
+
+```markdown
+## Experiment N: [Name]
+**Date:** YYYY-MM-DD
+**Branch:** feature/[branch-name]
+**MLflow Run ID:** [run_id]
+**Objective:** [What we're trying to achieve]
+
+### Configuration
+- **Architecture:** [Model description]
+- **Hyperparameters:**
+  - Learning rate: X
+  - Batch size: X
+  - Epochs: X
+  - Optimizer: X
+  - [Other params]
+
+### Results
+- **Training Accuracy:** X.XX%
+- **Validation Accuracy:** X.XX%
+- **Test Accuracy:** X.XX% (if evaluated)
+- **Training Time:** X minutes
+- **Best Epoch:** X
+
+### Metrics
+| Metric | Train | Val | Test |
+|--------|-------|-----|------|
+| Accuracy | X.XX% | X.XX% | X.XX% |
+| Precision | X.XX | X.XX | X.XX |
+| Recall | X.XX | X.XX | X.XX |
+| Loss | X.XXX | X.XXX | X.XXX |
+
+### Insights
+- What worked well?
+- What didn't work?
+- Unexpected findings?
+- Next steps?
+
+### Artifacts
+- Model checkpoint: `models/[filename].pt`
+- Training curves: `experiments/plots/[filename].png`
+- MLflow link: http://localhost:5000/#/experiments/[experiment_id]/runs/[run_id]
+
+---
+```
+
+## Experiments
+
+*Experiments will be logged below as they are conducted.*
+
+---
+
+## Summary
+
+| Exp # | Date | Model | Val Acc | Test Acc | Notes |
+|-------|------|-------|---------|----------|-------|
+| - | - | - | - | - | Experiments to be added |

planning.md

@@ -0,0 +1,1727 @@
+# Planning Document: MNIST Handwritten Digit Recognition
+
+## Current Status
+
+**Current Phase:** Phase 1 - Data Pipeline & Quality Analysis  
+**Last Updated:** December 28, 2025  
+**Status:** 🟡 STARTING - Project scaffolding in progress
+
+### Quick Summary
+
+**Project Goal:**
+Develop production-ready CNN for MNIST digit classification (28×28 grayscale → 0-9 labels) emphasizing:
+- Data quality analysis throughout pipeline
+- Software engineering best practices
+- Deployment to Hugging Face Spaces
+
+**Deliverables:**
+- 20-30 page report documenting complete workflow
+- Functional Jupyter notebook validating solution
+- Deployed model on Hugging Face with interactive interface
+
+**Current Progress:**
+- ✅ Environment setup (conda env `ai_engg`, Python 3.10)
+- ✅ Raw MNIST data available (60k train, 10k test, IDX format)
+- ✅ Workflow conventions defined
+- ⬜ Planning document (this file) - in progress
+- ⬜ Code implementation - not started
+
+---
+
+## Phase Status Overview
+
+| Phase | Tasks | Status | Key Milestone | Est. Time |
+|-------|-------|--------|---------------|-----------|
+| 0: Setup | 4 | ⬜ | Dependencies + project structure | 1-2h |
+| 1: Data Pipeline | 6 | ⬜ | Quality analysis + augmentation | 6-8h |
+| 2: Model Development | 5 | ⬜ | Trained CNN with >98% accuracy | 8-10h |
+| 3: Deployment | 4 | ⬜ | Live Hugging Face Space | 4-6h |
+| 4: Documentation | 3 | ⬜ | Final 20-30 page report | 6-8h |
+
+**Total Estimated Time:** 25-34 hours
+
+---
+
+## Critical Context
+
+### Project Constraints
+- **Dataset:** MNIST (60,000 train, 10,000 test) - no external data allowed
+- **Framework:** PyTorch (specified in environment setup)
+- **Architecture:** CNN required (not simpler models)
+- **Deployment:** Hugging Face Spaces (free tier)
+- **Evaluation Metrics:** Accuracy, precision, recall (per spec)
+
+### Success Criteria
+- **Functional:** Model achieves ≥98% test accuracy (baseline: 97-98%)
+- **SE Quality:** Code passes `ruff` linting, has tests, modular design
+- **Documentation:** Complete 20-30 page report covering all requirements
+- **Production:** Working Hugging Face Space accepting digit images
+
+### Data Specifics
+- **Format:** IDX binary (not standard images) - requires custom loader
+- **Preprocessing:** Normalization (0-255 → 0-1), reshape for CNN
+- **Augmentation:** Rotation, translation, scaling to improve robustness
+- **Quality:** High quality dataset, but must document analysis process
+
+---
+
+## Phase 0: Project Setup (Pre-Flight)
+
+> **Purpose:** Establish development infrastructure before coding begins
+
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL (blocks all other work)  
+**Estimated Time:** 1-2 hours
+
+---
+
+### **Task 0.1:** Create requirements.txt with pinned versions
+**Status:** ✅ COMPLETE  
+**Priority:** CRITICAL  
+**Objective:** Lock dependency versions for reproducibility
+
+**Implementation:**
+- [ ] Create `requirements.txt` with core dependencies:
+  ```
+  numpy==1.24.3
+  matplotlib==3.7.1
+  torch==2.0.1
+  torchvision==0.15.2
+  jupyter==1.0.0
+  scikit-learn==1.3.0
+  ruff==0.0.270
+  pytest==7.4.0
+  mlflow==2.9.2
+  gradio==3.50.0
+  pillow==10.0.0
+  ```
+- [ ] Test installation: `pip install -r requirements.txt`
+- [ ] Verify imports work in Python REPL
+- [ ] Document installation in README.md (create if needed)
+
+**Success Criteria:**
+- requirements.txt exists and installs without errors
+- All imports verified working
+
+**Estimated Time:** 20 minutes
+
+---
+
+### **Task 0.2:** Configure .gitignore
+**Status:** ✅ COMPLETE  
+**Priority:** HIGH  
+**Objective:** Prevent committing generated files and large binaries
+
+**Implementation:**
+- [ ] Create `.gitignore` with standard Python patterns:
+  ```
+  # Python
+  __pycache__/
+  *.pyc
+  *.pyo
+  .ipynb_checkpoints/
+  *.egg-info/
+  
+  # Data (keep raw, ignore processed)
+  data/processed/*.parquet
+  data/processed/*.npy
+  *.pt
+  *.pth
+  
+  # Experiments
+  experiments/
+  runs/
+  mlruns/
+  
+  # Environment
+  .env
+  .venv/
+  venv/
+  ```
+- [ ] Test: create dummy files and verify git ignores them
+- [ ] Commit .gitignore
+
+**Success Criteria:**
+- Generated files don't appear in `git status`
+- Raw data and source code still tracked
+
+**Estimated Time:** 15 minutes
+
+---
+
+### **Task 0.3:** Initialize project documentation
+**Status:** ✅ COMPLETE  
+**Priority:** MEDIUM  
+**Objective:** Create documentation templates for tracking work
+
+**Implementation:**
+- [ ] Create `docs/experiment_log.md` with template:
+  ```markdown
+  # Experiment Log
+  
+  ## Experiment 1: Baseline CNN
+  **Date:** YYYY-MM-DD
+  **Branch:** feature/baseline-cnn
+  **Objective:** Establish baseline performance
+  **Architecture:** [describe]
+  **Hyperparameters:** [list]
+  **Results:** [metrics]
+  **Insights:** [what learned]
+  ```
+- [ ] Create `scripts/README.md` documenting utility modules
+- [ ] Update `.github/copilot-instructions.md` with planning.md reference
+- [ ] Commit documentation templates
+
+**Success Criteria:**
+- Templates exist and are usable
+- Copilot instructions point to planning.md
+
+**Estimated Time:** 20 minutes
+
+---
+
+### **Task 0.4:** Verify data integrity
+**Status:** ✅ COMPLETE  
+**Priority:** HIGH  
+**Objective:** Ensure MNIST files are complete and uncorrupted
+
+**Implementation:**
+- [ ] Check file sizes match expected:
+  - `train-images-idx3-ubyte` or `.gz`: ~47 MB
+  - `train-labels-idx1-ubyte` or `.gz`: ~60 KB
+  - `t10k-images-idx3-ubyte` or `.gz`: ~7.8 MB
+  - `t10k-labels-idx1-ubyte` or `.gz`: ~10 KB
+- [ ] Extract `.gz` files if compressed: `gunzip data/raw/*.gz`
+- [ ] Verify magic numbers using reference loader from notebook
+- [ ] Document data provenance in `data/raw/README.md`
+
+**Success Criteria:**
+- All files present and correct size
+- Magic numbers verified (2051 for images, 2049 for labels)
+- Data source documented
+
+**Estimated Time:** 15 minutes
+
+---
+
+### **Task 0.5:** Setup MLflow experiment tracking
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Configure MLflow for experiment tracking and model registry
+
+**Implementation:**
+- [ ] Create `mlruns/` directory (git-ignored for artifacts)
+- [ ] Create `scripts/mlflow_setup.py`:
+  ```python
+  import mlflow
+  from pathlib import Path
+  
+  # Set tracking URI to local directory
+  MLFLOW_TRACKING_URI = Path("mlruns").resolve().as_uri()
+  mlflow.set_tracking_uri(MLFLOW_TRACKING_URI)
+  
+  # Create experiment
+  EXPERIMENT_NAME = "mnist-digit-classification"
+  
+  def setup_mlflow():
+      """Initialize MLflow experiment"""
+      try:
+          experiment_id = mlflow.create_experiment(EXPERIMENT_NAME)
+      except:
+          experiment_id = mlflow.get_experiment_by_name(EXPERIMENT_NAME).experiment_id
+      
+      mlflow.set_experiment(EXPERIMENT_NAME)
+      return experiment_id
+  ```
+- [ ] Update `.gitignore` to include:
+  ```
+  mlruns/
+  mlartifacts/
+  ```
+- [ ] Create launch script `scripts/launch_mlflow_ui.sh`:
+  ```bash
+  #!/bin/bash
+  mlflow ui --backend-store-uri mlruns --port 5000
+  ```
+- [ ] Make executable: `chmod +x scripts/launch_mlflow_ui.sh`
+- [ ] Test: Run `./scripts/launch_mlflow_ui.sh` and access http://localhost:5000
+
+**Success Criteria:**
+- MLflow UI launches without errors
+- Experiment "mnist-digit-classification" visible
+- Tracking URI configured correctly
+
+**Estimated Time:** 30 minutes
+
+---
+
+## Phase 1: Data Pipeline & Quality Analysis
+
+> **Purpose:** Build robust data loading and preprocessing pipeline with comprehensive quality analysis
+
+**Status:** ⬜ NOT STARTED  
+**Prerequisites:** Phase 0 complete  
+**Estimated Time:** 6-8 hours
+
+---
+
+### **Task 1.1:** Extract MnistDataloader to reusable module
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Move data loader from notebook to `scripts/` for reuse
+
+**Implementation:**
+- [ ] Create `scripts/data_loader.py`
+- [ ] Extract `MnistDataloader` class from `data/raw/read-mnist-dataset.ipynb`
+- [ ] Add type hints:
+  ```python
+  from typing import Tuple
+  import numpy as np
+  from numpy.typing import NDArray
+  
+  class MnistDataloader:
+      def __init__(
+          self, 
+          training_images_filepath: str,
+          training_labels_filepath: str,
+          test_images_filepath: str,
+          test_labels_filepath: str
+      ) -> None:
+          ...
+      
+      def load_data(self) -> Tuple[
+          Tuple[list[NDArray[np.uint8]], list[int]], 
+          Tuple[list[NDArray[np.uint8]], list[int]]
+      ]:
+          ...
+  ```
+- [ ] Add docstrings (class and methods)
+- [ ] Handle file not found errors gracefully
+- [ ] Test loading: verify shapes (60000, 28, 28) and (10000, 28, 28)
+
+**Success Criteria:**
+- Module imports successfully: `from scripts.data_loader import MnistDataloader`
+- Loads data without errors
+- Returns correct shapes and data types
+
+**Estimated Time:** 45 minutes
+
+---
+
+### **Task 1.2:** Build data exploration notebook
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Visual and statistical exploration of MNIST dataset
+
+**Implementation:**
+- [ ] Create `notebooks/01_data_exploration.ipynb`
+- [ ] Load data using new `MnistDataloader` module
+- [ ] Visualizations:
+  - Grid of sample images (10×10) with labels
+  - One sample per digit class (0-9) side-by-side
+  - Pixel intensity histograms (overall and per-class)
+  - Image dimension verification (all 28×28)
+- [ ] Statistical analysis:
+  - Class balance: count per digit (should be ~6000 each for training)
+  - Pixel value range: min/max (should be 0-255)
+  - Missing values check (should be zero)
+  - Mean/std pixel intensity per class
+- [ ] Document findings in markdown cells
+
+**Expected Findings:**
+- MNIST is well-balanced (~6000 samples per digit in training)
+- No missing values or corrupted images
+- Some digits harder to distinguish (4/9, 3/8, 5/6)
+
+**Deliverables:**
+- [ ] Notebook with visualizations and analysis
+- [ ] Summary of data quality findings
+
+**Success Criteria:**
+- Clear visualizations rendering correctly
+- Statistical properties documented
+- Findings support data quality section of report
+
+**Estimated Time:** 1.5 hours
+
+---
+
+### **Task 1.3:** Implement data quality analysis module
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Systematic quality checks for report documentation
+
+**Implementation:**
+- [ ] Create `scripts/data_quality.py` with functions:
+  ```python
+  def check_missing_values(images, labels) -> dict:
+      """Check for NaN or missing values"""
+      
+  def check_outliers(images) -> dict:
+      """Identify pixels outside 0-255 range"""
+      
+  def check_class_balance(labels) -> dict:
+      """Compute samples per class and imbalance ratio"""
+      
+  def check_image_dimensions(images) -> dict:
+      """Verify all images are 28x28"""
+      
+  def generate_quality_report(train_data, test_data) -> dict:
+      """Run all checks and return comprehensive report"""
+  ```
+- [ ] Create unit tests: `tests/test_data_quality.py`
+- [ ] Run quality checks on train and test sets
+- [ ] Save report as JSON: `data/quality_report.json`
+
+**Success Criteria:**
+- All quality check functions implemented with tests
+- Report confirms high data quality (no issues for MNIST)
+- JSON report available for documentation
+
+**Estimated Time:** 1.5 hours
+
+---
+
+### **Task 1.4:** Create preprocessing pipeline
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Normalize and prepare data for CNN input
+
+**Implementation:**
+- [ ] Create `scripts/preprocessing.py`:
+  ```python
+  import torch
+  from torch.utils.data import Dataset, DataLoader
+  
+  class MnistDataset(Dataset):
+      def __init__(self, images, labels, transform=None):
+          """
+          Args:
+              images: List of 28x28 numpy arrays
+              labels: List of integer labels (0-9)
+              transform: Optional torchvision transforms
+          """
+          self.images = images
+          self.labels = labels
+          self.transform = transform
+      
+      def __getitem__(self, idx):
+          image = self.images[idx]
+          label = self.labels[idx]
+          
+          # Normalize to [0, 1]
+          image = image.astype(np.float32) / 255.0
+          
+          # Add channel dimension: (28, 28) -> (1, 28, 28)
+          image = torch.tensor(image).unsqueeze(0)
+          label = torch.tensor(label, dtype=torch.long)
+          
+          if self.transform:
+              image = self.transform(image)
+          
+          return image, label
+  ```
+- [ ] Test pipeline:
+  - Verify normalization (values in [0, 1])
+  - Check tensor shapes: images (B, 1, 28, 28), labels (B,)
+  - Test DataLoader batching
+- [ ] Document preprocessing steps in docstrings
+
+**Success Criteria:**
+- MnistDataset works with PyTorch DataLoader
+- Data properly normalized and shaped for CNN
+- No data leakage (train/test separate)
+
+**Estimated Time:** 1 hour
+
+---
+
+### **Task 1.5:** Implement data augmentation
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Generate augmented training data for robustness
+
+**Implementation:**
+- [ ] Create `scripts/augmentation.py`:
+  ```python
+  from torchvision import transforms
+  
+  def get_augmentation_pipeline():
+      """Return composition of augmentation transforms"""
+      return transforms.Compose([
+          transforms.RandomRotation(degrees=15),  # ±15° rotation
+          transforms.RandomAffine(
+              degrees=0, 
+              translate=(0.1, 0.1),  # ±10% translation
+              scale=(0.9, 1.1)       # 90-110% zoom
+          ),
+          # Note: already normalized in Dataset
+      ])
+  ```
+- [ ] Test augmentations visually:
+  - Create `notebooks/02_augmentation_demo.ipynb`
+  - Show original vs augmented images side-by-side
+  - Verify labels remain correct
+  - Check augmentation doesn't distort digits beyond recognition
+- [ ] Document augmentation rationale:
+  - Why these parameters? (realistic handwriting variations)
+  - Expected impact on generalization
+  - Trade-offs (training time vs accuracy)
+
+**Design Decision:**
+Apply augmentations **on-the-fly** during training (not pre-generate). Reasons:
+- Infinite variations per epoch
+- Saves disk space
+- Standard PyTorch practice
+
+**Success Criteria:**
+- Augmentation pipeline integrates with MnistDataset
+- Visual verification shows realistic variations
+- Documented in experiment log
+
+**Estimated Time:** 1.5 hours
+
+---
+
+### **Task 1.6:** Create train/validation split
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Split 60k training data into train/val sets
+
+**Implementation:**
+- [ ] Add split function to `scripts/preprocessing.py`:
+  ```python
+  from sklearn.model_selection import train_test_split
+  
+  def create_train_val_split(
+      images, 
+      labels, 
+      val_size: float = 0.15,
+      random_state: int = 42
+  ):
+      """
+      Split data into train and validation sets.
+      
+      Args:
+          val_size: Fraction for validation (default 15% = 9000 samples)
+          random_state: Seed for reproducibility
+      
+      Returns:
+          (train_images, train_labels, val_images, val_labels)
+      """
+      return train_test_split(
+          images, labels, 
+          test_size=val_size, 
+          random_state=random_state,
+          stratify=labels  # Maintain class balance
+      )
+  ```
+- [ ] Test split:
+  - Verify sizes: 51000 train, 9000 val (for 15% split)
+  - Check class balance maintained in both sets
+  - Verify no data leakage (no overlap)
+- [ ] Document split strategy in `docs/experiment_log.md`
+
+**Success Criteria:**
+- Train/val split maintains class balance
+- Reproducible (same split every run with fixed seed)
+- Documented split rationale
+
+**Estimated Time:** 30 minutes
+
+---
+
+## Phase 2: Model Development & Training
+
+> **Purpose:** Design, implement, and train CNN architecture with rigorous evaluation
+
+**Status:** ⬜ NOT STARTED  
+**Prerequisites:** Phase 1 complete (data pipeline working)  
+**Estimated Time:** 8-10 hours
+
+---
+
+### **Task 2.1:** Design baseline CNN architecture
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Implement simple but effective CNN for MNIST
+
+**Implementation:**
+- [ ] Create `scripts/models.py`:
+  ```python
+  import torch.nn as nn
+  
+  class BaselineCNN(nn.Module):
+      """
+      Baseline CNN for MNIST classification.
+      
+      Architecture:
+          Conv1: 1 -> 32 filters, 3x3, ReLU, MaxPool(2x2)
+          Conv2: 32 -> 64 filters, 3x3, ReLU, MaxPool(2x2)
+          Flatten
+          FC1: 64*7*7 -> 128, ReLU, Dropout(0.5)
+          FC2: 128 -> 10 (output logits)
+      
+      Expected parameters: ~100k
+      Expected accuracy: 98-99%
+      """
+      def __init__(self):
+          super().__init__()
+          self.conv1 = nn.Conv2d(1, 32, kernel_size=3, padding=1)
+          self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1)
+          self.pool = nn.MaxPool2d(2, 2)
+          self.fc1 = nn.Linear(64 * 7 * 7, 128)
+          self.fc2 = nn.Linear(128, 10)
+          self.dropout = nn.Dropout(0.5)
+      
+      def forward(self, x):
+          # Implementation here
+          ...
+  ```
+- [ ] Document architecture choices:
+  - Why 2 conv layers? (balance simplicity vs capacity)
+  - Why 32→64 filters? (standard progression)
+  - Why dropout 0.5? (prevent overfitting)
+- [ ] Test model:
+  - Verify forward pass with dummy input (1, 1, 28, 28)
+  - Check output shape (1, 10)
+  - Count parameters: `sum(p.numel() for p in model.parameters())`
+
+**Design Rationale:**
+- Start simple: baseline must work before trying complex architectures
+- Proven pattern: 2 conv layers sufficient for MNIST
+- Dropout critical: MNIST is small, overfitting likely
+
+**Success Criteria:**
+- Model instantiates without errors
+- Forward pass produces correct output shape
+- Architecture documented with justification
+
+**Estimated Time:** 1 hour
+
+---
+
+### **Task 2.2:** Implement training pipeline
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Build robust training loop with logging
+
+**Implementation:**
+- [ ] Create `scripts/train.py`:
+  ```python
+  import torch
+  import torch.nn as nn
+  import torch.optim as optim
+  from typing import Dict, List
+  
+  def train_epoch(
+      model, 
+      train_loader, 
+      criterion, 
+      optimizer, 
+      device
+  ) -> Dict[str, float]:
+      """Train for one epoch, return metrics"""
+      model.train()
+      total_loss = 0.0
+      correct = 0
+      total = 0
+      
+      for images, labels in train_loader:
+          images, labels = images.to(device), labels.to(device)
+          
+          optimizer.zero_grad()
+          outputs = model(images)
+          loss = criterion(outputs, labels)
+          loss.backward()
+          optimizer.step()
+          
+          total_loss += loss.item()
+          _, predicted = outputs.max(1)
+          correct += predicted.eq(labels).sum().item()
+          total += labels.size(0)
+      
+      return {
+          'loss': total_loss / len(train_loader),
+          'accuracy': 100.0 * correct / total
+      }
+  
+  def validate(model, val_loader, criterion, device) -> Dict[str, float]:
+      """Evaluate on validation set"""
+      # Similar structure, no gradient computation
+      ...
+  
+  def train_model(
+      model, 
+      train_loader, 
+      val_loader, 
+      num_epochs: int = 10,
+      learning_rate: float = 0.001,
+      device: str = 'cuda' if torch.cuda.is_available() else 'cpu'
+  ) -> Dict[str, List[float]]:
+      """Full training loop with history tracking"""
+      ...
+  ```
+- [ ] Add learning rate scheduling:
+  ```python
+  scheduler = optim.lr_scheduler.ReduceLROnPlateau(
+      optimizer, mode='min', patience=3, factor=0.5
+  )
+  ```
+- [ ] Implement early stopping (patience=5 epochs)
+- [ ] Save checkpoints:
+  - Best model (lowest val loss): `models/best_model.pt`
+  - Last model: `models/last_model.pt`
+- [ ] Log training history to JSON: `experiments/training_history.json`
+
+**Success Criteria:**
+- Training loop runs without errors
+- Validation accuracy improves over epochs
+- Checkpoints saved correctly
+- History available for plotting
+
+**Estimated Time:** 2 hours
+
+---
+
+### **Task 2.2b:** Integrate MLflow tracking
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Add MLflow logging to training pipeline
+
+**Implementation:**
+- [ ] Update `scripts/train.py` to log with MLflow:
+  ```python
+  import mlflow
+  from scripts.mlflow_setup import setup_mlflow
+  
+  def train_model(
+      model, 
+      train_loader, 
+      val_loader, 
+      num_epochs: int = 10,
+      learning_rate: float = 0.001,
+      device: str = 'cuda' if torch.cuda.is_available() else 'cpu'
+  ) -> Dict[str, List[float]]:
+      """Full training loop with MLflow tracking"""
+      
+      # Initialize MLflow
+      setup_mlflow()
+      
+      with mlflow.start_run():
+          # Log hyperparameters
+          mlflow.log_params({
+              'num_epochs': num_epochs,
+              'learning_rate': learning_rate,
+              'batch_size': train_loader.batch_size,
+              'optimizer': 'Adam',
+              'architecture': model.__class__.__name__,
+              'device': device
+          })
+          
+          # Training loop
+          for epoch in range(num_epochs):
+              train_metrics = train_epoch(...)
+              val_metrics = validate(...)
+              
+              # Log metrics
+              mlflow.log_metrics({
+                  'train_loss': train_metrics['loss'],
+                  'train_accuracy': train_metrics['accuracy'],
+                  'val_loss': val_metrics['loss'],
+                  'val_accuracy': val_metrics['accuracy']
+              }, step=epoch)
+              
+              # Save best model
+              if val_metrics['loss'] < best_val_loss:
+                  best_val_loss = val_metrics['loss']
+                  torch.save(model.state_dict(), 'models/best_model.pt')
+                  
+                  # Log model to MLflow
+                  mlflow.pytorch.log_model(
+                      model, 
+                      "model",
+                      registered_model_name="mnist-cnn-baseline"
+                  )
+          
+          # Log final artifacts
+          mlflow.log_artifact('models/best_model.pt')
+          mlflow.log_artifact('experiments/training_history.json')
+          
+          return history
+  ```
+- [ ] Test MLflow logging:
+  - Run short training (2-3 epochs)
+  - Check MLflow UI for logged params, metrics, and artifacts
+  - Verify model appears in model registry
+- [ ] Document MLflow workflow in `docs/experiment_log.md`
+
+**Success Criteria:**
+- All hyperparameters logged automatically
+- Metrics tracked per epoch in MLflow UI
+- Models saved to MLflow model registry
+- Training curves visible in MLflow UI
+
+**Estimated Time:** 1.5 hours
+
+---
+
+### **Task 2.3:** Train baseline model
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Train baseline CNN and establish performance benchmark
+
+**Implementation:**
+- [ ] Create `notebooks/03_train_baseline.ipynb`
+- [ ] Configure training:
+  ```python
+  config = {
+      'batch_size': 64,
+      'num_epochs': 15,
+      'learning_rate': 0.001,
+      'optimizer': 'Adam',
+      'weight_decay': 1e-5,
+      'device': 'cuda' if available else 'cpu'
+  }
+  ```
+- [ ] Train model with augmentation
+- [ ] Track metrics:
+  - Training loss/accuracy per epoch
+  - Validation loss/accuracy per epoch
+  - Best validation accuracy achieved
+  - Training time per epoch
+- [ ] Visualize training curves (loss and accuracy)
+- [ ] Test on test set (use ONLY ONCE to avoid overfitting to test)
+
+**Expected Results:**
+- Validation accuracy: 98-99% (typical for MNIST CNN)
+- Training time: ~2-5 min per epoch on CPU, <1 min on GPU
+- Convergence: plateaus after 10-12 epochs
+
+**Deliverables:**
+- [ ] Trained model checkpoint: `models/baseline_cnn_best.pt`
+- [ ] Training history: `experiments/baseline_training.json`
+- [ ] Notebook with training curves and analysis
+
+**Success Criteria:**
+- Validation accuracy ≥ 98.0%
+- No severe overfitting (train/val gap < 2%)
+- Reproducible (fixed random seeds)
+
+**Estimated Time:** 2-3 hours (includes training time)
+
+---
+
+### **Task 2.4:** Comprehensive model evaluation
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Compute all required metrics for report
+
+**Implementation:**
+- [ ] Create `scripts/evaluate.py`:
+  ```python
+  from sklearn.metrics import (
+      accuracy_score, 
+      precision_recall_fscore_support,
+      confusion_matrix,
+      classification_report
+  )
+  
+  def evaluate_model(model, test_loader, device) -> Dict:
+      """Compute all evaluation metrics"""
+      model.eval()
+      all_preds = []
+      all_labels = []
+      
+      with torch.no_grad():
+          for images, labels in test_loader:
+              images = images.to(device)
+              outputs = model(images)
+              _, predicted = outputs.max(1)
+              all_preds.extend(predicted.cpu().numpy())
+              all_labels.extend(labels.numpy())
+      
+      # Compute metrics
+      accuracy = accuracy_score(all_labels, all_preds)
+      precision, recall, f1, _ = precision_recall_fscore_support(
+          all_labels, all_preds, average='macro'
+      )
+      conf_matrix = confusion_matrix(all_labels, all_preds)
+      
+      # Per-class metrics
+      per_class = classification_report(
+          all_labels, all_preds, 
+          target_names=[str(i) for i in range(10)],
+          output_dict=True
+      )
+      
+      return {
+          'accuracy': accuracy,
+          'precision': precision,
+          'recall': recall,
+          'f1_score': f1,
+          'confusion_matrix': conf_matrix.tolist(),
+          'per_class_metrics': per_class
+      }
+  ```
+- [ ] Create `notebooks/04_model_evaluation.ipynb`
+- [ ] Generate visualizations:
+  - Confusion matrix heatmap (seaborn)
+  - Per-class precision/recall bar charts
+  - Misclassified examples (show images + predictions)
+- [ ] Perform error analysis:
+  - Which digit pairs confused most? (e.g., 4/9, 3/8)
+  - Are errors systematic or random?
+  - Visualize top-10 worst predictions
+- [ ] Save evaluation report: `experiments/evaluation_report.json`
+
+**Success Criteria:**
+- All required metrics computed (accuracy, precision, recall)
+- Confusion matrix shows strong diagonal (high accuracy per class)
+- Error analysis identifies patterns
+- Visualizations ready for report
+
+**Estimated Time:** 2 hours
+
+---
+
+### **Task 2.5:** Experiment with improvements (leveraging MLflow)
+**Status:** ⬜ NOT STARTED  
+**Priority:** MEDIUM  
+**Objective:** Systematic hyperparameter tuning using MLflow experiment tracking
+
+**Potential Experiments:**
+1. **Deeper architecture**: Add 3rd conv layer (64→128 filters)
+2. **Batch normalization**: Add after each conv layer
+3. **Different optimizer**: Try SGD with momentum vs Adam
+4. **Learning rate tuning**: Grid search [0.0001, 0.001, 0.01]
+5. **Regularization**: Try different dropout rates [0.3, 0.5, 0.7]
+
+**Implementation:**
+- [ ] Create `notebooks/05_hyperparameter_tuning.ipynb`
+- [ ] Use MLflow for systematic tracking:
+  ```python
+  import mlflow
+  from itertools import product
+  
+  # Define search space
+  learning_rates = [0.0001, 0.001, 0.01]
+  dropout_rates = [0.3, 0.5, 0.7]
+  batch_sizes = [32, 64, 128]
+  
+  # Grid search
+  for lr, dropout, batch_size in product(learning_rates, dropout_rates, batch_sizes):
+      with mlflow.start_run(run_name=f"lr{lr}_drop{dropout}_bs{batch_size}"):
+          # Train and log
+          ...
+  ```
+- [ ] Use MLflow UI to compare experiments:
+  - Sort runs by validation accuracy
+  - Visualize parameter impact
+  - Identify best configuration
+- [ ] Document findings in `docs/experiment_log.md`
+
+**Decision Rule:**
+Only pursue if baseline achieves ≥98% but want to push to 99%+. MLflow makes this more efficient than manual tracking.
+
+**Success Criteria:**
+- All experiments logged in MLflow with comparable metrics
+- Best model identified through MLflow comparison
+- Clear documentation of what worked/didn't work
+
+**Estimated Time:** 2-3 hours
+
+---
+
+## Phase 3: Deployment to Hugging Face
+
+> **Purpose:** Package model for production and deploy to Hugging Face Spaces
+
+**Status:** ⬜ NOT STARTED  
+**Prerequisites:** Phase 2 complete (trained model achieving ≥98% accuracy)  
+**Estimated Time:** 4-6 hours
+
+---
+
+### **Task 3.1:** Create inference module
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Build clean inference interface for deployment
+
+**Implementation:**
+- [ ] Create `scripts/inference.py`:
+  ```python
+  import torch
+  from PIL import Image
+  import numpy as np
+  
+  class DigitClassifier:
+      """Production inference wrapper"""
+      
+      def __init__(self, model_path: str, device: str = 'cpu'):
+          self.device = device
+          self.model = self.load_model(model_path)
+          self.model.eval()
+      
+      def load_model(self, path: str):
+          """Load model from checkpoint"""
+          from scripts.models import BaselineCNN
+          model = BaselineCNN()
+          model.load_state_dict(torch.load(path, map_location=self.device))
+          return model.to(self.device)
+      
+      def preprocess(self, image: Image.Image) -> torch.Tensor:
+          """
+          Preprocess image for model input.
+          Accepts PIL Image or numpy array.
+          Handles resizing, normalization, etc.
+          """
+          # Convert to grayscale if RGB
+          if image.mode != 'L':
+              image = image.convert('L')
+          
+          # Resize to 28x28 if needed
+          if image.size != (28, 28):
+              image = image.resize((28, 28), Image.Resampling.LANCZOS)
+          
+          # Convert to tensor and normalize
+          img_array = np.array(image).astype(np.float32) / 255.0
+          img_tensor = torch.tensor(img_array).unsqueeze(0).unsqueeze(0)
+          return img_tensor.to(self.device)
+      
+      def predict(self, image: Image.Image) -> dict:
+          """
+          Predict digit from image.
+          
+          Returns:
+              {
+                  'digit': int (0-9),
+                  'confidence': float (0-1),
+                  'probabilities': list of 10 floats
+              }
+          """
+          img_tensor = self.preprocess(image)
+          
+          with torch.no_grad():
+              outputs = self.model(img_tensor)
+              probabilities = torch.softmax(outputs, dim=1)[0]
+              confidence, predicted = torch.max(probabilities, dim=0)
+          
+          return {
+              'digit': int(predicted.item()),
+              'confidence': float(confidence.item()),
+              'probabilities': probabilities.cpu().numpy().tolist()
+          }
+  ```
+- [ ] Test inference module:
+  - Load test set images
+  - Verify predictions match evaluation results
+  - Test with various image formats (PNG, JPG, different sizes)
+  - Test edge cases (blank image, non-digit image)
+
+**Success Criteria:**
+- Clean API: `classifier.predict(image)` → results
+- Handles various input formats gracefully
+- Fast inference (<100ms per image on CPU)
+
+**Estimated Time:** 1.5 hours
+
+---
+
+### **Task 3.2:** Build Gradio interface
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Create interactive web UI for digit recognition
+
+**Implementation:**
+- [ ] Create `app.py`:
+  ```python
+  import gradio as gr
+  from scripts.inference import DigitClassifier
+  from PIL import Image
+  
+  # Initialize classifier
+  classifier = DigitClassifier('models/baseline_cnn_best.pt')
+  
+  def predict_digit(image):
+      """Gradio interface function"""
+      if image is None:
+          return "Please draw or upload a digit", None
+      
+      # Convert to PIL Image
+      if isinstance(image, np.ndarray):
+          image = Image.fromarray(image.astype('uint8'))
+      
+      # Get prediction
+      result = classifier.predict(image)
+      
+      # Format output
+      label = f"Predicted Digit: {result['digit']}"
+      confidence = f"Confidence: {result['confidence']:.2%}"
+      
+      # Create probability chart
+      probs = {str(i): result['probabilities'][i] for i in range(10)}
+      
+      return f"{label}\n{confidence}", probs
+  
+  # Create Gradio interface
+  demo = gr.Interface(
+      fn=predict_digit,
+      inputs=gr.Image(
+          sources=['upload', 'canvas'],
+          type='pil',
+          label="Draw or upload a digit (0-9)",
+          image_mode='L'  # Grayscale
+      ),
+      outputs=[
+          gr.Textbox(label="Prediction"),
+          gr.BarPlot(label="Confidence per Digit")
+      ],
+      title="MNIST Digit Classifier",
+      description="Draw a digit (0-9) or upload an image. Model trained on MNIST dataset.",
+      examples=[
+          # Add example images from test set
+      ],
+      theme="default"
+  )
+  
+  if __name__ == "__main__":
+      demo.launch()
+  ```
+- [ ] Test locally: `python app.py`
+  - Test drawing on canvas
+  - Test uploading images
+  - Verify probability bars update correctly
+  - Test responsive design (mobile/desktop)
+- [ ] Add example images:
+  - Extract 10 images from test set (one per digit)
+  - Save as `examples/digit_0.png`, etc.
+  - Include in interface for quick testing
+
+**Success Criteria:**
+- Interface launches locally without errors
+- Users can draw or upload digits
+- Predictions display clearly with confidence scores
+- Responsive and intuitive UX
+
+**Estimated Time:** 2 hours
+
+---
+
+### **Task 3.3:** Deploy to Hugging Face Spaces
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Make model publicly accessible via Hugging Face
+
+**Implementation:**
+- [ ] Create Hugging Face account (if needed): https://huggingface.co/join
+- [ ] Create new Space:
+  - Name: `mnist-digit-classifier`
+  - SDK: Gradio
+  - Hardware: CPU (free tier sufficient)
+- [ ] Prepare deployment files:
+  ```
+  .
+  ├── app.py                      # Gradio interface
+  ├── requirements.txt            # Deployment dependencies
+  ├── models/
+  │   └── baseline_cnn_best.pt   # Model checkpoint
+  ├── scripts/
+  │   ├── models.py               # Model architecture
+  │   └── inference.py            # Inference wrapper
+  └── README.md                   # Space documentation
+  ```
+- [ ] Create deployment `requirements.txt`:
+  ```
+  torch==2.0.1
+  torchvision==0.15.2
+  gradio==3.50.0
+  pillow==10.0.0
+  numpy==1.24.3
+  ```
+- [ ] Write Space README.md:
+  ```markdown
+  ---
+  title: MNIST Digit Classifier
+  emoji: 🔢
+  colorFrom: blue
+  colorTo: purple
+  sdk: gradio
+  sdk_version: 3.50.0
+  app_file: app.py
+  pinned: false
+  ---
+  
+  # MNIST Digit Classifier
+  
+  CNN model for handwritten digit recognition (0-9).
+  
+  ## Model Details
+  - Architecture: 2-layer CNN
+  - Accuracy: XX.X% on MNIST test set
+  - Training data: 60,000 handwritten digits
+  
+  ## Usage
+  Draw a digit or upload an image to get predictions.
+  ```
+- [ ] Deploy:
+  - Initialize git repo: `git init`
+  - Add Hugging Face remote
+  - Push code: `git push`
+  - Monitor build logs for errors
+- [ ] Test deployed Space:
+  - Visit Space URL
+  - Test drawing interface
+  - Verify predictions match local testing
+  - Check loading time (<10s initial load)
+
+**Success Criteria:**
+- Space builds successfully
+- Public URL accessible: `https://huggingface.co/spaces/<username>/mnist-digit-classifier`
+- Interface works identically to local version
+- Model makes accurate predictions
+
+**Estimated Time:** 1.5 hours
+
+---
+
+### **Task 3.4:** Document deployment
+**Status:** ⬜ NOT STARTED  
+**Priority:** MEDIUM  
+**Objective:** Create usage guide for deployed model
+
+**Implementation:**
+- [ ] Update Space README with:
+  - Model architecture details
+  - Training methodology
+  - Performance metrics (accuracy, precision, recall)
+  - Example usage (screenshots)
+  - Limitations (works best on centered digits)
+  - Citation/attribution
+- [ ] Create `docs/deployment_guide.md`:
+  - How to run locally
+  - How to deploy to other platforms
+  - API documentation (if adding API endpoint)
+  - Troubleshooting common issues
+- [ ] Add Space URL to main project README
+- [ ] Take screenshots for report
+
+**Success Criteria:**
+- Clear documentation for users and developers
+- README includes all necessary information
+- Screenshots captured for report
+
+**Estimated Time:** 45 minutes
+
+---
+
+### **Task 3.5:** Create Docker container
+**Status:** ⬜ NOT STARTED  
+**Priority:** HIGH  
+**Objective:** Containerize application for reproducible deployment
+
+**Implementation:**
+- [ ] Create `Dockerfile`:
+  ```dockerfile
+  # Use official Python runtime as base
+  FROM python:3.10-slim
+  
+  # Set working directory
+  WORKDIR /app
+  
+  # Install system dependencies
+  RUN apt-get update && apt-get install -y \
+      build-essential \
+      && rm -rf /var/lib/apt/lists/*
+  
+  # Copy requirements and install Python dependencies
+  COPY requirements.txt .
+  RUN pip install --no-cache-dir -r requirements.txt
+  
+  # Copy application code
+  COPY scripts/ ./scripts/
+  COPY models/ ./models/
+  COPY app.py .
+  
+  # Expose Gradio port
+  EXPOSE 7860
+  
+  # Run application
+  CMD ["python", "app.py"]
+  ```
+- [ ] Create `.dockerignore`:
+  ```
+  __pycache__
+  *.pyc
+  .git
+  .gitignore
+  data/
+  notebooks/
+  experiments/
+  mlruns/
+  *.md
+  .venv/
+  venv/
+  ```
+- [ ] Build Docker image:
+  ```bash
+  docker build -t mnist-classifier:latest .
+  ```
+- [ ] Test container locally:
+  ```bash
+  docker run -p 7860:7860 mnist-classifier:latest
+  ```
+  - Access http://localhost:7860
+  - Verify inference works
+  - Check resource usage
+- [ ] Document Docker usage in `README.md`
+
+**Success Criteria:**
+- Docker image builds without errors
+- Container runs application successfully
+- Inference works identically to local setup
+- Image size reasonable (<1GB)
+
+**Estimated Time:** 1.5 hours
+
+---
+
+### **Task 3.6:** Create docker-compose setup
+**Status:** ⬜ NOT STARTED  
+**Priority:** MEDIUM  
+**Objective:** Multi-container setup with MLflow UI for complete dev environment
+
+**Implementation:**
+- [ ] Create `docker-compose.yml`:
+  ```yaml
+  version: '3.8'
+  
+  services:
+    mnist-app:
+      build: .
+      ports:
+        - "7860:7860"
+      volumes:
+        - ./models:/app/models
+      environment:
+        - PYTHONUNBUFFERED=1
+    
+    mlflow:
+      image: ghcr.io/mlflow/mlflow:v2.9.2
+      ports:
+        - "5000:5000"
+      volumes:
+        - ./mlruns:/mlflow/mlruns
+      command: mlflow server --host 0.0.0.0 --port 5000 --backend-store-uri /mlflow/mlruns
+  ```
+- [ ] Test multi-container setup:
+  ```bash
+  docker-compose up -d
+  ```
+  - Access app: http://localhost:7860
+  - Access MLflow: http://localhost:5000
+- [ ] Create management scripts:
+  - `scripts/docker_start.sh`: Start containers
+  - `scripts/docker_stop.sh`: Stop containers
+  - `scripts/docker_logs.sh`: View logs
+- [ ] Document docker-compose workflow in `docs/deployment_guide.md`
+
+**Success Criteria:**
+- Both services start successfully
+- Can access app and MLflow UI simultaneously
+- Volumes persist data correctly
+- Easy startup/shutdown
+
+**Estimated Time:** 1 hour
+
+---
+
+## Phase 4: Final Documentation & Report
+
+> **Purpose:** Compile comprehensive 20-30 page report and finalize deliverables
+
+**Status:** ⬜ NOT STARTED  
+**Prerequisites:** Phases 1-3 complete  
+**Estimated Time:** 6-8 hours
+
+---
+
+### **Task 4.1:** Write technical report
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Create final 20-30 page report covering all requirements
+
+**Report Structure:**
+```markdown
+# MNIST Handwritten Digit Recognition: A Data Quality-Focused Approach
+
+## Executive Summary (1 page)
+- Project overview
+- Key achievements
+- Final model performance
+
+## 1. Introduction (2 pages)
+- Problem statement
+- Objectives
+- Approach overview
+
+## 2. Data Pipeline (4-5 pages)
+- MNIST dataset description
+- IDX format handling
+- Data loading implementation
+- **Data Quality Analysis** (detailed):
+  - Missing values check
+  - Outlier detection
+  - Class balance analysis
+  - Visual inspection results
+- Preprocessing steps (normalization)
+- Train/validation/test split strategy
+
+## 3. Data Augmentation (3-4 pages)
+- Motivation and rationale
+- Augmentation techniques:
+  - Random rotation (±15°)
+  - Random translation (±10%)
+  - Random scaling (90-110%)
+- Implementation details
+- Visual examples (before/after)
+- Impact on model performance (with/without augmentation)
+
+## 4. Model Architecture (3-4 pages)
+- CNN design rationale
+- Architecture details:
+  - Layer specifications
+  - Parameter counts
+  - Activation functions
+- Architectural diagram
+- Design trade-offs and alternatives considered
+
+## 5. Training Methodology (3-4 pages)
+- Training configuration (hyperparameters)
+- Loss function and optimizer selection
+- Learning rate scheduling
+- Early stopping and regularization
+- Training curves (loss and accuracy)
+- Convergence analysis
+
+## 6. Evaluation & Results (4-5 pages)
+- **Metrics** (as required):
+  - Accuracy: X.X%
+  - Precision (macro): X.X%
+  - Recall (macro): X.X%
+- Confusion matrix analysis
+- Per-class performance
+- Error analysis:
+  - Most common misclassifications
+  - Challenging digit pairs
+  - Failure case examples
+- Comparison to baselines/literature
+
+## 7. Software Engineering Practices (3-4 pages)
+- **Code organization**:
+  - Modular design (scripts/ structure)
+  - Separation of concerns
+- **Version control**:
+  - Git workflow
+  - Commit conventions
+  - Branch strategy
+- **Code quality**:
+  - Linting (ruff)
+  - Type hints and docstrings
+  - Testing (if implemented)
+- **Experiment tracking**:
+  - MLflow setup and workflow
+  - Experiment comparison methodology
+  - Model registry and versioning
+- **Documentation**:
+  - Inline comments
+  - Module documentation
+  - README files
+
+## 8. Deployment (3-4 pages)
+- **Containerization**:
+  - Docker setup and rationale
+  - Multi-stage builds (if applicable)
+  - Container orchestration (docker-compose)
+- **MLflow integration**:
+  - Model serving from registry
+  - Experiment reproducibility
+- **Hugging Face Spaces**:
+  - Platform setup and configuration
+  - Interface design (Gradio)
+  - Inference pipeline
+  - Usage examples and screenshots
+  - Performance considerations (latency, resource usage)
+- **Deployment alternatives**:
+  - Local Docker deployment
+  - Docker vs Hugging Face comparison
+
+## 9. Challenges & Lessons Learned (1-2 pages)
+- Technical challenges encountered
+- Solutions implemented
+- Insights gained
+- Future improvements
+
+## 10. Conclusion (1 page)
+- Summary of achievements
+- Key takeaways
+- Potential extensions
+
+## References
+- MNIST dataset citation
+- Framework documentation (PyTorch)
+- Relevant papers/tutorials
+
+## Appendix
+- A: Code snippets (key functions)
+- B: Additional visualizations
+- C: Experiment log summary
+```
+
+**Implementation:**
+- [ ] Draft each section using content from notebooks and experiment log
+- [ ] Include all visualizations (plots, diagrams, screenshots)
+- [ ] Ensure data quality analysis is comprehensive (major focus)
+- [ ] Proofread for clarity and technical accuracy
+- [ ] Format professionally (LaTeX or polished Markdown)
+
+**Success Criteria:**
+- Report is 20-30 pages (excluding appendix)
+- All required topics covered with sufficient depth
+- Data quality analysis prominent
+- SE practices clearly documented
+- Professional presentation quality
+
+**Estimated Time:** 5-6 hours (spread over multiple sessions)
+
+---
+
+### **Task 4.2:** Create final validation notebook
+**Status:** ⬜ NOT STARTED  
+**Priority:** CRITICAL  
+**Objective:** Single notebook demonstrating complete solution
+
+**Implementation:**
+- [ ] Create `notebooks/99_final_solution.ipynb`
+- [ ] Structure:
+  ```
+  1. Setup & Imports
+  2. Data Loading
+  3. Data Quality Analysis (with outputs)
+  4. Preprocessing & Augmentation Demo
+  5. Model Definition
+  6. Training (or load pre-trained)
+  7. Evaluation (all metrics)
+  8. Visualizations (confusion matrix, examples)
+  9. Inference Examples
+  10. Summary
+  ```
+- [ ] Requirements:
+  - Runs end-to-end without errors
+  - Clear markdown explanations
+  - All outputs visible (no need to re-run)
+  - Reproduces key results from report
+- [ ] Test notebook:
+  - Restart kernel and run all cells
+  - Verify no missing imports or path issues
+  - Check output matches report
+
+**Success Criteria:**
+- Notebook validates entire solution
+- Can be run by evaluators to verify results
+- Well-documented with markdown cells
+- All cells execute successfully
+
+**Estimated Time:** 2 hours
+
+---
+
+### **Task 4.3:** Finalize project documentation
+**Status:** ⬜ NOT STARTED  
+**Priority:** MEDIUM  
+**Objective:** Ensure all documentation is complete and polished
+
+**Implementation:**
+- [ ] Update main `README.md`:
+  ```markdown
+  # MNIST Digit Classifier
+  
+  CNN-based handwritten digit recognition with emphasis on data quality.
+  
+  ## Quick Links
+  - 🚀 [Live Demo](https://huggingface.co/spaces/...)
+  - 📄 [Full Report](docs/final_report.pdf)
+  - 📓 [Solution Notebook](notebooks/99_final_solution.ipynb)
+  
+  ## Project Structure
+  [Describe directories]
+  
+  ## Setup
+  [Installation instructions]
+  
+  ## Usage
+  [How to run training, evaluation, inference]
+  
+  ## Results
+  [Key metrics summary]
+  
+  ## Citation
+  [If applicable]
+  ```
+- [ ] Review all documentation files:
+  - `docs/DEVELOPMENT_WORKFLOW.md` - accurate?
+  - `docs/experiment_log.md` - complete?
+  - `scripts/README.md` - describes all modules?
+  - `.github/copilot-instructions.md` - up to date?
+- [ ] Clean up repository:
+  - Remove temporary files
+  - Archive experiment artifacts
+  - Organize `experiments/` directory
+  - Verify `.gitignore` working correctly
+- [ ] Final git commit:
+  - Commit message: `docs: finalize project documentation`
+  - Tag release: `git tag v1.0.0`
+
+**Success Criteria:**
+- All documentation files complete and accurate
+- README provides clear project overview
+- Repository clean and organized
+- Easy for others to understand and reproduce
+
+**Estimated Time:** 1 hour
+
+---
+
+## Next Steps
+
+**Immediate Actions (Start Here):**
+1. Complete Phase 0: Project Setup (1-2 hours)
+   - Create requirements.txt
+   - Configure .gitignore
+   - Initialize documentation
+   - Verify data integrity
+
+2. Start Phase 1: Data Pipeline (6-8 hours)
+   - Extract MnistDataloader to module
+   - Build data exploration notebook
+   - Implement quality analysis
+
+**Decision Points:**
+- **After Task 2.3** (baseline training): If accuracy ≥98%, proceed to deployment. If <98%, debug before continuing.
+- **After Task 2.4** (evaluation): Decide whether to pursue Task 2.5 (improvements) or move to deployment.
+- **After Task 3.3** (deployment): If deployment issues, can document locally and note limitations in report.
+
+**Risk Mitigation:**
+- **Time constraint**: Focus on baseline (skip Task 2.5) if time limited
+- **Deployment issues**: Have local demo ready as backup
+- **Model underperformance**: Document honestly and analyze why
+
+---
+
+## Metrics & Success Criteria
+
+**Model Performance Targets:**
+- Minimum: ≥ 97% test accuracy (acceptable baseline)
+- Target: ≥ 98% test accuracy (standard CNN performance)
+- Stretch: ≥ 99% test accuracy (competitive)
+
+**Code Quality Targets:**
+- Zero `ruff` linting errors
+- Type hints on all functions in `scripts/`
+- Docstrings on all public functions
+- Modular design (no monolithic files >500 lines)
+
+**Documentation Targets:**
+- Report: 20-30 pages covering all required topics
+- Data quality analysis: ≥3 pages with visualizations
+- SE practices section: clearly demonstrates best practices
+- Final notebook: executable and reproducible
+
+**Deployment Targets:**
+- Hugging Face Space live and accessible
+- Inference latency: <500ms per prediction
+- Interface intuitive (can be used without instructions)
+
+---
+
+## Workflow Reference
+
+**Standard Task Workflow:**
+1. **Review**: Read task requirements, check dependencies
+2. **Design**: Plan implementation approach
+3. **Implement**: Write code with tests
+4. **Run**: Execute and validate results
+5. **Assess**: Analyze outcomes, document findings
+6. **Commit**: Save progress with clear message
+
+**Commit Message Format:**
+```
+type: brief summary (50 chars max)
+
+Detailed description if needed (wrap at 72 chars).
+Include rationale, trade-offs, and any important context.
+
+Related tasks: #1.2, #2.3
+```
+
+**Types:** `feat`, `fix`, `docs`, `refactor`, `test`, `chore`
+
+**Branch Strategy:**
+- `main`: stable code only
+- `feature/<task-name>`: for each major task
+- Merge to main after task completion and validation
+
+---
+
+## Resources
+
+**Documentation:**
+- [MNIST Official](http://yann.lecun.com/exdb/mnist/)
+- [PyTorch Tutorials](https://pytorch.org/tutorials/)
+- [Hugging Face Spaces Docs](https://huggingface.co/docs/hub/spaces)
+- [Gradio Documentation](https://gradio.app/docs/)
+
+**Project Files:**
+- Planning: [planning.md](planning.md) (this file)
+- Workflow: [docs/DEVELOPMENT_WORKFLOW.md](docs/DEVELOPMENT_WORKFLOW.md)
+- Experiments: [docs/experiment_log.md](docs/experiment_log.md)
+- Problem: [docs/problem_statement.md](docs/problem_statement.md)
+
+**Code Reference:**
+- Data loader: [data/raw/read-mnist-dataset.ipynb](data/raw/read-mnist-dataset.ipynb)
+- AI instructions: [.github/copilot-instructions.md](.github/copilot-instructions.md)
+
+---
+
+## Notes
+
+**Critical Success Factors:**
+1. **Data quality analysis is paramount** - this is a key differentiator per spec
+2. **SE best practices must be demonstrable** - not just good code, but documented practices
+3. **Report quality matters** - 20-30 pages well-written, not just verbose
+4. **Reproducibility** - others must be able to run and verify results
+
+**Common Pitfalls to Avoid:**
+- ❌ Training on test set (data leakage)
+- ❌ Not fixing random seeds (non-reproducible)
+- ❌ Overfitting to validation set (test multiple times)
+- ❌ Skipping data quality analysis (required!)
+- ❌ Poor documentation (hard to evaluate)
+
+**Time Management:**
+- Baseline is sufficient - don't over-optimize
+- Data quality and documentation are as important as model performance
+- Leave buffer for report writing (6-8 hours)
+- Test deployment early (can be time sink if issues)
+
+**Questions/Uncertainties:**
+- [ ] Preferred report format? (PDF, Markdown, LaTeX)
+- [ ] Should tests be included? (Not required, but good practice)
+- [ ] Team submission or individual? (Affects documentation scope)
+- [ ] Evaluation rubric available? (Would inform priorities)
+
+---
+
+**Last Updated:** December 28, 2025  
+**Status:** Phase 0 - Ready to begin  
+**Next Task:** Task 0.1 - Create requirements.txt

requirements.txt

@@ -0,0 +1,25 @@
+# Core dependencies for MNIST digit classification project
+# Python 3.10 (conda environment: ai_engg)
+
+# Core ML libraries
+numpy==1.24.3
+matplotlib==3.7.1
+torch==2.0.1
+torchvision==0.15.2
+scikit-learn==1.3.0
+
+# Experiment tracking
+mlflow==2.9.2
+
+# Deployment
+gradio==3.50.0
+pillow==10.0.0
+
+# Development
+jupyter==1.0.0
+ruff==0.0.270
+pytest==7.4.0
+
+# Additional utilities
+pandas==2.0.3
+seaborn==0.12.2

scripts/README.md

@@ -0,0 +1,72 @@
+# Scripts Directory
+
+This directory contains reusable Python modules for the MNIST digit classification project.
+
+## Modules
+
+### Data Processing
+- **`data_loader.py`** - MNIST data loading from IDX binary format
+  - `MnistDataloader` class for loading train/test data
+  
+- **`preprocessing.py`** - Data preprocessing and PyTorch Dataset
+  - `MnistDataset` - PyTorch Dataset with normalization
+  - `create_train_val_split()` - Split training data into train/val
+  
+- **`data_quality.py`** - Data quality analysis functions
+  - Quality checks: missing values, outliers, class balance
+  - `generate_quality_report()` - Comprehensive quality report
+  
+- **`augmentation.py`** - Data augmentation pipeline
+  - `get_augmentation_pipeline()` - Transform composition for training
+
+### Model
+- **`models.py`** - CNN architectures
+  - `BaselineCNN` - 2-layer CNN baseline model
+  
+- **`train.py`** - Training pipeline
+  - `train_epoch()` - Single epoch training
+  - `validate()` - Validation evaluation
+  - `train_model()` - Complete training loop with MLflow logging
+  
+- **`evaluate.py`** - Model evaluation
+  - `evaluate_model()` - Comprehensive metrics computation
+  - Accuracy, precision, recall, confusion matrix
+  
+- **`inference.py`** - Production inference
+  - `DigitClassifier` - Inference wrapper for deployment
+
+### Experiment Tracking
+- **`mlflow_setup.py`** - MLflow configuration
+  - `setup_mlflow()` - Initialize MLflow experiment
+  - Tracking URI and experiment management
+
+### Utilities
+- **`launch_mlflow_ui.sh`** - Launch MLflow UI server
+- **`docker_start.sh`** - Start Docker containers
+- **`docker_stop.sh`** - Stop Docker containers
+- **`docker_logs.sh`** - View Docker container logs
+
+## Usage
+
+All modules are designed to be imported and used in notebooks or other scripts:
+
+```python
+# Example: Load data
+from scripts.data_loader import MnistDataloader
+
+loader = MnistDataloader(
+    training_images_filepath='data/raw/train-images.idx3-ubyte',
+    training_labels_filepath='data/raw/train-labels.idx1-ubyte',
+    test_images_filepath='data/raw/t10k-images.idx3-ubyte',
+    test_labels_filepath='data/raw/t10k-labels.idx1-ubyte'
+)
+(x_train, y_train), (x_test, y_test) = loader.load_data()
+```
+
+## Development Guidelines
+
+- All functions include type hints
+- All public functions have docstrings
+- Follow naming conventions (snake_case for functions, PascalCase for classes)
+- Run `ruff check . --fix` before committing
+- Add unit tests in `tests/` directory for critical functions