{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# \ud83d\udc04 Cattle Breed Classifier \u2014 Colab Runner\n",
                "\n",
                "This notebook sets up the environment on **Google Colab** and runs all training notebooks sequentially.\n",
                "\n",
                "### What this does\n",
                "1. Verifies GPU availability\n",
                "2. Clones the repo (dataset included)\n",
                "3. Installs missing dependencies\n",
                "4. Runs each notebook in order:\n",
                "   - `00_data_audit` \u2192 `01_mlp_baseline` \u2192 `02_cnn_from_scratch` \u2192 `03_resnet_transfer_learning` \u2192 `04_vit_transfer_learning` \u2192 `05_model_comparison`\n",
                "\n",
                "> \u26a0\ufe0f **Runtime**: Select **GPU** runtime before running: *Runtime \u2192 Change runtime type \u2192 T4 GPU*"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 0. Check GPU"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import torch\n",
                "print(f'PyTorch version: {torch.__version__}')\n",
                "print(f'CUDA available:  {torch.cuda.is_available()}')\n",
                "if torch.cuda.is_available():\n",
                "    print(f'GPU:             {torch.cuda.get_device_name(0)}')\n",
                "    print(f'Memory:          {torch.cuda.get_device_properties(0).total_mem / 1024**3:.1f} GB')\n",
                "else:\n",
                "    print('\u26a0\ufe0f  No GPU detected. Go to Runtime \u2192 Change runtime type \u2192 T4 GPU')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 1. Clone Repository"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import os\n",
                "\n",
                "REPO_URL = 'https://github.com/ajitsingh98/cattle-breed-classifier-webapp.git'\n",
                "REPO_DIR = '/content/cattle-breed-classifier-webapp'\n",
                "\n",
                "if not os.path.exists(REPO_DIR):\n",
                "    !git clone {REPO_URL} {REPO_DIR}\n",
                "else:\n",
                "    print(f'Repo already cloned at {REPO_DIR}')\n",
                "    !cd {REPO_DIR} && git pull\n",
                "\n",
                "os.chdir(REPO_DIR)\n",
                "print(f'\\nWorking directory: {os.getcwd()}')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 2. Install Dependencies"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Colab already has PyTorch, torchvision, numpy, pandas, matplotlib, seaborn, PIL, scikit-learn.\n",
                "# Install only the missing packages.\n",
                "!pip install -q PyYAML gdown aiofiles aiohttp nest_asyncio"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 3. Set Up Python Path"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import sys\n",
                "from pathlib import Path\n",
                "\n",
                "PROJECT_ROOT = Path(REPO_DIR).resolve()\n",
                "if str(PROJECT_ROOT) not in sys.path:\n",
                "    sys.path.insert(0, str(PROJECT_ROOT))\n",
                "\n",
                "# Create artifact directories\n",
                "for d in ['ml/artifacts/manifests', 'ml/artifacts/checkpoints',\n",
                "          'ml/artifacts/figures', 'ml/artifacts/logs', 'ml/artifacts/reports']:\n",
                "    (PROJECT_ROOT / d).mkdir(parents=True, exist_ok=True)\n",
                "\n",
                "# Verify dataset exists\n",
                "data_dir = PROJECT_ROOT / 'Cattle_Resized'\n",
                "class_dirs = sorted([d for d in data_dir.iterdir() if d.is_dir()])\n",
                "total_images = sum(len(list(d.glob('*'))) for d in class_dirs)\n",
                "print(f'Dataset: {len(class_dirs)} breeds, {total_images} images')\n",
                "print(f'Project root: {PROJECT_ROOT}')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 4. Run All Notebooks\n",
                "\n",
                "Each notebook is executed in order using `nbconvert`. Output is displayed inline.\n",
                "\n",
                "You can also **skip this cell** and open each notebook individually from the file browser on the left:\n",
                "> `cattle-breed-classifier-webapp/ml/notebooks/`"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "!pip install -q papermill jupyter ipykernel"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import subprocess, time\n",
                "\n",
                "NOTEBOOKS = [\n",
                "    '00_data_audit.ipynb',\n",
                "    '01_mlp_baseline.ipynb',\n",
                "    '02_cnn_from_scratch.ipynb',\n",
                "    '03_resnet_transfer_learning.ipynb',\n",
                "    '04_vit_transfer_learning.ipynb',\n",
                "    '05_model_comparison.ipynb',\n",
                "]\n",
                "\n",
                "NOTEBOOK_DIR = PROJECT_ROOT / 'ml' / 'notebooks'\n",
                "results = {}\n",
                "\n",
                "for nb_name in NOTEBOOKS:\n",
                "    nb_path = NOTEBOOK_DIR / nb_name\n",
                "    print(f'\\n{\"=\" * 60}')\n",
                "    print(f'\u25b6 Running: {nb_name}')\n",
                "    print(f'{\"=\" * 60}')\n",
                "\n",
                "    start = time.time()\n",
                "    # We use papermill because it supports streaming cell outputs (--log-output)\n",
                "    result = subprocess.run(\n",
                "        [\n",
                "            \"papermill\",\n",
                "            str(nb_path),\n",
                "            str(nb_path), # overwrite inplace so output is saved in the notebook\n",
                "            \"--log-output\",\n",
                "            \"--kernel\", \"python3\"\n",
                "        ],\n",
                "        cwd=str(PROJECT_ROOT),\n",
                "        env={**os.environ, \"PYTHONPATH\": str(PROJECT_ROOT)},\n",
                "    )\n",
                "    elapsed = time.time() - start\n",
                "\n",
                "    if result.returncode == 0:\n",
                "        status = '\u2705 PASSED'\n",
                "    else:\n",
                "        status = '\u274c FAILED'\n",
                "\n",
                "    results[nb_name] = {'status': status, 'time': elapsed}\n",
                "    print(f'{status} ({elapsed:.1f}s)')\n",
                "\n",
                "# Summary\n",
                "print(f'\\n{\"=\" * 60}')\n",
                "print('Summary')\n",
                "print(f'{\"=\" * 60}')\n",
                "for nb, info in results.items():\n",
                "    print(f\"  {info['status']}  {nb:45s}  {info['time']:6.1f}s\")\n",
                "total_time = sum(r['time'] for r in results.values())\n",
                "print(f'\\nTotal time: {total_time/60:.1f} minutes')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 5. Download Artifacts (Optional)\n",
                "\n",
                "After training completes, download the best model checkpoint and reports."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# List generated artifacts\n",
                "import glob\n",
                "\n",
                "print('=== Checkpoints ===')\n",
                "for f in sorted(glob.glob(str(PROJECT_ROOT / 'ml/artifacts/checkpoints/*.pth'))):\n",
                "    size_mb = os.path.getsize(f) / 1024 / 1024\n",
                "    print(f'  {Path(f).name:40s} {size_mb:8.1f} MB')\n",
                "\n",
                "print('\\n=== Figures ===')\n",
                "for f in sorted(glob.glob(str(PROJECT_ROOT / 'ml/artifacts/figures/*.png'))):\n",
                "    print(f'  {Path(f).name}')\n",
                "\n",
                "print('\\n=== Reports ===')\n",
                "for f in sorted(glob.glob(str(PROJECT_ROOT / 'ml/artifacts/reports/*.json'))):\n",
                "    print(f'  {Path(f).name}')"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Zip and download artifacts\n",
                "!cd {REPO_DIR} && zip -r /content/artifacts.zip ml/artifacts/\n",
                "\n",
                "from google.colab import files\n",
                "files.download('/content/artifacts.zip')\n",
                "print('\\n\u2705 Download started! Check your browser downloads.')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "---\n",
                "\n",
                "### \ud83d\udca1 Running Notebooks Individually\n",
                "\n",
                "Instead of the automated runner above, you can open each notebook directly:\n",
                "\n",
                "1. In the **Colab file browser** (left panel), navigate to:  \n",
                "   `cattle-breed-classifier-webapp/ml/notebooks/`\n",
                "2. Double-click any `.ipynb` file to open it in a new tab\n",
                "3. Run cells with `Shift+Enter`\n",
                "\n",
                "**Important**: Each notebook auto-detects the project root, so they work both from the automated runner and when opened individually."
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "name": "python",
            "version": "3.10.0"
        },
        "colab": {
            "provenance": [],
            "gpuType": "T4"
        },
        "accelerator": "GPU"
    },
    "nbformat": 4,
    "nbformat_minor": 4
}