{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 01 \u2014 Data and preprocessing\n", "\n", "Before any model can learn, the data has to be *clean*, *consistent*, and *the right size*. This notebook walks through every preprocessing step the project performs, the math behind each, and how to verify it with code.\n", "\n", "## Roadmap\n", "\n", "1. The datasets we use, and why\n", "2. The shape of one training example\n", "3. **Why downscale?** \u2014 memory math\n", "4. **How we downscale** \u2014 LANCZOS resampling explained\n", "5. **Quality scoring** \u2014 variance-of-Laplacian for blur, brightness, contrast\n", "6. Augmentation: RandAugment, MixUp, CutMix \u2014 visualised\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# === Colab bootstrap ===\n", "# Safe to re-run. On a local clone with `pip install -e .` already done this\n", "# is a no-op; on Colab it clones the repo + installs deps the first time.\n", "import os, sys, subprocess\n", "from pathlib import Path\n", "\n", "REPO_URL = \"https://github.com/theDocWho/car-crash-fix-amount-predictor.git\"\n", "REPO_DIR = Path(\"car-crash-fix-amount-predictor\")\n", "\n", "IN_COLAB = \"google.colab\" in sys.modules\n", "if IN_COLAB and not REPO_DIR.exists():\n", " subprocess.run([\"git\", \"clone\", \"--depth\", \"1\", REPO_URL], check=True)\n", "if IN_COLAB:\n", " os.chdir(REPO_DIR)\n", " subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\",\n", " \"-r\", \"requirements.txt\"], check=True)\n", "\n", "# Make `ccdp` importable whether or not the package was installed editable.\n", "src_path = Path(\"src\").resolve()\n", "if str(src_path) not in sys.path:\n", " sys.path.insert(0, str(src_path))\n", "\n", "print(\"ccdp path:\", src_path)\n", "print(\"running in Colab:\", IN_COLAB)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Datasets\n", "\n", "| Dataset | Used for | Source |\n", "|---|---|---|\n", "| **CarDD** (Wang et al. 2023) | Damage classifier + detector training | Kaggle: `eduardo4jesus/cardd` |\n", "| **Stanford Cars** | Car make/model identifier | Kaggle: `eduardo4jesus/stanford-cars-dataset` |\n", "\n", "The cost target is **synthetic** \u2014 derived from a versioned parts-cost catalog \u00d7 car age \u00d7 a small Gaussian noise term. There is no public dataset that pairs damaged-car photos with real repair invoices, and inventing one would be dishonest. We document this trade-off in `PLAN.md \u00a73`.\n", "\n", "## 2. One training example\n", "\n", "```\n", "{\n", " \"image\": ,\n", " \"damage\": [\"dent\", \"scratch\"], # multi-label\n", " \"bboxes\": [BBox(damage_type=\"dent\", x_center=..., y_center=..., width=..., height=...)],\n", " \"make\": \"toyota\",\n", " \"model\": \"camry\",\n", " \"year\": 2015,\n", " \"cost_usd\": 812.50, # synthetic target\n", "}\n", "```\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Peek at the schema in code form.\n", "from ccdp.data.schema import DAMAGE_TYPES, BBox, infer_part_from_damage\n", "import inspect\n", "\n", "print(\"Damage classes:\", DAMAGE_TYPES)\n", "print()\n", "print(\"BBox fields:\")\n", "print(inspect.getsource(BBox))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Why downscale?\n", "\n", "A modern phone shoots photos at ~12 megapixels. Stored as RGB uint8 that is:\n", "\n", "$$\n", "12{,}000{,}000 \\text{ pixels} \\times 3 \\text{ channels} \\times 1 \\text{ byte} = 36 \\text{ MB per image}\n", "$$\n", "\n", "For a batch of 32, that's **1.15 GB** of pixels \u2014 more than free-Colab T4 VRAM headroom after the model is loaded. And **the model never sees that resolution anyway**: ResNet50 inputs are 224\u00d7224, YOLOv8 is 640\u00d7640. So we resize once, up front, to a long edge of **1600 px**.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Concrete memory comparison.\n", "def bytes_for(w, h, c=3, dtype_bytes=1):\n", " return w * h * c * dtype_bytes\n", "\n", "raw_phone = bytes_for(4000, 3000)\n", "downscaled = bytes_for(1600, 1200)\n", "classifier = bytes_for(224, 224)\n", "detector = bytes_for(640, 640)\n", "\n", "print(f\"Raw phone photo (4000x3000): {raw_phone / 1e6:6.1f} MB\")\n", "print(f\"Downscaled (1600x1200): {downscaled / 1e6:6.1f} MB\")\n", "print(f\"Classifier input (224x224): {classifier / 1e3:6.1f} KB\")\n", "print(f\"Detector input (640x640): {detector / 1e3:6.1f} KB\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. How we downscale \u2014 LANCZOS resampling\n", "\n", "When you shrink an image, you have to pick which pixels of the original to keep. The naive way is **nearest-neighbour**: for each output pixel, copy the closest input pixel. It's fast but creates jagged edges \u2014 bad for our models, which are trying to detect dent and scratch *edges*.\n", "\n", "**LANCZOS** is a higher-order filter that blends multiple input pixels using the function\n", "\n", "$$\n", "L(x) = \\begin{cases} \\text{sinc}(x)\\,\\text{sinc}(x/a) & |x| < a \\\\ 0 & \\text{otherwise}\\end{cases}\n", "$$\n", "\n", "where $\\text{sinc}(x) = \\sin(\\pi x) / (\\pi x)$ and $a$ (the *kernel size*) is typically 3. The result preserves edges better than nearest-neighbour or bilinear, which is why Pillow recommends it for downsizing.\n", "\n", "Let's see the difference visually.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from PIL import Image\n", "import matplotlib.pyplot as plt\n", "\n", "# Build a synthetic high-frequency test pattern (alternating black/white bars).\n", "big = np.zeros((400, 400, 3), dtype=np.uint8)\n", "big[:, ::4] = 255 # vertical stripes every 4 pixels\n", "img = Image.fromarray(big)\n", "\n", "nearest = img.resize((100, 100), resample=Image.NEAREST)\n", "bilinear = img.resize((100, 100), resample=Image.BILINEAR)\n", "lanczos = img.resize((100, 100), resample=Image.LANCZOS)\n", "\n", "fig, axes = plt.subplots(1, 4, figsize=(12, 4))\n", "for ax, im, title in zip(axes, [img, nearest, bilinear, lanczos],\n", " [\"original 400x400\", \"NEAREST\", \"BILINEAR\", \"LANCZOS\"]):\n", " ax.imshow(im); ax.set_title(title); ax.axis(\"off\")\n", "plt.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "LANCZOS produces the smoothest stripes \u2014 the others alias into Moir\u00e9 patterns. Now let's call the project's own downscaler.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from ccdp.preprocess.pipeline import normalize_for_inference, quality_report\n", "from PIL import Image\n", "import numpy as np\n", "\n", "# Make a fake \"phone photo\" \u2014 4000x3000 random noise just for size demo.\n", "fake = Image.fromarray(np.random.randint(0, 255, (3000, 4000, 3), dtype=np.uint8))\n", "print(\"input :\", fake.size)\n", "small = normalize_for_inference(fake, max_long_edge=1600)\n", "print(\"output:\", small.size)\n", "print(\"note: long edge clamped to 1600, aspect ratio preserved.\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Quality scoring \u2014 is the image even usable?\n", "\n", "If a user uploads a phone photo so blurry that even a human can't tell a dent from a reflection, the model will guess wildly and the cost estimate will be junk. So we **measure** image quality and surface it in the API response.\n", "\n", "### Variance of the Laplacian (the standard blur metric)\n", "\n", "The Laplacian is a 2D second-derivative operator. Apply this 3\u00d73 kernel as a convolution to a greyscale image:\n", "\n", "$$\n", "\\nabla^2 = \\begin{bmatrix} 0 & 1 & 0 \\\\ 1 & -4 & 1 \\\\ 0 & 1 & 0 \\end{bmatrix}\n", "$$\n", "\n", "For a sharp image with crisp edges, the result has *high variance* \u2014 lots of strongly-positive and strongly-negative responses near edges. A blurry image has *low variance* because edges are smeared.\n", "\n", "This single number \u2014 `Var(\u2207\u00b2 image)` \u2014 is the textbook \"is it blurry?\" metric (Pech-Pacheco et al., 2000).\n", "\n", "Let's reproduce it.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from PIL import Image\n", "import matplotlib.pyplot as plt\n", "from ccdp.preprocess.pipeline import _sharpness_score, quality_report\n", "\n", "# Build a sharp image (edges) and a blurry copy of the same.\n", "sharp_arr = np.zeros((256, 256), dtype=np.uint8)\n", "sharp_arr[64:192, 64:192] = 255 # crisp white square on black\n", "sharp = Image.fromarray(sharp_arr).convert(\"RGB\")\n", "\n", "blurry = sharp.filter(__import__(\"PIL.ImageFilter\", fromlist=[\"GaussianBlur\"]).GaussianBlur(radius=8))\n", "\n", "print(f\"Sharpness sharp: {_sharpness_score(sharp):8.1f}\")\n", "print(f\"Sharpness blurry: {_sharpness_score(blurry):8.1f}\")\n", "\n", "fig, axes = plt.subplots(1, 2, figsize=(8, 4))\n", "axes[0].imshow(sharp); axes[0].set_title(\"sharp\"); axes[0].axis(\"off\")\n", "axes[1].imshow(blurry); axes[1].set_title(\"blurry\"); axes[1].axis(\"off\")\n", "plt.show()\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# The full quality report the API returns:\n", "quality_report(sharp)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Augmentation \u2014 making models robust\n", "\n", "A model that has only ever seen well-lit, centered, crisp photos will fail on real user uploads. We *augment* the training set by randomly distorting each image differently each epoch. Three popular techniques:\n", "\n", "- **RandAugment** \u2014 pick N random transforms (rotate, color jitter, posterize\u2026) from a list and chain them. Reduces hyperparameter tuning.\n", "- **MixUp** \u2014 for two images $x_i, x_j$ with labels $y_i, y_j$, train on $(\\lambda x_i + (1-\\lambda) x_j,\\ \\lambda y_i + (1-\\lambda) y_j)$ where $\\lambda \\sim \\text{Beta}(\\alpha, \\alpha)$.\n", "- **CutMix** \u2014 paste a random rectangle from image B onto image A; mix labels by the area ratio.\n", "\n", "The intuition: these force the model to make decisions based on *content* rather than memorising exact pixel arrangements.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Visualise MixUp on two synthetic images.\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "img_a = np.zeros((128, 128, 3), dtype=np.float32); img_a[..., 0] = 1.0 # red\n", "img_b = np.zeros((128, 128, 3), dtype=np.float32); img_b[..., 2] = 1.0 # blue\n", "\n", "fig, axes = plt.subplots(1, 5, figsize=(14, 3))\n", "for ax, lam in zip(axes, [1.0, 0.75, 0.5, 0.25, 0.0]):\n", " mix = lam * img_a + (1 - lam) * img_b\n", " ax.imshow(mix); ax.set_title(f\"\u03bb={lam}\"); ax.axis(\"off\")\n", "plt.suptitle(\"MixUp: linear blend between two images and their labels\")\n", "plt.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Next:** open `02_classifier_resnet50.ipynb` to learn what a convolutional network actually computes, why ResNet50 has *skip connections*, and how to train one yourself.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.11" } }, "nbformat": 4, "nbformat_minor": 5 }