--- license: mit library_name: pytorch tags: - computer-vision - image-segmentation - edge-detection - line-art - anime datasets: - custom metrics: - dice - iou pipeline_tag: image-segmentation --- # Anime Line Art Extraction Segmentation Model ## Model Description ### Overview This model performs automatic line art extraction from anime images using a deep learning segmentation approach. The goal of the model is to identify edge structures that form the visual outlines of characters and objects in anime frames. Extracting clean line art typically requires manual tracing by artists or complex rule-based algorithms. This project explores whether a deep learning segmentation model can learn pixel-level edge structures directly from images. The model takes an RGB anime frame as input and produces a binary edge mask representing the predicted line art structure. ### Problem & Context Problem: Extracting clean line art from images normally requires manual tracing by hand or specialized algorithms. Why this matters: - Speed up animation production pipelines - Assist manga and illustration workflows - Help beginners learn drawing by tracing outlines - Improve visual quality by upscaling blurry line art - Generate datasets for generative AI models How computer vision helps: Deep learning segmentation models can learn pixel-level edge structures directly from images. ### Training Approach The model was trained as a semantic segmentation model using the PyTorch segmentation framework. Frameworks used: - PyTorch - segmentation_models_pytorch Since no pretrained model exists specifically for anime line extraction, the model was trained using a custom dataset and automatically generated edge masks. ### Intended Use Cases Potential applications include: - Animation pipelines – converting frames into base line structures - Digital art tools – assisting artists by generating sketch outlines - Image upscaling workflows – improving visual quality of blurry lines - Dataset generation – automatically creating line art datasets for training generative models Example research question explored in this project: Can a segmentation model trained on edge masks produce usable line art for artistic workflows? -------------------------------------------------- # Training Data ## Dataset Source Images were collected manually from screenshots of anime episodes. The dataset was assembled specifically for this project to capture common line art structures present in anime animation. Dataset characteristics: Total images: 480 Original resolution: 1920×1080 Training resolution: 256×256 Task type: Binary segmentation ## Classes Although this is a binary segmentation task, the detected edges represent multiple visual structures: - Character outlines - Hair edges - Facial outlines - Clothing folds - Background structures Pixel labels: 0 = background 1 = line / edge ## Dataset Split Train: 384 images Validation: 96 images Test: Not used A separate test set was not included due to the relatively small dataset size. The validation set was used to monitor training performance and evaluate model results. ## Data Collection Methodology Images were collected manually from anime episode screenshots. Frames were chosen to capture a variety of characters, poses, lighting conditions, and scene compositions. All images were resized to 256×256 resolution to standardize input dimensions for training. ## Annotation Process Manually labeling line art masks for hundreds of images would be extremely time consuming. Instead, an automated annotation pipeline was used to approximate line structures. Annotation pipeline: Anime Image → Grayscale conversion → Canny edge detection → Binary edge mask Tools used: Python, Google Colab (Jupyter Notebook), OpenCV, PyTorch Work performed for the dataset: - Collected ~480 anime images - Generated masks automatically using Canny edge detection - Manually inspected mask quality visually This approach allowed rapid dataset creation while still ensuring that the generated masks captured meaningful line structures. ## Data Augmentation No data augmentation techniques were applied. Images were only resized and normalized during preprocessing. ## Known Dataset Biases Several limitations exist in the dataset: - Images are exclusively anime style, creating stylistic bias - Edge masks generated automatically contain noise - Some thin edges may be missing due to limitations of Canny detection - Dataset size is relatively small for deep learning segmentation -------------------------------------------------- # Training Procedure ## Training Framework The model was implemented using: PyTorch segmentation_models_pytorch This library provides segmentation architectures suitable for pixel-level prediction tasks. ## Model Architecture Architecture: Encoder: ResNet18 Decoder: U-Net Input: RGB image Output: binary edge mask U-Net was selected because it performs well for segmentation tasks and works effectively with relatively small datasets. ## Training Hardware Training was conducted using Google Colab. Typical environment: GPU: NVIDIA T4 VRAM: ~16 GB Training time: approximately 1–2 hours ## Hyperparameters Epochs: 30 Batch size: 8 Optimizer: Adam Learning rate: 0.0001 Loss function: Binary Cross Entropy + Dice Loss ## Preprocessing Steps Images were preprocessed using: Resize to 256×256 Normalize using ImageNet statistics mean = [0.485, 0.456, 0.406] std = [0.229, 0.224, 0.225] -------------------------------------------------- # Evaluation Results ## Metrics Because this project uses semantic segmentation rather than object detection, evaluation metrics are calculated at the pixel level. Metrics used: Dice Coefficient – measures overlap between predicted masks and ground truth masks Intersection over Union (IoU) – measures intersection divided by union of predicted and ground truth masks ## Validation Performance Dice coefficient: ~0.35 IoU: ~0.21 These metrics indicate that the model is able to detect meaningful edge structures but struggles with extremely thin line details. ## Key Observations What worked well: - Learned major character outlines - Captured hair boundaries - Detected facial structures Failure cases: - Small thin lines - Dark scenes - Shading lines interpreted as edges - Excessive background detail These results show that the model learned meaningful edge structures despite the noisy annotations generated from Canny edge detection. ## Visual Examples Typical evaluation visualizations include: - Input anime frame - Ground truth edge mask - Model predicted mask These comparisons help visually evaluate whether predicted edges align with important structures in the image. ## Performance Analysis The model demonstrates that segmentation networks can learn edge patterns from anime images even when trained with automatically generated masks. However, the task presents several challenges: 1. Thin line structures are difficult for segmentation models 2. Automatic annotations introduce noise 3. Low contrast scenes reduce edge detectability Because the model was only trained for 30 epochs, additional training may improve performance. However, improving annotation quality or training at higher resolution would likely have a larger impact. -------------------------------------------------- # Limitations and Biases ## Known Failure Cases The model struggles with: - Extremely thin lines - Low contrast scenes - Dark shading regions - Highly detailed backgrounds These cases often produce incomplete or noisy edge predictions. ## Annotation Noise Ground truth labels were generated automatically using Canny edge detection. This introduces issues such as: - Missing edges - False edges from shading - Broken line segments Because the model learns from these masks, the maximum achievable accuracy is limited by the quality of the annotations. ## Dataset Bias The dataset contains only anime frames, introducing strong stylistic bias. The model may perform poorly on: - Photographs - Western illustration styles - Non-anime artwork ## Resolution Limitations Images were resized from 1920×1080 to 256×256 for training. This downscaling removes fine details and makes thin lines harder to detect. ## Sample Size Limitations The dataset contains only 480 images, which is relatively small for training deep neural networks. A larger dataset would likely improve generalization. ## Inappropriate Use Cases This model should not be used for: - Photographic edge detection - Medical image segmentation - Object detection tasks The model is specifically designed for anime-style line structure extraction. -------------------------------------------------- # Future Work Possible improvements include: - Expanding the dataset to thousands of images - Training at higher resolution (512×512 or higher) - Improving annotation quality with manual corrections - Exploring diffusion-based line reconstruction models Additional research directions include: - Object detection models for automatic removal of occlusions - Line art upscaling techniques - Using detected edges for stitching animation panning shots