{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Video Subtitle Remover - Google Colab\n", "\n", "This notebook allows you to run Video Subtitle Remover in Google Colab with free GPU access.\n", "\n", "**New in this version:**\n", "- \ud83d\udcdd Subtitle extraction to SRT files with OCR\n", "- \ud83e\udd16 Automatic model downloading from Hugging Face\n", "- \u26a1 Improved performance\n", "\n", "**Requirements:**\n", "- Google account\n", "- Video file (upload to Google Drive or use sample)\n", "\n", "**Colab Environment:**\n", "- Python: 3.10\n", "- CUDA: 12.2 (Tesla T4 GPU)\n", "- GPU Memory: 15GB\n", "\n", "**Recommended Algorithm:** STTN (fastest for Colab's limited runtime)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Check GPU and Environment" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# Check GPU availability\n", "!nvidia-smi\n", "\n", "import torch\n", "print(f\"Python version: 3.10\")\n", "print(f\"PyTorch version: {torch.__version__}\")\n", "print(f\"CUDA available: {torch.cuda.is_available()}\")\n", "if torch.cuda.is_available():\n", " print(f\"CUDA version: {torch.version.cuda}\")\n", " print(f\"GPU: {torch.cuda.get_device_name(0)}\")\n", " print(f\"GPU Memory: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f} GB\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2: Clone Repository" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Clone the repository\n", "!git clone https://huggingface.co/Rasta02/dataku\n", "%cd dataku" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2.5: Verify Installation\n", "\n", "Quick check that the repository was cloned successfully." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# Verify the repository structure\n", "import os\n", "\n", "repo_path = '/content/dataku'\n", "if os.path.exists(repo_path):\n", " print('\u2713 Repository cloned successfully')\n", " print(f' Path: {repo_path}')\n", "else:\n", " print('\u274c Repository not found')\n", " print(' Please run Step 2 first')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 3: Install Dependencies\n", "\n", "Colab already has many packages. We'll install missing ones." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# Install dependencies\n", "# Note: Colab already has torch, torchvision, opencv-python, numpy, etc.\n", "!pip install -q filesplit==3.0.2 albumentations scikit-image imgaug pyclipper lmdb\n", "!pip install -q PyYAML omegaconf tqdm easydict scikit-learn pandas webdataset\n", "!pip install -q protobuf av einops paddleocr paddle2onnx onnxruntime-gpu\n", "\n", "# Install PaddlePaddle GPU version (compatible with Colab)\n", "!pip install -q paddlepaddle-gpu==2.6.2\n", "\n", "# Advanced Inpainting Models (Optional - only install if using these modes)\n", "# Uncomment the line below to enable Stable Diffusion, DiffuEraser, E2FGVI\n", "# !pip install -q diffusers transformers accelerate\n", "\n", "print(\"\u2713 Dependencies installed!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4: Mount Google Drive (Optional)\n", "\n", "Mount your Google Drive to access videos stored there." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from google.colab import drive\n", "drive.mount('/content/drive')\n", "\n", "# Your videos will be accessible at:\n", "# /content/drive/MyDrive/your_video.mp4" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 5: Configure Settings\n", "\n", "Adjust these settings based on your needs:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# === CONFIGURATION ===\n", "\n", "# Algorithm selection\n", "# Options:\n", "# 'STTN' - Fast, real-time video (recommended for Colab)\n", "# 'LAMA' - High quality for images/animation\n", "# 'PROPAINTER' - Best quality, very slow, high VRAM\n", "# 'SD' - Stable Diffusion (NEW - requires extra install above)\n", "# 'DIFFUERASER' - Specialized subtitle removal (Coming soon)\n", "# 'E2FGVI' - Fast flow-guided (Coming soon)\n", "ALGORITHM = 'STTN'\n", "\n", "# STTN Settings (recommended for Colab)\n", "STTN_SKIP_DETECTION = True # Much faster, processes entire subtitle area\n", "STTN_MAX_LOAD_NUM = 40 # Reduce if OOM (30-50 for T4 GPU)\n", "STTN_NEIGHBOR_STRIDE = 5\n", "STTN_REFERENCE_LENGTH = 10\n", "\n", "# LAMA Settings\n", "LAMA_SUPER_FAST = False # Set True for faster but lower quality\n", "\n", "# ProPainter Settings (requires 16GB+ GPU, not recommended for Colab)\n", "PROPAINTER_MAX_LOAD_NUM = 40 # Very low for T4 GPU\n", "\n", "# Stable Diffusion Settings (NEW)\n", "SD_STEPS = 50 # More steps = better quality but slower\n", "SD_GUIDANCE_SCALE = 7.5 # How much to follow the prompt\n", "SD_PROMPT = \"natural scene, high quality\" # Text guidance\n", "\n", "# Video path (change this)\n", "# Option 1: Use sample video\n", "VIDEO_PATH = '/content/dataku/test/test.mp4'\n", "\n", "# Option 2: Use video from Google Drive (uncomment)\n", "# VIDEO_PATH = '/content/drive/MyDrive/my_video.mp4'\n", "\n", "# Subtitle area (optional, in pixels: ymin, ymax, xmin, xmax)\n", "# None = auto-detect subtitle area\n", "SUBTITLE_AREA = None\n", "\n", "# Example: Bottom 20% of 1080p video\n", "# SUBTITLE_AREA = (864, 1080, 0, 1920)\n", "\n", "print(f\"Configuration:\")\n", "print(f\" Algorithm: {ALGORITHM}\")\n", "print(f\" Video: {VIDEO_PATH}\")\n", "print(f\" Subtitle area: {SUBTITLE_AREA or 'Auto-detect'}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 6: Apply Configuration" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# Modify config.py with our settings\n", "import sys\n", "sys.path.insert(0, '/content/dataku')\n", "sys.path.insert(0, '/content/dataku/backend')\n", "\n", "from backend import config\n", "from backend.config import InpaintMode\n", "\n", "# Apply algorithm selection\n", "if ALGORITHM == 'STTN':\n", " config.MODE = InpaintMode.STTN\n", " config.STTN_SKIP_DETECTION = STTN_SKIP_DETECTION\n", " config.STTN_MAX_LOAD_NUM = STTN_MAX_LOAD_NUM\n", " config.STTN_NEIGHBOR_STRIDE = STTN_NEIGHBOR_STRIDE\n", " config.STTN_REFERENCE_LENGTH = STTN_REFERENCE_LENGTH\n", "elif ALGORITHM == 'LAMA':\n", " config.MODE = InpaintMode.LAMA\n", " config.LAMA_SUPER_FAST = LAMA_SUPER_FAST\n", "elif ALGORITHM == 'PROPAINTER':\n", " config.MODE = InpaintMode.PROPAINTER\n", " config.PROPAINTER_MAX_LOAD_NUM = PROPAINTER_MAX_LOAD_NUM\n", "elif ALGORITHM == 'SD':\n", " config.MODE = InpaintMode.STABLE_DIFFUSION\n", " config.SD_STEPS = SD_STEPS\n", " config.SD_GUIDANCE_SCALE = SD_GUIDANCE_SCALE\n", " config.SD_PROMPT = SD_PROMPT\n", "elif ALGORITHM == 'DIFFUERASER':\n", " config.MODE = InpaintMode.DIFFUERASER\n", " print('\u26a0\ufe0f DiffuEraser not yet implemented, will fall back to LAMA')\n", "elif ALGORITHM == 'E2FGVI':\n", " config.MODE = InpaintMode.E2FGVI\n", " print('\u26a0\ufe0f E2FGVI not yet implemented, will fall back to STTN')\n", "\n", "print(f\"\u2713 Configuration applied!\")\n", "print(f\" Using device: {config.device}\")\n", "print(f\" Mode: {config.MODE.value}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 7: Process Video" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "import multiprocessing\n", "from backend.main import SubtitleRemover\n", "import os\n", "\n", "# Check if video exists\n", "if not os.path.exists(VIDEO_PATH):\n", " print(f\"\u274c Error: Video not found at {VIDEO_PATH}\")\n", " print(\"Please upload your video or update VIDEO_PATH\")\n", "else:\n", " print(f\"Processing video: {VIDEO_PATH}\")\n", " print(f\"This may take several minutes...\")\n", " print(f\"\")\n", " \n", " # Set multiprocessing start method\n", " try:\n", " multiprocessing.set_start_method(\"spawn\")\n", " except:\n", " pass\n", " \n", " # Create SubtitleRemover instance\n", " sr = SubtitleRemover(VIDEO_PATH, sub_area=SUBTITLE_AREA, gui_mode=False)\n", " \n", " # Run processing\n", " sr.run()\n", " \n", " # Output location\n", " output_path = sr.video_out_name\n", " print(f\"\\n\u2713 Processing complete!\")\n", " print(f\"Output saved to: {output_path}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 8: Download Result\n", "\n", "Download the processed video to your computer." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "from google.colab import files\n", "import os\n", "\n", "# Get output filename\n", "video_name = os.path.basename(VIDEO_PATH)\n", "video_name_no_ext = os.path.splitext(video_name)[0]\n", "output_file = f\"{video_name_no_ext}_no_sub.mp4\"\n", "output_path = os.path.join(os.path.dirname(VIDEO_PATH), output_file)\n", "\n", "if os.path.exists(output_path):\n", " print(f\"Downloading: {output_file}\")\n", " files.download(output_path)\n", "else:\n", " print(f\"\u274c Output file not found: {output_path}\")\n", " print(\"Check if processing completed successfully.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 9: Save to Google Drive (Optional)\n", "\n", "Save the output video to Google Drive instead of downloading." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "import shutil\n", "import os\n", "\n", "# Destination in Google Drive\n", "drive_output_path = '/content/drive/MyDrive/video_subtitle_remover_output/'\n", "\n", "# Create directory if it doesn't exist\n", "os.makedirs(drive_output_path, exist_ok=True)\n", "\n", "# Copy output file\n", "if os.path.exists(output_path):\n", " dest_file = os.path.join(drive_output_path, output_file)\n", " shutil.copy(output_path, dest_file)\n", " print(f\"\u2713 Saved to Google Drive: {dest_file}\")\n", "else:\n", " print(f\"\u274c Output file not found\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Troubleshooting\n", "\n", "### Out of Memory (OOM)\n", "```python\n", "# Reduce batch size\n", "STTN_MAX_LOAD_NUM = 30\n", "# Or use LAMA\n", "ALGORITHM = 'LAMA'\n", "```\n", "\n", "### Processing too slow\n", "```python\n", "# Enable skip detection\n", "STTN_SKIP_DETECTION = True\n", "# Or use super fast mode\n", "ALGORITHM = 'LAMA'\n", "LAMA_SUPER_FAST = True\n", "```\n", "\n", "### Subtitles not removed\n", "```python\n", "# Set manual subtitle area (bottom 20% of 1080p)\n", "SUBTITLE_AREA = (864, 1080, 0, 1920)\n", "```\n", "\n", "### GPU not detected\n", "1. Runtime \u2192 Change runtime type \u2192 Hardware accelerator \u2192 GPU\n", "2. Restart runtime\n", "\n", "### Session timeout\n", "- Colab free tier has time limits\n", "- Process smaller videos (<10 min)\n", "- Or use Colab Pro for longer sessions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Advanced: Batch Processing\n", "\n", "Process multiple videos from Google Drive:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "import glob\n", "import os\n", "from backend.main import SubtitleRemover\n", "\n", "# Folder containing videos\n", "video_folder = '/content/dataku/test/'\n", "\n", "# Find all MP4 files\n", "video_files = glob.glob(os.path.join(video_folder, '*.mp4'))\n", "\n", "print(f\"Found {len(video_files)} videos to process\")\n", "\n", "for i, video_path in enumerate(video_files, 1):\n", " print(f\"\\n[{i}/{len(video_files)}] Processing: {os.path.basename(video_path)}\")\n", " \n", " try:\n", " sr = SubtitleRemover(video_path, sub_area=SUBTITLE_AREA, gui_mode=False)\n", " sr.run()\n", " print(f\"\u2713 Complete: {sr.video_out_name}\")\n", " except Exception as e:\n", " print(f\"\u274c Error: {e}\")\n", " continue\n", "\n", "print(f\"\\n\u2713 Batch processing complete!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tips for Best Results\n", "\n", "1. **Use STTN with skip detection** - Fastest for Colab\n", "2. **Keep videos under 10 minutes** - Avoid session timeout\n", "3. **Set manual subtitle area** - For better accuracy\n", "4. **Monitor GPU memory** - Use `!nvidia-smi` in separate cell\n", "5. **Save to Drive frequently** - Avoid data loss on timeout\n", "\n", "## Performance Expectations (Colab T4 GPU)\n", "\n", "| Video Length | Algorithm | Time |\n", "|--------------|-----------|------|\n", "| 1 min 720p | STTN (skip) | ~30s |\n", "| 5 min 720p | STTN (skip) | ~2min |\n", "| 1 min 720p | LAMA | ~3min |\n", "| 5 min 720p | LAMA | ~15min |\n", "\n", "ProPainter is not recommended for Colab due to memory limitations." ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }