Abdo96 commited on
Commit
86e3fda
·
verified ·
1 Parent(s): 6eae290

Upload 19 files

Browse files
Files changed (19) hide show
  1. .gitignore +8 -0
  2. DEPLOY_HF.md +112 -0
  3. README.md +110 -8
  4. activate.sh +4 -0
  5. app.py +540 -0
  6. cli.py +111 -0
  7. config.py +109 -0
  8. docs/README_full.md +212 -0
  9. models/__init__.py +1 -0
  10. models/model_manager.py +419 -0
  11. packages.txt +3 -0
  12. pipeline.py +386 -0
  13. requirements.txt +33 -0
  14. run.sh +4 -0
  15. run_cli.sh +4 -0
  16. setup.sh +259 -0
  17. utils/__init__.py +1 -0
  18. utils/blur_engine.py +173 -0
  19. utils/video_processor.py +281 -0
.gitignore ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ venv/
2
+ __pycache__/
3
+ *.pyc
4
+ *.pyo
5
+ .video_blur_ai_temp/
6
+ blur_output_*/
7
+ .DS_Store
8
+ .ipynb_checkpoints/
DEPLOY_HF.md ADDED
@@ -0,0 +1,112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Deploying to Hugging Face Spaces (ZeroGPU)
2
+ # النشر على Hugging Face Spaces باستخدام ZeroGPU
3
+
4
+ ---
5
+
6
+ ## English
7
+
8
+ ### 1. Create the Space
9
+ - Go to <https://huggingface.co/new-space>.
10
+ - **SDK:** Gradio. **Hardware:** select **ZeroGPU** (Nvidia A100, on-demand).
11
+ - Give it a name and create it.
12
+
13
+ > ZeroGPU is free for PRO users and for Spaces owned by orgs with the right plan.
14
+ > If ZeroGPU isn't offered on your account, a paid GPU (e.g. T4/A10G) also works —
15
+ > the code auto-detects a normal CUDA GPU too.
16
+
17
+ ### 2. Add the files
18
+ Upload every file in this folder to the Space repo (drag-and-drop in the
19
+ **Files** tab, or use git):
20
+
21
+ ```bash
22
+ git clone https://huggingface.co/spaces/<user>/<space-name>
23
+ cd <space-name>
24
+ cp -r /path/to/video-blur-ai/* .
25
+ git add .
26
+ git commit -m "Video Blur AI — image + video, ZeroGPU"
27
+ git push
28
+ ```
29
+
30
+ Make sure these are present at the repo root:
31
+ `app.py`, `pipeline.py`, `config.py`, `requirements.txt`, `packages.txt`,
32
+ `README.md`, and the `models/` and `utils/` folders.
33
+
34
+ ### 3. Let it build
35
+ The Space installs `packages.txt` (ffmpeg + libs) then `requirements.txt`
36
+ (this compiles SAM 2 — a few minutes the first time). When it says
37
+ **Running**, open the app.
38
+
39
+ ### 4. First run
40
+ The **first** image/video is slow: Grounding DINO + SAM 2 download and load
41
+ onto the GPU. Subsequent runs are fast. This is normal for ZeroGPU.
42
+
43
+ ### 5. Optional: Variables (Settings → Variables and secrets)
44
+ | Variable | Default | Meaning |
45
+ | -------------------- | ------- | ----------------------------------------- |
46
+ | `MAX_VIDEO_SECONDS` | `30` | Longest video accepted (GPU time budget) |
47
+ | `MAX_HEIGHT` | `720` | Frames downscaled to this height |
48
+ | `GPU_DURATION_VIDEO` | `120` | Max seconds a video job may hold the GPU |
49
+ | `GPU_DURATION_IMAGE` | `60` | Max seconds an image job may hold the GPU |
50
+ | `SAM2_BUILD_CUDA` | — | Set to `0` only if the SAM 2 build fails |
51
+
52
+ ### Troubleshooting
53
+ - **`libGL.so.1` / cv2 import error** → confirm `packages.txt` has `libgl1`.
54
+ - **`ffmpeg not found`** → confirm `packages.txt` has `ffmpeg`.
55
+ - **Job killed mid-video** → the video exceeded the GPU time budget; shorten it
56
+ or raise `GPU_DURATION_VIDEO` / lower `MAX_HEIGHT`.
57
+ - **SAM 2 build fails** → add variable `SAM2_BUILD_CUDA=0` and restart the build.
58
+ - **Everything runs on CPU** → hardware isn't ZeroGPU/GPU; check Space settings.
59
+
60
+ ---
61
+
62
+ ## العربية
63
+
64
+ ### 1) إنشاء الـ Space
65
+ - افتح <https://huggingface.co/new-space>.
66
+ - **SDK:** اختر Gradio. **Hardware:** اختر **ZeroGPU** (كرت A100 عند الطلب).
67
+ - سمّي المشروع وأنشئه.
68
+
69
+ > ZeroGPU متاح مجانًا لمشتركي PRO وللمنظمات ذات الخطة المناسبة. لو مش ظاهر عندك،
70
+ > أي كرت مدفوع (زي T4/A10G) هيشتغل برضه — الكود بيكتشف كرت CUDA العادي تلقائيًا.
71
+
72
+ ### 2) رفع الملفات
73
+ ارفع كل الملفات اللي في الفولدر ده إلى مستودع الـ Space (بالسحب والإفلات من تبويب
74
+ **Files**، أو باستخدام git):
75
+
76
+ ```bash
77
+ git clone https://huggingface.co/spaces/<user>/<space-name>
78
+ cd <space-name>
79
+ cp -r /path/to/video-blur-ai/* .
80
+ git add .
81
+ git commit -m "Video Blur AI — صورة + فيديو، ZeroGPU"
82
+ git push
83
+ ```
84
+
85
+ اتأكد إن الملفات دي موجودة في جذر المستودع:
86
+ `app.py` و`pipeline.py` و`config.py` و`requirements.txt` و`packages.txt`
87
+ و`README.md` وفولدر `models/` وفولدر `utils/`.
88
+
89
+ ### 3) البناء (Build)
90
+ الـ Space هيثبّت `packages.txt` (ffmpeg والمكتبات) وبعدها `requirements.txt`
91
+ (ده بيبني SAM 2 وبياخد كام دقيقة أول مرة). لما تلاقي الحالة **Running** افتح التطبيق.
92
+
93
+ ### 4) أول تشغيل
94
+ أول صورة أو فيديو بيكون بطيء لأن النماذج (Grounding DINO + SAM 2) بتتنزّل وتتحمّل
95
+ على الكرت. التشغيلات اللي بعده بتبقى سريعة — وده طبيعي مع ZeroGPU.
96
+
97
+ ### 5) متغيّرات اختيارية (Settings → Variables and secrets)
98
+ | المتغيّر | الافتراضي | المعنى |
99
+ | -------------------- | --------- | --------------------------------------------- |
100
+ | `MAX_VIDEO_SECONDS` | `30` | أطول فيديو مسموح به (ميزانية وقت الكرت) |
101
+ | `MAX_HEIGHT` | `720` | تصغير الإطارات لهذا الارتفاع |
102
+ | `GPU_DURATION_VIDEO` | `120` | أقصى ثوانٍ يمسك فيها الفيديو الكرت |
103
+ | `GPU_DURATION_IMAGE` | `60` | أقصى ثوانٍ تمسك فيها الصورة الكرت |
104
+ | `SAM2_BUILD_CUDA` | — | حطّها `0` فقط لو فشل بناء SAM 2 |
105
+
106
+ ### حل المشاكل
107
+ - **خطأ `libGL.so.1` أو استيراد cv2** → اتأكد إن `packages.txt` فيها `libgl1`.
108
+ - **`ffmpeg not found`** → اتأكد إن `packages.txt` فيها `ffmpeg`.
109
+ - **الفيديو بيتوقف في النص** → الفيديو تعدّى ميزانية وقت الكرت؛ قصّره أو زوّد
110
+ `GPU_DURATION_VIDEO` أو قلّل `MAX_HEIGHT`.
111
+ - **فشل بناء SAM 2** → ضيف متغيّر `SAM2_BUILD_CUDA=0` وأعد البناء.
112
+ - **كله بيشتغل على CPU** → الهاردوير مش ZeroGPU/GPU؛ راجع إعدادات الـ Space.
README.md CHANGED
@@ -1,15 +1,117 @@
1
  ---
2
- title: Easy Blury
3
- emoji: 🔥
4
- colorFrom: indigo
5
- colorTo: pink
6
  sdk: gradio
7
- sdk_version: 6.20.0
8
- python_version: '3.12'
9
  app_file: app.py
10
  pinned: false
11
  license: apache-2.0
12
- short_description: it can blur any image or videos by prompting the model
13
  ---
14
 
15
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Video Blur AI
3
+ emoji: 🎬
4
+ colorFrom: purple
5
+ colorTo: blue
6
  sdk: gradio
7
+ sdk_version: 4.44.1
 
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
+ short_description: Blur anything in images & video by text prompt (ZeroGPU)
12
  ---
13
 
14
+ # 🎬 Video Blur AI
15
+
16
+ **Text-prompted image & video object blurring.** Type what you want to blur —
17
+ `face`, `hand`, `license plate`, `text`, `logo` — and the AI segments and blurs it.
18
+
19
+ Built with **Grounding DINO** (text → boxes) + **SAM 2** (boxes → masks + tracking),
20
+ and adapted to run on **Hugging Face Spaces with ZeroGPU**.
21
+
22
+ ---
23
+
24
+ ## ✨ Features
25
+
26
+ - **Image *and* video** input — upload either.
27
+ - **Text-Prompted Blur** — describe what to blur in plain words.
28
+ - **Precise Segmentation** — pixel-level masks via SAM 2, not just boxes.
29
+ - **Video Tracking** — SAM 2 tracks objects across frames automatically.
30
+ - **Blur Types** — Gaussian blur, pixelation (mosaic), or solid black.
31
+ - **Audio Preserved** — the original audio track is kept for videos.
32
+ - **Detection Preview** — test your prompt on a frame before processing.
33
+
34
+ ## 🚀 Using it
35
+
36
+ 1. Open the **🖼️ Image** or **🎬 Video** tab and upload a file.
37
+ 2. Type what to blur, e.g. `face. license plate. text.`
38
+ 3. (Video) hit **Preview Detection** to sanity-check, then **Process**.
39
+ 4. Download the result.
40
+
41
+ ### Prompt cheatsheet
42
+
43
+ | Want to blur | Prompt |
44
+ | -------------- | ----------------------------- |
45
+ | Faces | `face.` |
46
+ | Hands | `hand.` |
47
+ | License plates | `license plate.` |
48
+ | Text / signs | `text. writing.` |
49
+ | People | `person.` |
50
+ | Logos / brands | `logo. brand.` |
51
+ | Multiple | `face. license plate. text.` |
52
+
53
+ ### Settings
54
+
55
+ - **Blur Strength** (11–151): higher = more blur. 51 is a good default.
56
+ - **Edge Feathering** (3–51): smooths blur edges. 11 looks natural.
57
+ - **Detection Threshold** (0.1–0.9): lower = more detections. 0.3 is balanced.
58
+ - **Model Quality**: *Tiny* is faster, *Base* is more accurate.
59
+
60
+ ---
61
+
62
+ ## ⚡ ZeroGPU notes
63
+
64
+ On ZeroGPU the GPU is attached only while a job runs. So:
65
+
66
+ - The **first run is slower** (models download + load); later runs are fast.
67
+ - Demo limits keep each job inside the GPU time budget:
68
+ **video ≤ 30 s**, processed at **≤ 720p**.
69
+ - Raise `MAX_VIDEO_SECONDS`, `MAX_HEIGHT`, `GPU_DURATION_VIDEO` in the Space
70
+ **Variables** if you have a longer budget or paid hardware.
71
+
72
+ See **DEPLOY_HF.md** for full deployment steps (English + العربية).
73
+
74
+ ---
75
+
76
+ ## 🖥️ Run locally
77
+
78
+ ```bash
79
+ # 1. install torch for your CUDA (or CPU)
80
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
81
+ # 2. install the rest
82
+ pip install -r requirements.txt
83
+ # 3. run
84
+ python app.py # http://localhost:7860
85
+ ```
86
+
87
+ The same `app.py` runs locally (the `spaces` decorator becomes a no-op off-Spaces).
88
+
89
+ You can also process from the command line:
90
+
91
+ ```bash
92
+ python cli.py input.mp4 --prompt "face. license plate." --blur-type pixelate
93
+ ```
94
+
95
+ ---
96
+
97
+ ## 📁 Structure
98
+
99
+ ```
100
+ video-blur-ai/
101
+ ├── app.py # Gradio app (Image + Video tabs, @spaces.GPU)
102
+ ├── pipeline.py # Orchestrates detect → segment/track → blur → assemble
103
+ ├── config.py # Settings + ZeroGPU-aware device detection
104
+ ├── cli.py # Command-line interface (video)
105
+ ├── models/model_manager.py # Grounding DINO + SAM 2 loading & inference
106
+ ├── utils/video_processor.py # FFmpeg-based video I/O
107
+ ├── utils/blur_engine.py # Blur application & mask compositing
108
+ ├── requirements.txt # Python deps (torch provided by ZeroGPU)
109
+ ├── packages.txt # apt deps (ffmpeg, libgl1, libglib2.0-0)
110
+ └── DEPLOY_HF.md # Deployment guide (EN + AR)
111
+ ```
112
+
113
+ ## 📝 License & Credits
114
+
115
+ Apache 2.0. Built on [SAM 2](https://github.com/facebookresearch/sam2) (Meta AI),
116
+ [Grounding DINO](https://github.com/IDEA-Research/GroundingDINO) (IDEA Research),
117
+ and [Gradio](https://gradio.app/) (Hugging Face).
activate.sh ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ #!/bin/bash
2
+ source "/home/abdo96/.venvs/video-blur-ai/bin/activate"
3
+ cd "/media/abdo96/ABODSALOMHA/LLMs_apps_github/video-blur-ai"
4
+ echo "✅ Activated. venv: /home/abdo96/.venvs/video-blur-ai"
app.py ADDED
@@ -0,0 +1,540 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Video Blur AI — Hugging Face Spaces (ZeroGPU) edition
3
+ Text-prompted image & video object blurring using Grounded SAM 2.
4
+
5
+ Type what you want to blur ("face. license plate. text.") and the AI
6
+ segments and blurs it — on an uploaded image or video.
7
+
8
+ Run locally:
9
+ python app.py # CPU or local CUDA
10
+ On Hugging Face Spaces:
11
+ Set hardware to "ZeroGPU"; this file is the app entrypoint.
12
+ """
13
+ import os
14
+
15
+ # ── Import `spaces` BEFORE torch so ZeroGPU can hook CUDA correctly. ──
16
+ # When running locally (no `spaces` package) we fall back to a no-op shim
17
+ # so the exact same file runs on CPU / local GPU unchanged.
18
+ try:
19
+ import spaces # type: ignore
20
+ _HAS_SPACES = True
21
+ except Exception: # pragma: no cover - only hit off-Spaces
22
+ _HAS_SPACES = False
23
+
24
+ class _SpacesShim:
25
+ @staticmethod
26
+ def GPU(*dargs, **dkwargs):
27
+ # Supports both @spaces.GPU and @spaces.GPU(duration=...)
28
+ if len(dargs) == 1 and callable(dargs[0]) and not dkwargs:
29
+ return dargs[0]
30
+
31
+ def _decorator(fn):
32
+ return fn
33
+
34
+ return _decorator
35
+
36
+ spaces = _SpacesShim() # type: ignore
37
+
38
+ import tempfile
39
+ import gradio as gr
40
+ import numpy as np
41
+ from pathlib import Path
42
+
43
+ from config import get_config
44
+ from pipeline import VideoBlurPipeline
45
+
46
+ # ─── Spaces / ZeroGPU tunables (override via Space "Variables") ─────
47
+ ON_ZERO_GPU = os.environ.get("SPACES_ZERO_GPU", "").lower() in ("1", "true", "yes", "on")
48
+ MAX_VIDEO_SECONDS = int(os.environ.get("MAX_VIDEO_SECONDS", "30"))
49
+ MAX_HEIGHT = int(os.environ.get("MAX_HEIGHT", "720"))
50
+ GPU_DURATION_VIDEO = int(os.environ.get("GPU_DURATION_VIDEO", "120"))
51
+ GPU_DURATION_IMAGE = int(os.environ.get("GPU_DURATION_IMAGE", "60"))
52
+
53
+ # ─── Global State ──────────────────────────────────────────────
54
+ config = get_config()
55
+ config.video.max_resolution = MAX_HEIGHT
56
+ pipeline = VideoBlurPipeline(config)
57
+
58
+
59
+ # ─── Custom CSS ────────────────────────────────────────────────
60
+ CUSTOM_CSS = """
61
+ .gradio-container {
62
+ max-width: 1200px !important;
63
+ margin: auto !important;
64
+ font-family: 'Segoe UI', system-ui, -apple-system, sans-serif !important;
65
+ }
66
+ .app-header {
67
+ text-align: center;
68
+ padding: 20px 0;
69
+ margin-bottom: 10px;
70
+ background: linear-gradient(135deg, #0f0c29, #302b63, #24243e);
71
+ border-radius: 16px;
72
+ color: white;
73
+ }
74
+ .app-header h1 {
75
+ font-size: 2.2em !important;
76
+ font-weight: 700 !important;
77
+ margin: 0 !important;
78
+ background: linear-gradient(90deg, #00d2ff, #7b2ff7, #ff6b6b);
79
+ -webkit-background-clip: text;
80
+ -webkit-text-fill-color: transparent;
81
+ background-clip: text;
82
+ }
83
+ .app-header p { color: #a0a0c0; font-size: 1.05em; margin: 8px 0 0 0; }
84
+ .prompt-input textarea {
85
+ font-size: 1.15em !important;
86
+ border: 2px solid #7b2ff7 !important;
87
+ border-radius: 12px !important;
88
+ padding: 14px !important;
89
+ transition: all 0.3s ease !important;
90
+ }
91
+ .prompt-input textarea:focus {
92
+ border-color: #00d2ff !important;
93
+ box-shadow: 0 0 0 3px rgba(0, 210, 255, 0.15) !important;
94
+ }
95
+ .process-btn {
96
+ background: linear-gradient(135deg, #7b2ff7, #00d2ff) !important;
97
+ border: none !important;
98
+ font-size: 1.15em !important;
99
+ font-weight: 600 !important;
100
+ padding: 14px 32px !important;
101
+ border-radius: 12px !important;
102
+ transition: all 0.3s ease !important;
103
+ }
104
+ .process-btn:hover {
105
+ transform: translateY(-2px) !important;
106
+ box-shadow: 0 6px 20px rgba(123, 47, 247, 0.35) !important;
107
+ }
108
+ .preview-btn {
109
+ background: linear-gradient(135deg, #ff6b6b, #ffa07a) !important;
110
+ border: none !important;
111
+ border-radius: 12px !important;
112
+ font-weight: 600 !important;
113
+ }
114
+ .status-text { font-size: 1.05em; padding: 10px; border-radius: 8px; text-align: center; }
115
+ """
116
+
117
+
118
+ EXAMPLE_PROMPTS = [
119
+ ["face."],
120
+ ["face. hand."],
121
+ ["license plate."],
122
+ ["text. writing."],
123
+ ["person."],
124
+ ["logo. brand."],
125
+ ["phone. screen."],
126
+ ["face. license plate. text."],
127
+ ]
128
+
129
+
130
+ # ─── Helpers ───────────────────────────────────────────────────
131
+
132
+ def _apply_model_quality(model_quality: str):
133
+ """Switch the Grounding DINO checkpoint based on the quality toggle."""
134
+ if model_quality == "base":
135
+ config.model.gdino_model_id = "IDEA-Research/grounding-dino-base"
136
+ else:
137
+ config.model.gdino_model_id = "IDEA-Research/grounding-dino-tiny"
138
+
139
+
140
+ def _resolve_path(file_obj):
141
+ """Gradio can hand back a path string or a file-like object."""
142
+ if isinstance(file_obj, str):
143
+ return file_obj
144
+ return file_obj.name if hasattr(file_obj, "name") else str(file_obj)
145
+
146
+
147
+ # ─── Processing Functions (GPU-decorated for ZeroGPU) ──────────
148
+
149
+ @spaces.GPU(duration=GPU_DURATION_IMAGE)
150
+ def process_image(
151
+ image,
152
+ text_prompt,
153
+ blur_type,
154
+ blur_strength,
155
+ edge_feather,
156
+ detection_threshold,
157
+ model_quality,
158
+ progress=gr.Progress(),
159
+ ):
160
+ """Blur objects in a single uploaded image."""
161
+ if image is None:
162
+ gr.Warning("⚠️ Please upload an image first!")
163
+ return None, "❌ No image uploaded"
164
+
165
+ if not text_prompt or not text_prompt.strip():
166
+ gr.Warning("⚠️ Please enter a text prompt describing what to blur!")
167
+ return None, "❌ No prompt entered"
168
+
169
+ text_prompt = text_prompt.strip()
170
+ _apply_model_quality(model_quality)
171
+
172
+ def gradio_progress(value, text):
173
+ progress(value, desc=text)
174
+
175
+ try:
176
+ # gr.Image(type="numpy") gives RGB; the pipeline works in BGR.
177
+ image_bgr = np.ascontiguousarray(image[..., ::-1])
178
+
179
+ result_bgr = pipeline.process_image(
180
+ image_bgr=image_bgr,
181
+ text_prompt=text_prompt,
182
+ blur_type=blur_type,
183
+ blur_strength=int(blur_strength),
184
+ edge_feather=int(edge_feather),
185
+ detection_threshold=float(detection_threshold),
186
+ progress_callback=gradio_progress,
187
+ )
188
+
189
+ result_rgb = np.ascontiguousarray(result_bgr[..., ::-1])
190
+ return result_rgb, "✅ Done!"
191
+
192
+ except Exception as e:
193
+ error_msg = f"❌ Error: {str(e)}"
194
+ gr.Warning(error_msg)
195
+ return None, error_msg
196
+
197
+
198
+ @spaces.GPU(duration=GPU_DURATION_VIDEO)
199
+ def process_video(
200
+ video_file,
201
+ text_prompt,
202
+ blur_type,
203
+ blur_strength,
204
+ edge_feather,
205
+ processing_mode,
206
+ keyframe_interval,
207
+ detection_threshold,
208
+ model_quality,
209
+ progress=gr.Progress(),
210
+ ):
211
+ """Blur objects across an uploaded video."""
212
+ if video_file is None:
213
+ gr.Warning("⚠️ Please upload a video first!")
214
+ return None, "❌ No video uploaded"
215
+
216
+ if not text_prompt or not text_prompt.strip():
217
+ gr.Warning("⚠️ Please enter a text prompt describing what to blur!")
218
+ return None, "❌ No prompt entered"
219
+
220
+ text_prompt = text_prompt.strip()
221
+ _apply_model_quality(model_quality)
222
+
223
+ def gradio_progress(value, text):
224
+ progress(value, desc=text)
225
+
226
+ try:
227
+ video_path = _resolve_path(video_file)
228
+
229
+ # Keep each request within the ZeroGPU time budget.
230
+ try:
231
+ info = pipeline.video_processor.get_video_info(video_path)
232
+ if info.duration > MAX_VIDEO_SECONDS:
233
+ msg = (
234
+ f"❌ Video is {info.duration:.0f}s long. On this demo the limit "
235
+ f"is {MAX_VIDEO_SECONDS}s so it fits the ZeroGPU time budget. "
236
+ f"Please trim it, or raise MAX_VIDEO_SECONDS / GPU_DURATION_VIDEO "
237
+ f"in the Space variables."
238
+ )
239
+ gr.Warning(msg)
240
+ return None, msg
241
+ except Exception:
242
+ # If probing fails, let the pipeline surface the real error below.
243
+ pass
244
+
245
+ output_dir = tempfile.mkdtemp(prefix="blur_output_")
246
+ output_name = f"blurred_{Path(video_path).stem}.mp4"
247
+ output_path = os.path.join(output_dir, output_name)
248
+
249
+ result_path = pipeline.process_video(
250
+ video_path=video_path,
251
+ text_prompt=text_prompt,
252
+ output_path=output_path,
253
+ blur_type=blur_type,
254
+ blur_strength=int(blur_strength),
255
+ edge_feather=int(edge_feather),
256
+ processing_mode=processing_mode,
257
+ keyframe_interval=int(keyframe_interval),
258
+ detection_threshold=float(detection_threshold),
259
+ progress_callback=gradio_progress,
260
+ )
261
+
262
+ return result_path, "✅ Done! Video saved successfully."
263
+
264
+ except Exception as e:
265
+ error_msg = f"❌ Error: {str(e)}"
266
+ gr.Warning(error_msg)
267
+ return None, error_msg
268
+
269
+
270
+ @spaces.GPU(duration=GPU_DURATION_IMAGE)
271
+ def preview_detection(video_file, text_prompt, frame_slider):
272
+ """Preview detection on a single video frame."""
273
+ if video_file is None:
274
+ gr.Warning("⚠️ Please upload a video first!")
275
+ return None
276
+
277
+ if not text_prompt or not text_prompt.strip():
278
+ gr.Warning("⚠️ Please enter a prompt!")
279
+ return None
280
+
281
+ try:
282
+ video_path = _resolve_path(video_file)
283
+ vis = pipeline.preview_detection(
284
+ video_path,
285
+ text_prompt.strip(),
286
+ frame_number=int(frame_slider),
287
+ )
288
+ return np.ascontiguousarray(vis[..., ::-1]) # BGR -> RGB
289
+ except Exception as e:
290
+ gr.Warning(f"Preview error: {str(e)}")
291
+ return None
292
+
293
+
294
+ def get_video_info_text(video_file):
295
+ """Show basic metadata when a video is uploaded."""
296
+ if video_file is None:
297
+ return "No video uploaded", gr.Slider(maximum=0, value=0)
298
+
299
+ try:
300
+ video_path = _resolve_path(video_file)
301
+ info = pipeline.video_processor.get_video_info(video_path)
302
+
303
+ warn = ""
304
+ if info.duration > MAX_VIDEO_SECONDS:
305
+ warn = f" ⚠️ Longer than the {MAX_VIDEO_SECONDS}s demo limit — please trim."
306
+
307
+ text = (
308
+ f"📹 **{Path(video_path).name}**\n"
309
+ f"Resolution: {info.width}×{info.height} | "
310
+ f"FPS: {info.fps:.1f} | "
311
+ f"Frames: {info.total_frames} | "
312
+ f"Duration: {info.duration:.1f}s | "
313
+ f"Audio: {'Yes ✅' if info.has_audio else 'No ❌'}{warn}"
314
+ )
315
+ max_frame = max(0, info.total_frames - 1)
316
+ return text, gr.Slider(maximum=max_frame, value=0)
317
+
318
+ except Exception as e:
319
+ return f"❌ Error reading video: {e}", gr.Slider(maximum=0, value=0)
320
+
321
+
322
+ # ─── UI builders ───────────────────────────────────────────────
323
+
324
+ def _build_common_settings(include_video_controls: bool):
325
+ """Create the shared advanced-settings controls.
326
+
327
+ Returns a dict of components. New instances are created per call
328
+ because a Gradio component can't be reused across two layouts.
329
+ """
330
+ c = {}
331
+ c["model_quality"] = gr.Radio(
332
+ choices=[
333
+ ("⚡ Fast (Tiny model ~350MB, good accuracy)", "tiny"),
334
+ ("🎯 Accurate (Base model ~999MB, best accuracy)", "base"),
335
+ ],
336
+ value="base",
337
+ label="Model Quality",
338
+ info="Tiny downloads faster and runs faster. Base is more accurate.",
339
+ )
340
+ c["blur_type"] = gr.Radio(
341
+ choices=["gaussian", "pixelate", "black"],
342
+ value="gaussian",
343
+ label="Blur Type",
344
+ info="Gaussian = smooth blur, Pixelate = mosaic, Black = solid cover",
345
+ )
346
+ c["blur_strength"] = gr.Slider(
347
+ minimum=11, maximum=151, value=51, step=2,
348
+ label="Blur Strength", info="Higher = more blur (odd number)",
349
+ )
350
+ c["edge_feather"] = gr.Slider(
351
+ minimum=3, maximum=51, value=11, step=2,
352
+ label="Edge Feathering", info="Smooth transition at blur edges",
353
+ )
354
+ if include_video_controls:
355
+ c["processing_mode"] = gr.Radio(
356
+ choices=[
357
+ ("🎯 SAM 2 Video Tracking (recommended)", "video_tracking"),
358
+ ("⚡ Frame-by-Frame (faster, less consistent)", "frame_by_frame"),
359
+ ],
360
+ value="video_tracking",
361
+ label="Processing Mode",
362
+ )
363
+ c["keyframe_interval"] = gr.Slider(
364
+ minimum=1, maximum=30, value=5, step=1,
365
+ label="Keyframe Interval (frame-by-frame mode)",
366
+ info="Re-detect every N frames",
367
+ )
368
+ c["detection_threshold"] = gr.Slider(
369
+ minimum=0.1, maximum=0.9, value=0.3, step=0.05,
370
+ label="Detection Confidence Threshold",
371
+ info="Lower = more detections, higher = more precise",
372
+ )
373
+ return c
374
+
375
+
376
+ def create_app():
377
+ """Build the Gradio interface."""
378
+ theme = gr.themes.Soft(primary_hue="violet", secondary_hue="cyan")
379
+
380
+ with gr.Blocks(title="Video Blur AI", theme=theme, css=CUSTOM_CSS) as app:
381
+
382
+ gr.HTML("""
383
+ <div class="app-header">
384
+ <h1>🎬 Video Blur AI</h1>
385
+ <p>Type what you want to blur — faces, hands, text, logos, anything.</p>
386
+ <p style="font-size: 0.85em; color: #7a7a9a;">
387
+ Image &amp; Video · Grounding DINO + SAM 2 · ZeroGPU
388
+ </p>
389
+ </div>
390
+ """)
391
+
392
+ with gr.Tabs():
393
+
394
+ # ══════════════ IMAGE TAB ══════════════
395
+ with gr.Tab("🖼️ Image"):
396
+ with gr.Row():
397
+ with gr.Column(scale=1):
398
+ img_input = gr.Image(label="📤 Upload Image", type="numpy")
399
+ img_prompt = gr.Textbox(
400
+ label="✏️ What to blur? (Text Prompt)",
401
+ placeholder="e.g., face, license plate, text, logo...",
402
+ info="Separate multiple objects with periods: 'face. text.'",
403
+ lines=2,
404
+ elem_classes=["prompt-input"],
405
+ )
406
+ gr.Examples(
407
+ examples=EXAMPLE_PROMPTS,
408
+ inputs=[img_prompt],
409
+ label="💡 Example Prompts (click to use)",
410
+ )
411
+ with gr.Accordion("⚙️ Advanced Settings", open=False):
412
+ img_s = _build_common_settings(include_video_controls=False)
413
+ img_process_btn = gr.Button(
414
+ "🚀 Blur Image", variant="primary", size="lg",
415
+ elem_classes=["process-btn"],
416
+ )
417
+ with gr.Column(scale=1):
418
+ img_output = gr.Image(label="Blurred Image", type="numpy")
419
+ img_status = gr.Markdown("Ready to process", elem_classes=["status-text"])
420
+
421
+ img_process_btn.click(
422
+ fn=process_image,
423
+ inputs=[
424
+ img_input, img_prompt, img_s["blur_type"], img_s["blur_strength"],
425
+ img_s["edge_feather"], img_s["detection_threshold"], img_s["model_quality"],
426
+ ],
427
+ outputs=[img_output, img_status],
428
+ )
429
+
430
+ # ══════════════ VIDEO TAB ══════════════
431
+ with gr.Tab("🎬 Video"):
432
+ with gr.Row():
433
+ with gr.Column(scale=1):
434
+ video_input = gr.Video(label="📤 Upload Video")
435
+ video_info_text = gr.Markdown("No video uploaded")
436
+ vid_prompt = gr.Textbox(
437
+ label="✏️ What to blur? (Text Prompt)",
438
+ placeholder="e.g., face, hand, license plate, person, text, logo...",
439
+ info="Separate multiple objects with periods: 'face. hand. text.'",
440
+ lines=2,
441
+ elem_classes=["prompt-input"],
442
+ )
443
+ gr.Examples(
444
+ examples=EXAMPLE_PROMPTS,
445
+ inputs=[vid_prompt],
446
+ label="💡 Example Prompts (click to use)",
447
+ )
448
+ with gr.Accordion("⚙️ Advanced Settings", open=False):
449
+ vid_s = _build_common_settings(include_video_controls=True)
450
+
451
+ with gr.Column(scale=1):
452
+ with gr.Tab("🔍 Preview Detection"):
453
+ preview_image = gr.Image(label="Detection Preview", type="numpy")
454
+ frame_slider = gr.Slider(
455
+ minimum=0, maximum=100, value=0, step=1, label="Frame Number",
456
+ )
457
+ preview_btn = gr.Button(
458
+ "🔍 Preview Detection", variant="secondary",
459
+ elem_classes=["preview-btn"],
460
+ )
461
+ with gr.Tab("🎬 Result"):
462
+ video_output = gr.Video(label="Blurred Video")
463
+ vid_status = gr.Markdown("Ready to process", elem_classes=["status-text"])
464
+
465
+ vid_process_btn = gr.Button(
466
+ "🚀 Process Video", variant="primary", size="lg",
467
+ elem_classes=["process-btn"],
468
+ )
469
+
470
+ video_input.change(
471
+ fn=get_video_info_text,
472
+ inputs=[video_input],
473
+ outputs=[video_info_text, frame_slider],
474
+ )
475
+ preview_btn.click(
476
+ fn=preview_detection,
477
+ inputs=[video_input, vid_prompt, frame_slider],
478
+ outputs=[preview_image],
479
+ )
480
+ vid_process_btn.click(
481
+ fn=process_video,
482
+ inputs=[
483
+ video_input, vid_prompt, vid_s["blur_type"], vid_s["blur_strength"],
484
+ vid_s["edge_feather"], vid_s["processing_mode"], vid_s["keyframe_interval"],
485
+ vid_s["detection_threshold"], vid_s["model_quality"],
486
+ ],
487
+ outputs=[video_output, vid_status],
488
+ )
489
+
490
+ # ── Usage Guide ──
491
+ with gr.Accordion("📖 How to Use", open=False):
492
+ gr.Markdown(f"""
493
+ ### Quick Start
494
+ 1. **Upload** an image or a video (image: PNG/JPG · video: MP4/AVI/MOV/MKV)
495
+ 2. **Type** what you want to blur in the text prompt
496
+ 3. (Video) **Preview Detection** to check the AI found the right objects
497
+ 4. **Click** the process button to apply the blur
498
+ 5. **Download** your result
499
+
500
+ ### Tips for Better Results
501
+ - Use **specific** descriptions: `face` works better than `person's head`
502
+ - Separate multiple objects with **periods**: `face. license plate. text.`
503
+ - Lower the **detection threshold** if objects aren't being found
504
+ - **SAM 2 Video Tracking** gives smooth, consistent blur across frames
505
+ - **Frame-by-Frame** is better when objects appear/disappear frequently
506
+
507
+ ### ZeroGPU Notes
508
+ - The GPU is attached only while a job runs, so the **first run is slower**
509
+ (models download + load). Later runs are fast.
510
+ - Demo limits: video ≤ **{MAX_VIDEO_SECONDS}s**, processed at ≤ **{MAX_HEIGHT}p**.
511
+ Raise `MAX_VIDEO_SECONDS`, `MAX_HEIGHT`, and `GPU_DURATION_VIDEO`
512
+ in the Space **Variables** if you have a longer GPU budget.
513
+ """)
514
+
515
+ return app
516
+
517
+
518
+ # ─── Entry Point ───────────────────────────────────────────────
519
+
520
+ if __name__ == "__main__":
521
+ print("\n" + "=" * 60)
522
+ print(" 🎬 Video Blur AI - Starting...")
523
+ print("=" * 60)
524
+
525
+ demo = create_app()
526
+ demo.queue() # serialize GPU access + enable progress
527
+
528
+ if ON_ZERO_GPU or os.environ.get("SPACE_ID"):
529
+ # On Spaces, let the platform manage host/port.
530
+ demo.launch()
531
+ else:
532
+ import argparse
533
+
534
+ parser = argparse.ArgumentParser(description="Video Blur AI")
535
+ parser.add_argument("--port", type=int, default=7860, help="Server port")
536
+ parser.add_argument("--share", action="store_true", help="Create public link")
537
+ parser.add_argument("--host", type=str, default="0.0.0.0", help="Server host")
538
+ args = parser.parse_args()
539
+
540
+ demo.launch(server_name=args.host, server_port=args.port, share=args.share)
cli.py ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ Video Blur AI - Command Line Interface
4
+
5
+ Usage:
6
+ python cli.py input.mp4 --prompt "face. license plate." --output blurred.mp4
7
+ python cli.py input.mp4 --prompt "hand." --blur-type pixelate --strength 15
8
+ """
9
+ import argparse
10
+ import sys
11
+ from pathlib import Path
12
+
13
+ from config import get_config
14
+ from pipeline import VideoBlurPipeline
15
+
16
+
17
+ def main():
18
+ parser = argparse.ArgumentParser(
19
+ description="🎬 Video Blur AI - Text-prompted video object blurring",
20
+ formatter_class=argparse.RawDescriptionHelpFormatter,
21
+ epilog="""
22
+ Examples:
23
+ %(prog)s video.mp4 --prompt "face."
24
+ %(prog)s video.mp4 --prompt "face. license plate." --blur-type pixelate
25
+ %(prog)s video.mp4 --prompt "hand." --strength 99 --mode frame_by_frame
26
+ %(prog)s video.mp4 --prompt "person." --blur-type black --output censored.mp4
27
+ """
28
+ )
29
+
30
+ parser.add_argument("video", help="Input video file path")
31
+ parser.add_argument("-p", "--prompt", required=True,
32
+ help="Text prompt describing what to blur (e.g., 'face. hand.')")
33
+ parser.add_argument("-o", "--output", default=None,
34
+ help="Output video path (default: input_blurred.mp4)")
35
+
36
+ # Blur settings
37
+ parser.add_argument("--blur-type", choices=["gaussian", "pixelate", "black"],
38
+ default="gaussian", help="Type of blur effect")
39
+ parser.add_argument("--strength", type=int, default=51,
40
+ help="Blur strength (odd number, default: 51)")
41
+ parser.add_argument("--feather", type=int, default=11,
42
+ help="Edge feathering (odd number, default: 11)")
43
+
44
+ # Processing settings
45
+ parser.add_argument("--mode", choices=["video_tracking", "frame_by_frame"],
46
+ default="video_tracking", help="Processing mode")
47
+ parser.add_argument("--keyframe-interval", type=int, default=5,
48
+ help="Keyframe detection interval (frame_by_frame mode)")
49
+ parser.add_argument("--threshold", type=float, default=0.3,
50
+ help="Detection confidence threshold (0.1-0.9)")
51
+
52
+ args = parser.parse_args()
53
+
54
+ # Validate input
55
+ if not Path(args.video).exists():
56
+ print(f"❌ Video not found: {args.video}")
57
+ sys.exit(1)
58
+
59
+ # Generate output path if not specified
60
+ if args.output is None:
61
+ p = Path(args.video)
62
+ args.output = str(p.parent / f"{p.stem}_blurred{p.suffix}")
63
+
64
+ # Run pipeline
65
+ print("\n" + "="*60)
66
+ print(" 🎬 Video Blur AI - CLI Mode")
67
+ print("="*60)
68
+ print(f" Input: {args.video}")
69
+ print(f" Prompt: {args.prompt}")
70
+ print(f" Blur: {args.blur_type} (strength={args.strength})")
71
+ print(f" Mode: {args.mode}")
72
+ print(f" Output: {args.output}")
73
+ print("="*60 + "\n")
74
+
75
+ config = get_config()
76
+ pipeline = VideoBlurPipeline(config)
77
+
78
+ def cli_progress(value, text):
79
+ bar_length = 30
80
+ filled = int(bar_length * value)
81
+ bar = "█" * filled + "░" * (bar_length - filled)
82
+ print(f"\r [{bar}] {value*100:5.1f}% | {text}", end="", flush=True)
83
+ if value >= 1.0:
84
+ print()
85
+
86
+ try:
87
+ output = pipeline.process_video(
88
+ video_path=args.video,
89
+ text_prompt=args.prompt,
90
+ output_path=args.output,
91
+ blur_type=args.blur_type,
92
+ blur_strength=args.strength,
93
+ edge_feather=args.feather,
94
+ processing_mode=args.mode,
95
+ keyframe_interval=args.keyframe_interval,
96
+ detection_threshold=args.threshold,
97
+ progress_callback=cli_progress,
98
+ )
99
+
100
+ print(f"\n✅ Output saved to: {output}")
101
+
102
+ except KeyboardInterrupt:
103
+ print("\n\n⚠️ Processing cancelled by user")
104
+ sys.exit(1)
105
+ except Exception as e:
106
+ print(f"\n\n❌ Error: {e}")
107
+ sys.exit(1)
108
+
109
+
110
+ if __name__ == "__main__":
111
+ main()
config.py ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Configuration settings for Video Blur AI
3
+ Adapted for Hugging Face Spaces (ZeroGPU) + local use.
4
+ """
5
+ import os
6
+ from dataclasses import dataclass, field
7
+ from typing import Optional
8
+
9
+
10
+ @dataclass
11
+ class ModelConfig:
12
+ """Model configuration"""
13
+ # SAM 2 settings
14
+ sam2_checkpoint: str = "facebook/sam2.1-hiera-base-plus"
15
+ sam2_model_cfg: str = "configs/sam2.1/sam2.1_hiera_b+.yaml"
16
+
17
+ # Grounding DINO settings
18
+ gdino_model_id: str = "IDEA-Research/grounding-dino-base"
19
+ gdino_box_threshold: float = 0.3
20
+ gdino_text_threshold: float = 0.25
21
+
22
+ # Device
23
+ device: str = "cuda"
24
+ use_fp16: bool = True
25
+
26
+
27
+ @dataclass
28
+ class VideoConfig:
29
+ """Video processing configuration"""
30
+ # Frame extraction
31
+ max_resolution: int = 1080 # Max height for processing
32
+ keyframe_interval: int = 5 # Run detection every N frames
33
+
34
+ # Output
35
+ output_codec: str = "libx264"
36
+ output_crf: int = 18 # Quality (lower = better, 18-23 recommended)
37
+ output_preset: str = "medium" # Encoding speed preset
38
+ pixel_format: str = "yuv420p"
39
+
40
+ # Temp directory
41
+ temp_dir: str = os.path.join(os.path.expanduser("~"), ".video_blur_ai_temp")
42
+
43
+
44
+ @dataclass
45
+ class BlurConfig:
46
+ """Blur settings"""
47
+ blur_strength: int = 51 # Gaussian kernel size (must be odd)
48
+ edge_feather: int = 11 # Edge softening kernel (must be odd)
49
+ blur_type: str = "gaussian" # gaussian, pixelate, black
50
+ pixelate_size: int = 15 # Block size for pixelation
51
+
52
+
53
+ @dataclass
54
+ class AppConfig:
55
+ """Main application config"""
56
+ model: ModelConfig = field(default_factory=ModelConfig)
57
+ video: VideoConfig = field(default_factory=VideoConfig)
58
+ blur: BlurConfig = field(default_factory=BlurConfig)
59
+
60
+ # App settings
61
+ max_video_duration: int = 600 # Max 10 minutes
62
+ max_file_size_mb: int = 500
63
+ server_port: int = 7860
64
+ share: bool = False
65
+
66
+
67
+ def _is_zero_gpu() -> bool:
68
+ """Detect a Hugging Face ZeroGPU Space.
69
+
70
+ On ZeroGPU the physical GPU is attached only inside functions wrapped
71
+ with @spaces.GPU, so torch.cuda.is_available() is typically False at
72
+ import time. We therefore rely on the env var HF sets instead.
73
+ """
74
+ return os.environ.get("SPACES_ZERO_GPU", "").lower() in ("1", "true", "yes", "on")
75
+
76
+
77
+ def get_config() -> AppConfig:
78
+ """Get application configuration with device auto-detection."""
79
+ config = AppConfig()
80
+
81
+ import torch
82
+
83
+ if _is_zero_gpu():
84
+ # Target CUDA: the GPU becomes available inside @spaces.GPU calls,
85
+ # even though it is not attached to this (import-time) process.
86
+ config.model.device = "cuda"
87
+ config.model.use_fp16 = True
88
+ print("✅ Hugging Face ZeroGPU detected — targeting CUDA inside @spaces.GPU")
89
+ elif torch.cuda.is_available():
90
+ config.model.device = "cuda"
91
+ config.model.use_fp16 = True
92
+ gpu_name = torch.cuda.get_device_name(0)
93
+ vram_gb = torch.cuda.get_device_properties(0).total_memory / (1024 ** 3)
94
+ print(f"✅ GPU: {gpu_name} ({vram_gb:.1f} GB VRAM)")
95
+
96
+ # Use smaller model for low VRAM
97
+ if vram_gb < 8:
98
+ config.model.sam2_checkpoint = "facebook/sam2.1-hiera-small"
99
+ config.model.sam2_model_cfg = "configs/sam2.1/sam2.1_hiera_s.yaml"
100
+ print("📉 Low VRAM detected, using SAM 2 Small model")
101
+ else:
102
+ config.model.device = "cpu"
103
+ config.model.use_fp16 = False
104
+ print("⚠️ CUDA not available, using CPU (will be slow)")
105
+
106
+ # Create temp directory
107
+ os.makedirs(config.video.temp_dir, exist_ok=True)
108
+
109
+ return config
docs/README_full.md ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🎬 Video Blur AI
2
+
3
+ **Text-prompted video object blurring** — type what you want to blur, and AI does the rest.
4
+
5
+ Built with **Grounding DINO + SAM 2** for precise, prompt-driven segmentation and tracking.
6
+
7
+ <p align="center">
8
+ <img src="https://img.shields.io/badge/Python-3.10+-blue?logo=python" alt="Python">
9
+ <img src="https://img.shields.io/badge/PyTorch-2.1+-red?logo=pytorch" alt="PyTorch">
10
+ <img src="https://img.shields.io/badge/License-Apache%202.0-green" alt="License">
11
+ <img src="https://img.shields.io/badge/GPU-NVIDIA%208GB+-76b900?logo=nvidia" alt="GPU">
12
+ </p>
13
+
14
+ ---
15
+
16
+ ## ✨ Features
17
+
18
+ - **Text-Prompted Blur**: Type "face", "hand", "license plate" — blur anything describable
19
+ - **Precise Segmentation**: Pixel-level masks via SAM 2, not just bounding boxes
20
+ - **Video Tracking**: SAM 2's memory mechanism tracks objects across frames automatically
21
+ - **Multiple Blur Types**: Gaussian blur, pixelation (mosaic), or solid black overlay
22
+ - **Audio Preservation**: Original audio track is preserved in the output
23
+ - **Web Interface**: Beautiful Gradio UI with live preview and progress tracking
24
+ - **Detection Preview**: Test your prompt on any frame before processing
25
+
26
+ ---
27
+
28
+ ## 🏗️ Architecture
29
+
30
+ ```
31
+ ┌─────────────────────────────────────────────────────────────┐
32
+ │ Video Blur AI │
33
+ ├─────────────┬──────────────┬──────────────┬────────────────┤
34
+ │ Gradio UI │ Grounding │ SAM 2.1 │ FFmpeg + │
35
+ │ (Web App) │ DINO │ (Segment & │ OpenCV │
36
+ │ │ (Text → │ Track) │ (Video I/O │
37
+ │ │ Boxes) │ │ & Blur) │
38
+ ├─────────────┴──────────────┴──────────────┴────────────────┤
39
+ │ Processing Pipeline │
40
+ │ │
41
+ │ Video → Extract Frames → Detect (keyframe) → Track → │
42
+ │ → Generate Masks → Apply Blur → Reassemble → Output │
43
+ └─────────────────────────────────────────────────────────────┘
44
+ ```
45
+
46
+ ### Processing Modes
47
+
48
+ | Mode | How it works | Best for |
49
+ |------|-------------|----------|
50
+ | **SAM 2 Video Tracking** | Detect on frame 1, track through entire video | Consistent blur, moving objects |
51
+ | **Frame-by-Frame** | Re-detect every N frames, reuse masks between | Objects appearing/disappearing |
52
+
53
+ ---
54
+
55
+ ## 🚀 Quick Start
56
+
57
+ ### Prerequisites
58
+ - Python 3.10+
59
+ - NVIDIA GPU with 8+ GB VRAM (12+ GB recommended)
60
+ - CUDA 11.8+ / CUDA 12.x
61
+ - FFmpeg
62
+
63
+ ### Automated Setup (Recommended)
64
+ ```bash
65
+ git clone <this-repo>
66
+ cd video-blur-ai
67
+ chmod +x setup.sh
68
+ ./setup.sh
69
+ ```
70
+
71
+ ### Manual Setup
72
+ ```bash
73
+ # 1. Create virtual environment
74
+ python3 -m venv venv
75
+ source venv/bin/activate
76
+
77
+ # 2. Install PyTorch (CUDA 12.1)
78
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
79
+
80
+ # 3. Install SAM 2
81
+ pip install git+https://github.com/facebookresearch/sam2.git
82
+
83
+ # 4. Install Grounding DINO
84
+ pip install git+https://github.com/IDEA-Research/GroundingDINO.git
85
+
86
+ # 5. Install other dependencies
87
+ pip install -r requirements.txt
88
+ ```
89
+
90
+ ### Run the App
91
+ ```bash
92
+ source venv/bin/activate
93
+ python app.py
94
+ # Open http://localhost:7860
95
+ ```
96
+
97
+ For a public shareable link:
98
+ ```bash
99
+ python app.py --share
100
+ ```
101
+
102
+ ---
103
+
104
+ ## 💡 Usage Tips
105
+
106
+ ### Effective Prompts
107
+
108
+ | Want to blur | Prompt | Notes |
109
+ |-------------|--------|-------|
110
+ | Faces | `face.` | Works great for front & side views |
111
+ | Hands | `hand.` | Both palms and fists |
112
+ | License plates | `license plate.` | Vehicles in motion too |
113
+ | Text/Writing | `text. writing.` | Signs, documents, screens |
114
+ | People (full body) | `person.` | Entire silhouette |
115
+ | Logos/Brands | `logo. brand.` | On clothing, products |
116
+ | Multiple things | `face. license plate. text.` | Separate with periods |
117
+ | Phone screens | `phone screen.` | Mobile devices |
118
+
119
+ ### Settings Guide
120
+
121
+ - **Blur Strength** (11-151): Higher = more blur. 51 is good default.
122
+ - **Edge Feathering** (3-51): Smooths blur edges. 11 for natural look.
123
+ - **Detection Threshold** (0.1-0.9): Lower = more detections. 0.3 is balanced.
124
+ - **Keyframe Interval** (1-30): Frame-by-frame mode only. 5 = detect every 5 frames.
125
+
126
+ ---
127
+
128
+ ## 📁 Project Structure
129
+
130
+ ```
131
+ video-blur-ai/
132
+ ├── app.py # Main Gradio web application
133
+ ├── pipeline.py # Processing pipeline orchestrator
134
+ ├── config.py # All configuration settings
135
+ ├── setup.sh # Automated setup script
136
+ ├── requirements.txt # Python dependencies
137
+ ├── models/
138
+ │ ├── __init__.py
139
+ │ └── model_manager.py # Grounding DINO + SAM 2 loading & inference
140
+ ├── utils/
141
+ │ ├── __init__.py
142
+ │ ├── video_processor.py # FFmpeg-based video I/O
143
+ │ └── blur_engine.py # Blur application & mask compositing
144
+ └── README.md
145
+ ```
146
+
147
+ ---
148
+
149
+ ## 🔧 Technical Details
150
+
151
+ ### Models Used
152
+
153
+ | Model | Purpose | Size | Source |
154
+ |-------|---------|------|--------|
155
+ | Grounding DINO (SwinB) | Text → Bounding Boxes | ~700 MB | [IDEA Research](https://github.com/IDEA-Research/GroundingDINO) |
156
+ | SAM 2.1 Hiera B+ | Boxes → Masks + Video Tracking | ~350 MB | [Meta](https://github.com/facebookresearch/sam2) |
157
+
158
+ ### Performance (approximate)
159
+
160
+ | GPU | Video Tracking Mode | Frame-by-Frame Mode |
161
+ |-----|-------------------|---------------------|
162
+ | RTX 4090 | ~8-15 FPS | ~5-10 FPS |
163
+ | RTX 3080 | ~5-10 FPS | ~3-7 FPS |
164
+ | RTX 3070 | ~3-7 FPS | ~2-5 FPS |
165
+ | CPU only | ~0.1-0.3 FPS | ~0.1-0.2 FPS |
166
+
167
+ ### Memory Usage
168
+
169
+ | Component | VRAM | RAM |
170
+ |-----------|------|-----|
171
+ | Grounding DINO | ~3-4 GB | ~2 GB |
172
+ | SAM 2.1 B+ | ~4-6 GB | ~2 GB |
173
+ | Video frames | ~1-3 GB | ~2-8 GB |
174
+ | **Total** | **~8-13 GB** | **~6-12 GB** |
175
+
176
+ ---
177
+
178
+ ## 🔄 API / CLI Usage
179
+
180
+ You can also use the pipeline programmatically:
181
+
182
+ ```python
183
+ from config import get_config
184
+ from pipeline import VideoBlurPipeline
185
+
186
+ config = get_config()
187
+ pipeline = VideoBlurPipeline(config)
188
+
189
+ # Process a video
190
+ output = pipeline.process_video(
191
+ video_path="input.mp4",
192
+ text_prompt="face. license plate.",
193
+ blur_type="gaussian",
194
+ blur_strength=51,
195
+ )
196
+ print(f"Output: {output}")
197
+ ```
198
+
199
+ ---
200
+
201
+ ## 📝 License
202
+
203
+ Apache 2.0 — Both Grounding DINO and SAM 2 use Apache 2.0 compatible licenses.
204
+
205
+ ---
206
+
207
+ ## 🙏 Credits
208
+
209
+ - [SAM 2](https://github.com/facebookresearch/sam2) by Meta AI
210
+ - [Grounding DINO](https://github.com/IDEA-Research/GroundingDINO) by IDEA Research
211
+ - [Grounded SAM 2](https://github.com/IDEA-Research/Grounded-SAM-2) by IDEA Research
212
+ - [Gradio](https://gradio.app/) by Hugging Face
models/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # Video Blur AI - Models
models/model_manager.py ADDED
@@ -0,0 +1,419 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Model Manager - Handles loading and inference for Grounding DINO + SAM 2
3
+ """
4
+ import os
5
+ import torch
6
+ import numpy as np
7
+ from PIL import Image
8
+ from typing import List, Dict, Tuple, Optional
9
+ from dataclasses import dataclass
10
+
11
+ @dataclass
12
+ class DetectionResult:
13
+ """Single detection result"""
14
+ label: str
15
+ confidence: float
16
+ bbox: np.ndarray # [x1, y1, x2, y2]
17
+ mask: Optional[np.ndarray] = None # H x W binary mask
18
+
19
+
20
+ class ModelManager:
21
+ """Manages Grounding DINO + SAM 2 pipeline"""
22
+
23
+ def __init__(self, config):
24
+ self.config = config
25
+ self.device = config.model.device
26
+ self.gdino_model = None
27
+ self.gdino_processor = None
28
+ self.sam2_predictor = None
29
+ self.sam2_video_predictor = None
30
+ self._loaded = False
31
+
32
+ def load_models(self, progress_callback=None):
33
+ """Load all models into memory"""
34
+ if self._loaded:
35
+ return
36
+
37
+ if progress_callback:
38
+ progress_callback(0.1, "Loading Grounding DINO...")
39
+ self._load_grounding_dino()
40
+
41
+ if progress_callback:
42
+ progress_callback(0.5, "Loading SAM 2...")
43
+ self._load_sam2()
44
+
45
+ self._loaded = True
46
+ if progress_callback:
47
+ progress_callback(1.0, "Models loaded ✅")
48
+
49
+ def _load_grounding_dino(self):
50
+ """Load Grounding DINO model"""
51
+ try:
52
+ # Try HuggingFace Transformers first (easier setup)
53
+ from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection
54
+
55
+ model_id = self.config.model.gdino_model_id
56
+ print(f"📥 Loading {model_id} (cached in ~/.cache/huggingface after first download)")
57
+ self.gdino_processor = AutoProcessor.from_pretrained(model_id)
58
+ self.gdino_model = AutoModelForZeroShotObjectDetection.from_pretrained(
59
+ model_id
60
+ ).to(self.device)
61
+
62
+ if self.config.model.use_fp16 and self.device == "cuda":
63
+ self.gdino_model = self.gdino_model.half()
64
+
65
+ self.gdino_model.eval()
66
+ print(f"✅ Grounding DINO loaded from {model_id}")
67
+ self._gdino_backend = "transformers"
68
+
69
+ except Exception as e:
70
+ print(f"⚠️ Transformers loading failed ({e}), trying GroundingDINO package...")
71
+ self._load_grounding_dino_native()
72
+
73
+ def _load_grounding_dino_native(self):
74
+ """Fallback: Load Grounding DINO from official package"""
75
+ try:
76
+ from groundingdino.util.inference import load_model, predict
77
+ from huggingface_hub import hf_hub_download
78
+
79
+ # Download checkpoint
80
+ ckpt_path = hf_hub_download(
81
+ repo_id="ShilongLiu/GroundingDINO",
82
+ filename="groundingdino_swinb_cogcoor.pth"
83
+ )
84
+ config_path = hf_hub_download(
85
+ repo_id="ShilongLiu/GroundingDINO",
86
+ filename="GroundingDINO_SwinB.cfg.py"
87
+ )
88
+
89
+ self.gdino_model = load_model(config_path, ckpt_path, device=self.device)
90
+ self._gdino_backend = "native"
91
+ print("✅ Grounding DINO loaded (native)")
92
+ except ImportError:
93
+ raise RuntimeError(
94
+ "❌ Grounding DINO failed to load!\n"
95
+ "The HuggingFace Transformers backend failed, and the native package is not installed.\n"
96
+ "Fix: pip install git+https://github.com/IDEA-Research/GroundingDINO.git\n"
97
+ "Or check that 'transformers' is up to date: pip install -U transformers"
98
+ )
99
+
100
+ def _load_sam2(self):
101
+ """Load SAM 2 model"""
102
+ try:
103
+ # Try importing from sam2 (PyPI: pip install sam-2)
104
+ try:
105
+ from sam2.build_sam import build_sam2, build_sam2_video_predictor
106
+ from sam2.sam2_image_predictor import SAM2ImagePredictor
107
+ except ImportError:
108
+ # Older versions may have different import paths
109
+ from sam2.build_sam import build_sam2
110
+ from sam2.automatic_mask_generator import SAM2ImagePredictor
111
+ build_sam2_video_predictor = None
112
+
113
+ from huggingface_hub import hf_hub_download
114
+
115
+ checkpoint = self.config.model.sam2_checkpoint
116
+ model_cfg = self.config.model.sam2_model_cfg
117
+
118
+ # Download checkpoint from HuggingFace
119
+ ckpt_map = {
120
+ "facebook/sam2.1-hiera-base-plus": "sam2.1_hiera_base_plus.pt",
121
+ "facebook/sam2.1-hiera-small": "sam2.1_hiera_small.pt",
122
+ "facebook/sam2.1-hiera-large": "sam2.1_hiera_large.pt",
123
+ "facebook/sam2.1-hiera-tiny": "sam2.1_hiera_tiny.pt",
124
+ }
125
+ ckpt_file = ckpt_map.get(checkpoint, "sam2.1_hiera_base_plus.pt")
126
+
127
+ try:
128
+ ckpt_path = hf_hub_download(
129
+ repo_id=checkpoint,
130
+ filename=ckpt_file
131
+ )
132
+ except Exception:
133
+ # Try without version suffix
134
+ alt_file = ckpt_file.replace("sam2.1_", "sam2_")
135
+ ckpt_path = hf_hub_download(
136
+ repo_id=checkpoint,
137
+ filename=alt_file
138
+ )
139
+
140
+ # Build image predictor
141
+ sam2_model = build_sam2(model_cfg, ckpt_path, device=self.device)
142
+ self.sam2_predictor = SAM2ImagePredictor(sam2_model)
143
+
144
+ # Build video predictor (may not be available in all versions)
145
+ if build_sam2_video_predictor is not None:
146
+ try:
147
+ self.sam2_video_predictor = build_sam2_video_predictor(
148
+ model_cfg, ckpt_path, device=self.device
149
+ )
150
+ except Exception as e:
151
+ print(f"⚠️ Video predictor not available: {e}")
152
+ print(" Will use frame-by-frame mode only.")
153
+ self.sam2_video_predictor = None
154
+ else:
155
+ self.sam2_video_predictor = None
156
+
157
+ print(f"✅ SAM 2 loaded from {checkpoint}")
158
+
159
+ except Exception as e:
160
+ print(f"❌ SAM 2 loading failed: {e}")
161
+ print(" Make sure sam-2 is installed: pip install sam-2>=1.1.0")
162
+ raise
163
+
164
+ def detect_objects(self, image: np.ndarray, text_prompt: str) -> List[DetectionResult]:
165
+ """
166
+ Detect objects in image using text prompt via Grounding DINO
167
+
168
+ Args:
169
+ image: BGR numpy array (H, W, 3)
170
+ text_prompt: Text description of objects to detect (e.g., "face. hand. text.")
171
+
172
+ Returns:
173
+ List of DetectionResult with bounding boxes
174
+ """
175
+ # Normalize prompt - ensure it ends with period for GDINO
176
+ prompt = text_prompt.strip()
177
+ if not prompt.endswith("."):
178
+ prompt += "."
179
+
180
+ pil_image = Image.fromarray(image[..., ::-1]) # BGR -> RGB -> PIL
181
+
182
+ if self._gdino_backend == "transformers":
183
+ return self._detect_transformers(pil_image, prompt)
184
+ else:
185
+ return self._detect_native(image, prompt)
186
+
187
+ def _detect_transformers(self, pil_image: Image.Image, prompt: str) -> List[DetectionResult]:
188
+ """Detection using HuggingFace Transformers (auto-detects API version)"""
189
+ import inspect
190
+
191
+ inputs = self.gdino_processor(
192
+ images=pil_image,
193
+ text=prompt,
194
+ return_tensors="pt"
195
+ ).to(self.device)
196
+
197
+ with torch.no_grad():
198
+ if self.config.model.use_fp16 and self.device == "cuda":
199
+ with torch.autocast("cuda"):
200
+ outputs = self.gdino_model(**inputs)
201
+ else:
202
+ outputs = self.gdino_model(**inputs)
203
+
204
+ target_sizes = [pil_image.size[::-1]] # (H, W)
205
+ threshold = self.config.model.gdino_box_threshold
206
+
207
+ # Inspect the actual function signature to know which params it accepts
208
+ post_fn = self.gdino_processor.post_process_grounded_object_detection
209
+ sig = inspect.signature(post_fn)
210
+ param_names = list(sig.parameters.keys())
211
+
212
+ kwargs = {"target_sizes": target_sizes}
213
+ args = [outputs]
214
+
215
+ # Add threshold with correct name
216
+ if "threshold" in param_names:
217
+ kwargs["threshold"] = threshold
218
+ elif "box_threshold" in param_names:
219
+ kwargs["box_threshold"] = threshold
220
+ kwargs["text_threshold"] = self.config.model.gdino_text_threshold
221
+
222
+ # Add input_ids if accepted
223
+ if "input_ids" in param_names:
224
+ args.append(inputs.get("input_ids", None))
225
+
226
+ results = post_fn(*args, **kwargs)[0]
227
+
228
+ detections = []
229
+
230
+ # Handle both 'text_labels' (new) and 'labels' (old) keys
231
+ labels = results.get("text_labels", results.get("labels", []))
232
+
233
+ for bbox, score, label in zip(
234
+ results["boxes"].cpu().numpy(),
235
+ results["scores"].cpu().numpy(),
236
+ labels
237
+ ):
238
+ label_str = str(label) if not isinstance(label, str) else label
239
+ detections.append(DetectionResult(
240
+ label=label_str,
241
+ confidence=float(score),
242
+ bbox=bbox
243
+ ))
244
+
245
+ return detections
246
+
247
+ def _detect_native(self, image: np.ndarray, prompt: str) -> List[DetectionResult]:
248
+ """Detection using native GroundingDINO"""
249
+ from groundingdino.util.inference import predict
250
+ from groundingdino.util.utils import get_phrases_from_posmap
251
+ import groundingdino.datasets.transforms as T
252
+
253
+ transform = T.Compose([
254
+ T.RandomResize([800], max_size=1333),
255
+ T.ToTensor(),
256
+ T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
257
+ ])
258
+
259
+ pil_image = Image.fromarray(image[..., ::-1])
260
+ transformed, _ = transform(pil_image, None)
261
+
262
+ boxes, logits, phrases = predict(
263
+ model=self.gdino_model,
264
+ image=transformed,
265
+ caption=prompt,
266
+ box_threshold=self.config.model.gdino_box_threshold,
267
+ text_threshold=self.config.model.gdino_text_threshold,
268
+ device=self.device
269
+ )
270
+
271
+ h, w = image.shape[:2]
272
+ detections = []
273
+ for box, score, label in zip(boxes, logits, phrases):
274
+ # Convert from [cx, cy, w, h] normalized to [x1, y1, x2, y2] pixels
275
+ cx, cy, bw, bh = box.cpu().numpy()
276
+ x1 = (cx - bw/2) * w
277
+ y1 = (cy - bh/2) * h
278
+ x2 = (cx + bw/2) * w
279
+ y2 = (cy + bh/2) * h
280
+
281
+ detections.append(DetectionResult(
282
+ label=label,
283
+ confidence=float(score),
284
+ bbox=np.array([x1, y1, x2, y2])
285
+ ))
286
+
287
+ return detections
288
+
289
+ def segment_with_boxes(self, image: np.ndarray, boxes: np.ndarray) -> np.ndarray:
290
+ """
291
+ Generate segmentation masks from bounding boxes using SAM 2
292
+
293
+ Args:
294
+ image: BGR numpy array (H, W, 3)
295
+ boxes: Array of boxes [N, 4] in [x1, y1, x2, y2] format
296
+
297
+ Returns:
298
+ Combined binary mask (H, W) uint8
299
+ """
300
+ rgb_image = image[..., ::-1] # BGR -> RGB
301
+ self.sam2_predictor.set_image(rgb_image)
302
+
303
+ if len(boxes) == 0:
304
+ return np.zeros(image.shape[:2], dtype=np.uint8)
305
+
306
+ input_boxes = torch.tensor(boxes, dtype=torch.float32, device=self.device)
307
+
308
+ with torch.no_grad():
309
+ if self.config.model.use_fp16 and self.device == "cuda":
310
+ with torch.autocast("cuda"):
311
+ masks, scores, _ = self.sam2_predictor.predict(
312
+ box=input_boxes,
313
+ multimask_output=False,
314
+ )
315
+ else:
316
+ masks, scores, _ = self.sam2_predictor.predict(
317
+ box=input_boxes,
318
+ multimask_output=False,
319
+ )
320
+
321
+ # Combine all masks into single mask
322
+ if isinstance(masks, torch.Tensor):
323
+ masks = masks.cpu().numpy()
324
+
325
+ combined_mask = np.zeros(image.shape[:2], dtype=np.uint8)
326
+ for mask in masks:
327
+ if mask.ndim == 3:
328
+ mask = mask[0] # Take first mask if multimask
329
+ combined_mask = np.maximum(combined_mask, (mask > 0.5).astype(np.uint8) * 255)
330
+
331
+ return combined_mask
332
+
333
+ def init_video_tracking(self, frames_dir: str, detections: List[DetectionResult]) -> dict:
334
+ """
335
+ Initialize SAM 2 video tracking from first-frame detections
336
+
337
+ Args:
338
+ frames_dir: Directory containing numbered JPEG frames
339
+ detections: Detection results from first frame
340
+
341
+ Returns:
342
+ SAM 2 inference state
343
+ """
344
+ state = self.sam2_video_predictor.init_state(video_path=frames_dir)
345
+
346
+ # Add each detection as a tracking target
347
+ for idx, det in enumerate(detections):
348
+ box = det.bbox
349
+ _, _, mask_logits = self.sam2_video_predictor.add_new_points_or_box(
350
+ inference_state=state,
351
+ frame_idx=0,
352
+ obj_id=idx + 1,
353
+ box=box,
354
+ )
355
+
356
+ return state
357
+
358
+ def propagate_video(self, state, num_frames: int, progress_callback=None):
359
+ """
360
+ Propagate masks through all video frames
361
+
362
+ Args:
363
+ state: SAM 2 inference state
364
+ num_frames: Total number of frames
365
+ progress_callback: Optional callback(frame_idx, total_frames)
366
+
367
+ Returns:
368
+ Dict mapping frame_idx -> combined binary mask (H, W)
369
+ """
370
+ frame_masks = {}
371
+
372
+ for frame_idx, obj_ids, mask_logits in self.sam2_video_predictor.propagate_in_video(state):
373
+ # Combine all object masks
374
+ masks = (mask_logits > 0.0).cpu().numpy() # [N, 1, H, W]
375
+ combined = np.zeros(masks.shape[2:], dtype=np.uint8)
376
+
377
+ for mask in masks:
378
+ combined = np.maximum(combined, (mask[0] > 0).astype(np.uint8) * 255)
379
+
380
+ frame_masks[frame_idx] = combined
381
+
382
+ if progress_callback:
383
+ progress_callback(frame_idx, num_frames)
384
+
385
+ return frame_masks
386
+
387
+ def detect_and_segment_frame(self, frame: np.ndarray, text_prompt: str) -> np.ndarray:
388
+ """
389
+ Full pipeline: detect + segment on a single frame
390
+
391
+ Args:
392
+ frame: BGR numpy array
393
+ text_prompt: What to detect
394
+
395
+ Returns:
396
+ Binary mask (H, W) uint8, 0 or 255
397
+ """
398
+ detections = self.detect_objects(frame, text_prompt)
399
+
400
+ if not detections:
401
+ return np.zeros(frame.shape[:2], dtype=np.uint8)
402
+
403
+ boxes = np.array([d.bbox for d in detections])
404
+ mask = self.segment_with_boxes(frame, boxes)
405
+
406
+ return mask
407
+
408
+ def unload_models(self):
409
+ """Free GPU memory"""
410
+ self.gdino_model = None
411
+ self.gdino_processor = None
412
+ self.sam2_predictor = None
413
+ self.sam2_video_predictor = None
414
+ self._loaded = False
415
+
416
+ if torch.cuda.is_available():
417
+ torch.cuda.empty_cache()
418
+
419
+ print("🗑️ Models unloaded")
packages.txt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ ffmpeg
2
+ libgl1
3
+ libglib2.0-0
pipeline.py ADDED
@@ -0,0 +1,386 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Processing Pipeline - Orchestrates the complete video blur workflow
3
+
4
+ Two processing modes:
5
+ 1. SAM 2 Video Mode: Extract frames → detect on keyframe → track with SAM 2 → blur → assemble
6
+ Best for: consistent tracking, objects that move continuously
7
+
8
+ 2. Frame-by-Frame Mode: Read frame → detect + segment → blur → write
9
+ Best for: simplicity, objects appearing/disappearing frequently, lower VRAM
10
+ """
11
+ import os
12
+ import cv2
13
+ import time
14
+ import shutil
15
+ import numpy as np
16
+ from pathlib import Path
17
+ from typing import Optional, Callable
18
+
19
+ from config import AppConfig
20
+ from models.model_manager import ModelManager
21
+ from utils.video_processor import VideoProcessor
22
+ from utils.blur_engine import BlurEngine
23
+
24
+
25
+ class VideoBlurPipeline:
26
+ """Complete video blur processing pipeline"""
27
+
28
+ def __init__(self, config: AppConfig):
29
+ self.config = config
30
+ self.model_manager = ModelManager(config)
31
+ self.video_processor = VideoProcessor(config)
32
+ self.blur_engine = BlurEngine(config)
33
+ self._models_loaded = False
34
+ self._last_gdino_id = None
35
+
36
+ def ensure_models_loaded(self, progress_callback=None):
37
+ """Load models if not already loaded or if config changed"""
38
+ if not self._models_loaded or self._last_gdino_id != self.config.model.gdino_model_id:
39
+ # Unload old models if switching
40
+ if self._models_loaded:
41
+ self.model_manager.unload_models()
42
+ self._models_loaded = False
43
+
44
+ self.model_manager = ModelManager(self.config)
45
+ self.model_manager.load_models(progress_callback)
46
+ self._models_loaded = True
47
+ self._last_gdino_id = self.config.model.gdino_model_id
48
+
49
+ def process_video(
50
+ self,
51
+ video_path: str,
52
+ text_prompt: str,
53
+ output_path: Optional[str] = None,
54
+ blur_type: str = "gaussian",
55
+ blur_strength: int = 51,
56
+ edge_feather: int = 11,
57
+ processing_mode: str = "video_tracking",
58
+ keyframe_interval: int = 5,
59
+ detection_threshold: float = 0.3,
60
+ progress_callback: Optional[Callable] = None,
61
+ ) -> str:
62
+ """
63
+ Main entry point: Process video with text-prompted blur
64
+
65
+ Args:
66
+ video_path: Path to input video
67
+ text_prompt: What to blur (e.g., "face", "hand", "license plate")
68
+ output_path: Where to save result (auto-generated if None)
69
+ blur_type: "gaussian", "pixelate", or "black"
70
+ blur_strength: Blur kernel size
71
+ edge_feather: Edge softening amount
72
+ processing_mode: "video_tracking" or "frame_by_frame"
73
+ keyframe_interval: How often to re-detect (frame_by_frame mode)
74
+ detection_threshold: Confidence threshold for detection
75
+ progress_callback: Callback(progress_float, status_text)
76
+
77
+ Returns:
78
+ Path to output video
79
+ """
80
+ start_time = time.time()
81
+
82
+ # Setup
83
+ if output_path is None:
84
+ base = Path(video_path)
85
+ output_path = str(base.parent / f"{base.stem}_blurred{base.suffix}")
86
+
87
+ # Update config
88
+ self.config.model.gdino_box_threshold = detection_threshold
89
+ self.config.blur.blur_type = blur_type
90
+ self.config.blur.blur_strength = blur_strength
91
+ self.config.blur.edge_feather = edge_feather
92
+
93
+ # Load models
94
+ self._progress(progress_callback, 0.05, "🔄 Loading AI models...")
95
+ self.ensure_models_loaded()
96
+
97
+ # Get video info
98
+ self._progress(progress_callback, 0.1, "📹 Analyzing video...")
99
+ video_info = self.video_processor.get_video_info(video_path)
100
+ print(f"📹 Input: {video_info.width}x{video_info.height}, "
101
+ f"{video_info.fps:.1f} FPS, {video_info.total_frames} frames, "
102
+ f"{video_info.duration:.1f}s")
103
+
104
+ # Choose processing mode
105
+ if processing_mode == "video_tracking":
106
+ # Check if video predictor is available
107
+ if self.model_manager.sam2_video_predictor is None:
108
+ print("⚠️ SAM 2 video predictor not available, falling back to frame-by-frame mode")
109
+ processing_mode = "frame_by_frame"
110
+
111
+ if processing_mode == "video_tracking":
112
+ result = self._process_video_tracking(
113
+ video_path, text_prompt, output_path,
114
+ video_info, progress_callback
115
+ )
116
+ else:
117
+ result = self._process_frame_by_frame(
118
+ video_path, text_prompt, output_path,
119
+ video_info, keyframe_interval, progress_callback
120
+ )
121
+
122
+ elapsed = time.time() - start_time
123
+ fps_processed = video_info.total_frames / elapsed
124
+ self._progress(
125
+ progress_callback, 1.0,
126
+ f"✅ Done! {elapsed:.1f}s ({fps_processed:.1f} FPS)"
127
+ )
128
+
129
+ return result
130
+
131
+ def _process_video_tracking(
132
+ self, video_path, text_prompt, output_path,
133
+ video_info, progress_callback
134
+ ) -> str:
135
+ """
136
+ Mode 1: SAM 2 Video Tracking
137
+
138
+ 1. Extract all frames to disk
139
+ 2. Detect objects on first frame
140
+ 3. Initialize SAM 2 video tracker
141
+ 4. Propagate masks through video
142
+ 5. Apply blur to each frame
143
+ 6. Reassemble video with audio
144
+ """
145
+ temp_dir = os.path.join(self.config.video.temp_dir, "processing")
146
+ frames_dir = os.path.join(temp_dir, "frames")
147
+ blurred_dir = os.path.join(temp_dir, "blurred")
148
+ os.makedirs(blurred_dir, exist_ok=True)
149
+
150
+ try:
151
+ # Step 1: Extract frames
152
+ self._progress(progress_callback, 0.15, "🎞️ Extracting frames...")
153
+ self.video_processor.extract_frames_to_dir(
154
+ video_path, frames_dir,
155
+ max_height=self.config.video.max_resolution
156
+ )
157
+
158
+ # Load first frame for detection
159
+ frame_files = sorted(os.listdir(frames_dir))
160
+ first_frame_path = os.path.join(frames_dir, frame_files[0])
161
+ first_frame = cv2.imread(first_frame_path)
162
+
163
+ # Step 2: Detect objects
164
+ self._progress(progress_callback, 0.25, f"🔍 Detecting: '{text_prompt}'...")
165
+ detections = self.model_manager.detect_objects(first_frame, text_prompt)
166
+
167
+ if not detections:
168
+ self._progress(progress_callback, 1.0,
169
+ f"⚠️ No '{text_prompt}' detected in video. Returning original.")
170
+ shutil.copy2(video_path, output_path)
171
+ return output_path
172
+
173
+ print(f"🎯 Detected {len(detections)} objects: "
174
+ f"{[f'{d.label} ({d.confidence:.2f})' for d in detections]}")
175
+
176
+ # Step 3: Initialize video tracking
177
+ self._progress(progress_callback, 0.3, "🎯 Initializing tracking...")
178
+ state = self.model_manager.init_video_tracking(frames_dir, detections)
179
+
180
+ # Step 4: Propagate masks
181
+ self._progress(progress_callback, 0.35, "🔄 Tracking objects through video...")
182
+ total_frames = len(frame_files)
183
+
184
+ def track_progress(frame_idx, total):
185
+ p = 0.35 + (frame_idx / total) * 0.3 # 0.35 -> 0.65
186
+ self._progress(progress_callback, p,
187
+ f"🔄 Tracking: frame {frame_idx+1}/{total}")
188
+
189
+ frame_masks = self.model_manager.propagate_video(
190
+ state, total_frames, track_progress
191
+ )
192
+
193
+ # Step 5: Apply blur
194
+ self._progress(progress_callback, 0.65, "🌫️ Applying blur...")
195
+ for i, fname in enumerate(frame_files):
196
+ frame_path = os.path.join(frames_dir, fname)
197
+ frame = cv2.imread(frame_path)
198
+
199
+ mask = frame_masks.get(i, np.zeros(frame.shape[:2], dtype=np.uint8))
200
+
201
+ # Resize mask if needed
202
+ if mask.shape[:2] != frame.shape[:2]:
203
+ mask = cv2.resize(mask, (frame.shape[1], frame.shape[0]))
204
+ mask = (mask > 127).astype(np.uint8) * 255
205
+
206
+ blurred = self.blur_engine.apply_blur(frame, mask)
207
+
208
+ output_frame_path = os.path.join(blurred_dir, fname)
209
+ cv2.imwrite(output_frame_path, blurred, [cv2.IMWRITE_JPEG_QUALITY, 95])
210
+
211
+ p = 0.65 + (i / total_frames) * 0.25 # 0.65 -> 0.9
212
+ if i % 10 == 0:
213
+ self._progress(progress_callback, p,
214
+ f"🌫️ Blurring: frame {i+1}/{total_frames}")
215
+
216
+ # Step 6: Assemble video
217
+ self._progress(progress_callback, 0.9, "🎬 Assembling video...")
218
+ self.video_processor.assemble_video(
219
+ blurred_dir, output_path,
220
+ video_info.fps, original_video=video_path
221
+ )
222
+
223
+ return output_path
224
+
225
+ finally:
226
+ # Cleanup
227
+ self.video_processor.cleanup_temp(temp_dir)
228
+
229
+ def _process_frame_by_frame(
230
+ self, video_path, text_prompt, output_path,
231
+ video_info, keyframe_interval, progress_callback
232
+ ) -> str:
233
+ """
234
+ Mode 2: Frame-by-Frame Processing
235
+
236
+ Process each frame independently with periodic re-detection.
237
+ Uses OpenCV for reading and a temp directory for output frames.
238
+ """
239
+ temp_dir = os.path.join(self.config.video.temp_dir, "processing")
240
+ output_frames_dir = os.path.join(temp_dir, "output_frames")
241
+ os.makedirs(output_frames_dir, exist_ok=True)
242
+
243
+ try:
244
+ total_frames = video_info.total_frames
245
+ last_mask = None
246
+ frame_idx = 0
247
+
248
+ for idx, frame in self.video_processor.read_frames_generator(
249
+ video_path, max_height=self.config.video.max_resolution
250
+ ):
251
+ # Detect + segment on keyframes
252
+ if idx % keyframe_interval == 0 or last_mask is None:
253
+ mask = self.model_manager.detect_and_segment_frame(
254
+ frame, text_prompt
255
+ )
256
+ if mask.max() > 0:
257
+ last_mask = mask
258
+ else:
259
+ # Reuse last mask (approximate between keyframes)
260
+ mask = last_mask if last_mask is not None else \
261
+ np.zeros(frame.shape[:2], dtype=np.uint8)
262
+
263
+ # Apply blur
264
+ blurred = self.blur_engine.apply_blur(frame, mask)
265
+
266
+ # Save frame
267
+ fname = f"{idx+1:06d}.jpg"
268
+ cv2.imwrite(
269
+ os.path.join(output_frames_dir, fname),
270
+ blurred, [cv2.IMWRITE_JPEG_QUALITY, 95]
271
+ )
272
+
273
+ # Progress
274
+ if idx % 5 == 0:
275
+ p = 0.15 + (idx / max(total_frames, 1)) * 0.75
276
+ self._progress(progress_callback, p,
277
+ f"⚡ Processing: frame {idx+1}/{total_frames}")
278
+
279
+ frame_idx = idx
280
+
281
+ # Assemble video
282
+ self._progress(progress_callback, 0.9, "🎬 Assembling video...")
283
+ self.video_processor.assemble_video(
284
+ output_frames_dir, output_path,
285
+ video_info.fps, original_video=video_path
286
+ )
287
+
288
+ return output_path
289
+
290
+ finally:
291
+ self.video_processor.cleanup_temp(temp_dir)
292
+
293
+ def process_image(
294
+ self,
295
+ image_bgr: np.ndarray,
296
+ text_prompt: str,
297
+ blur_type: str = "gaussian",
298
+ blur_strength: int = 51,
299
+ edge_feather: int = 11,
300
+ detection_threshold: float = 0.3,
301
+ progress_callback: Optional[Callable] = None,
302
+ ) -> np.ndarray:
303
+ """
304
+ Process a single image with text-prompted blur.
305
+
306
+ Args:
307
+ image_bgr: Input image as a BGR numpy array (H, W, 3)
308
+ text_prompt: What to blur (e.g., "face. license plate.")
309
+ blur_type: "gaussian", "pixelate", or "black"
310
+ blur_strength: Blur kernel size (odd number)
311
+ edge_feather: Edge softening amount (odd number)
312
+ detection_threshold: Confidence threshold for detection
313
+
314
+ Returns:
315
+ Blurred image as a BGR numpy array (H, W, 3)
316
+ """
317
+ # Update config
318
+ self.config.model.gdino_box_threshold = detection_threshold
319
+ self.config.blur.blur_type = blur_type
320
+ self.config.blur.blur_strength = blur_strength
321
+ self.config.blur.edge_feather = edge_feather
322
+
323
+ # Load models (inside the GPU context on ZeroGPU)
324
+ self._progress(progress_callback, 0.1, "🔄 Loading AI models...")
325
+ self.ensure_models_loaded()
326
+
327
+ # Detect + segment
328
+ self._progress(progress_callback, 0.4, f"🔍 Detecting: '{text_prompt}'...")
329
+ mask = self.model_manager.detect_and_segment_frame(image_bgr, text_prompt)
330
+
331
+ if mask is None or mask.max() == 0:
332
+ self._progress(
333
+ progress_callback, 1.0,
334
+ f"⚠️ No '{text_prompt}' detected. Returning original image."
335
+ )
336
+ return image_bgr.copy()
337
+
338
+ # Apply blur
339
+ self._progress(progress_callback, 0.8, "🌫️ Applying blur...")
340
+ result = self.blur_engine.apply_blur(image_bgr, mask)
341
+
342
+ self._progress(progress_callback, 1.0, "✅ Done!")
343
+ return result
344
+
345
+ def preview_detection(
346
+ self,
347
+ video_path: str,
348
+ text_prompt: str,
349
+ frame_number: int = 0
350
+ ) -> np.ndarray:
351
+ """
352
+ Preview detection on a single frame (for UI)
353
+
354
+ Returns:
355
+ BGR image with detection overlay
356
+ """
357
+ self.ensure_models_loaded()
358
+
359
+ cap = cv2.VideoCapture(video_path)
360
+ cap.set(cv2.CAP_PROP_POS_FRAMES, frame_number)
361
+ ret, frame = cap.read()
362
+ cap.release()
363
+
364
+ if not ret:
365
+ raise ValueError(f"Cannot read frame {frame_number}")
366
+
367
+ # Resize if needed
368
+ max_h = self.config.video.max_resolution
369
+ if frame.shape[0] > max_h:
370
+ scale = max_h / frame.shape[0]
371
+ frame = cv2.resize(frame, (int(frame.shape[1] * scale), max_h))
372
+
373
+ # Detect and segment
374
+ mask = self.model_manager.detect_and_segment_frame(frame, text_prompt)
375
+
376
+ # Create visualization
377
+ vis = self.blur_engine.visualize_mask(frame, mask, color=(0, 200, 255))
378
+
379
+ return vis
380
+
381
+ @staticmethod
382
+ def _progress(callback, value, text):
383
+ """Safe progress callback"""
384
+ if callback:
385
+ callback(value, text)
386
+ print(f"[{value*100:.0f}%] {text}")
requirements.txt ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ── Video Blur AI — Hugging Face Spaces (ZeroGPU) ──
2
+ #
3
+ # NOTE ON torch: on ZeroGPU, torch/torchvision are provided by the
4
+ # runtime and are already CUDA-matched. Do NOT pin them here — listing
5
+ # them can trigger a reinstall of an incompatible build. For LOCAL runs,
6
+ # install torch first, e.g.:
7
+ # pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
8
+
9
+ # ZeroGPU: provides the @spaces.GPU decorator
10
+ spaces
11
+
12
+ # Segment Anything Model 2 (installed from source).
13
+ # ZeroGPU build images ship the CUDA toolkit, so the CUDA extension
14
+ # compiles at build time. If the build ever fails, add a Space variable
15
+ # SAM2_BUILD_CUDA=0 to fall back to the pure-PyTorch path.
16
+ git+https://github.com/facebookresearch/sam2.git
17
+
18
+ # Grounding DINO is loaded through transformers (no separate package)
19
+ transformers>=4.35.0
20
+
21
+ # Video & image processing
22
+ opencv-python>=4.8.0
23
+ numpy>=1.24.0
24
+ Pillow>=10.0.0
25
+ ffmpeg-python>=0.2.0
26
+
27
+ # Web UI
28
+ gradio>=4.20.0
29
+
30
+ # Utilities
31
+ tqdm>=4.65.0
32
+ supervision>=0.18.0
33
+ huggingface-hub>=0.20.0
run.sh ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ #!/bin/bash
2
+ source "/home/abdo96/.venvs/video-blur-ai/bin/activate"
3
+ cd "/media/abdo96/ABODSALOMHA/LLMs_apps_github/video-blur-ai"
4
+ python app.py "$@"
run_cli.sh ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ #!/bin/bash
2
+ source "/home/abdo96/.venvs/video-blur-ai/bin/activate"
3
+ cd "/media/abdo96/ABODSALOMHA/LLMs_apps_github/video-blur-ai"
4
+ python cli.py "$@"
setup.sh ADDED
@@ -0,0 +1,259 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ # ═══════════════════════════════════════════════════════════
3
+ # Video Blur AI - Automated Setup Script
4
+ # Handles NTFS/exFAT drives (no symlink support)
5
+ # ═══════════════════════════════════════════════════════════
6
+ set -e
7
+
8
+ echo "╔══════════════════════════════════════════════════╗"
9
+ echo "║ 🎬 Video Blur AI - Setup ║"
10
+ echo "╚══════════════════════════════════════════════════╝"
11
+ echo ""
12
+
13
+ PROJECT_DIR="$(cd "$(dirname "$0")" && pwd)"
14
+ echo "📂 Project: $PROJECT_DIR"
15
+
16
+ # ── Check Python ──
17
+ echo "🐍 Checking Python..."
18
+ if ! command -v python3 &> /dev/null; then
19
+ echo "❌ Python 3 not found! Install Python 3.10+"
20
+ exit 1
21
+ fi
22
+ PYTHON_VERSION=$(python3 -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
23
+ echo " Python $PYTHON_VERSION ✅"
24
+
25
+ # ── Check FFmpeg ──
26
+ echo "📹 Checking FFmpeg..."
27
+ if ! command -v ffmpeg &> /dev/null; then
28
+ echo "⚠️ FFmpeg not found. Installing..."
29
+ if command -v apt &> /dev/null; then
30
+ sudo apt update && sudo apt install -y ffmpeg
31
+ elif command -v brew &> /dev/null; then
32
+ brew install ffmpeg
33
+ else
34
+ echo "❌ Install FFmpeg manually: https://ffmpeg.org/download.html"
35
+ exit 1
36
+ fi
37
+ fi
38
+ echo " FFmpeg ✅"
39
+
40
+ # ── Check Git ──
41
+ echo "📦 Checking Git..."
42
+ if ! command -v git &> /dev/null; then
43
+ echo "⚠️ Git not found. Installing..."
44
+ if command -v apt &> /dev/null; then
45
+ sudo apt install -y git
46
+ else
47
+ echo "❌ Install Git: https://git-scm.com"
48
+ exit 1
49
+ fi
50
+ fi
51
+ echo " Git ✅"
52
+
53
+ # ── Check CUDA ──
54
+ echo "🎮 Checking GPU..."
55
+ if command -v nvidia-smi &> /dev/null; then
56
+ GPU_NAME=$(nvidia-smi --query-gpu=name --format=csv,noheader | head -1)
57
+ GPU_MEM=$(nvidia-smi --query-gpu=memory.total --format=csv,noheader | head -1)
58
+ echo " GPU: $GPU_NAME ($GPU_MEM) ✅"
59
+ else
60
+ echo " ⚠️ No NVIDIA GPU. Will use CPU (slow)."
61
+ fi
62
+
63
+ # ══════════════════════════════════════════════════════════
64
+ # Detect filesystem — NTFS/exFAT can't create symlinks
65
+ # ══════════════════════════════════════════════════════════
66
+ CAN_SYMLINK=true
67
+ TEST_LINK="$PROJECT_DIR/.symtest_link_$$"
68
+ TEST_FILE="$PROJECT_DIR/.symtest_file_$$"
69
+ touch "$TEST_FILE" 2>/dev/null || true
70
+ if ! ln -s "$TEST_FILE" "$TEST_LINK" 2>/dev/null; then
71
+ CAN_SYMLINK=false
72
+ fi
73
+ rm -f "$TEST_LINK" "$TEST_FILE" 2>/dev/null
74
+
75
+ VENV_DIR="$PROJECT_DIR/venv"
76
+
77
+ if [ "$CAN_SYMLINK" = false ]; then
78
+ echo ""
79
+ echo "════════════════════════════════════════════════════"
80
+ echo "⚠️ NTFS/exFAT drive detected (no symlink support)"
81
+ echo " venv will be created in your home directory"
82
+ echo "════════════════════════════════════════════════════"
83
+
84
+ VENV_DIR="$HOME/.venvs/video-blur-ai"
85
+ mkdir -p "$(dirname "$VENV_DIR")"
86
+
87
+ # Create helper scripts
88
+ cat > "$PROJECT_DIR/run.sh" << EOF
89
+ #!/bin/bash
90
+ source "$VENV_DIR/bin/activate"
91
+ cd "$PROJECT_DIR"
92
+ python app.py "\$@"
93
+ EOF
94
+ chmod +x "$PROJECT_DIR/run.sh"
95
+
96
+ cat > "$PROJECT_DIR/run_cli.sh" << EOF
97
+ #!/bin/bash
98
+ source "$VENV_DIR/bin/activate"
99
+ cd "$PROJECT_DIR"
100
+ python cli.py "\$@"
101
+ EOF
102
+ chmod +x "$PROJECT_DIR/run_cli.sh"
103
+
104
+ cat > "$PROJECT_DIR/activate.sh" << EOF
105
+ #!/bin/bash
106
+ source "$VENV_DIR/bin/activate"
107
+ cd "$PROJECT_DIR"
108
+ echo "✅ Activated. venv: $VENV_DIR"
109
+ EOF
110
+ chmod +x "$PROJECT_DIR/activate.sh"
111
+
112
+ echo " Created run.sh, run_cli.sh, activate.sh ✅"
113
+ fi
114
+
115
+ echo " venv location: $VENV_DIR"
116
+
117
+ # ── Create Virtual Environment ──
118
+ echo ""
119
+ echo "📦 Creating virtual environment..."
120
+ if [ ! -d "$VENV_DIR" ]; then
121
+ python3 -m venv "$VENV_DIR"
122
+ echo " Created ✅"
123
+ else
124
+ echo " Already exists ✅"
125
+ fi
126
+ source "$VENV_DIR/bin/activate"
127
+ pip install --upgrade pip -q
128
+ echo " Activated ✅"
129
+
130
+ # ── Install PyTorch ──
131
+ echo ""
132
+ echo "🔥 Installing PyTorch (this may take a while)..."
133
+ if command -v nvidia-smi &> /dev/null; then
134
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121 -q
135
+ else
136
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu -q
137
+ fi
138
+ echo " PyTorch ✅"
139
+
140
+ # ── Install Dependencies ──
141
+ echo ""
142
+ echo "📚 Installing dependencies..."
143
+ pip install -q \
144
+ "opencv-python>=4.8.0" \
145
+ "numpy>=1.24.0" \
146
+ "Pillow>=10.0.0" \
147
+ "ffmpeg-python>=0.2.0" \
148
+ "gradio>=4.20.0" \
149
+ "tqdm>=4.65.0" \
150
+ "supervision>=0.18.0" \
151
+ "huggingface-hub>=0.20.0" \
152
+ "transformers>=4.35.0"
153
+ echo " Dependencies ✅"
154
+
155
+ # ── Install SAM 2 ──
156
+ echo ""
157
+ echo "🎯 Installing SAM 2 from source (official method)..."
158
+
159
+ SAM2_DIR="$HOME/.sam2_source"
160
+ if [ -d "$SAM2_DIR" ] && python3 -c "import sam2" 2>/dev/null; then
161
+ echo " SAM 2 already installed ✅"
162
+ else
163
+ rm -rf "$SAM2_DIR"
164
+ git clone --depth 1 https://github.com/facebookresearch/sam2.git "$SAM2_DIR"
165
+ cd "$SAM2_DIR"
166
+
167
+ # Skip CUDA extension build (works fine without it)
168
+ SAM2_BUILD_CUDA=0 pip install -e "." -q
169
+
170
+ cd "$PROJECT_DIR"
171
+ echo " SAM 2 installed ✅"
172
+ fi
173
+
174
+ # ── Install Grounding DINO ──
175
+ # Grounding DINO is used via HuggingFace Transformers (AutoModelForZeroShotObjectDetection)
176
+ # No separate package needed! transformers handles it.
177
+ # If you need the native package as fallback, uncomment below:
178
+ echo ""
179
+ echo "🔍 Grounding DINO will load via HuggingFace Transformers ✅"
180
+ # pip install -q "groundingdino @ git+https://github.com/IDEA-Research/GroundingDINO.git"
181
+
182
+ # ── Download Models ──
183
+ echo ""
184
+ echo "⬇️ Downloading model weights (first time only, cached after)..."
185
+ python3 << 'PYEOF'
186
+ from huggingface_hub import hf_hub_download
187
+ import os
188
+
189
+ cache_dir = os.path.expanduser("~/.cache/huggingface")
190
+ print(f" Cache dir: {cache_dir}")
191
+
192
+ print(" Downloading Grounding DINO Base (~999MB)...")
193
+ try:
194
+ from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection
195
+ AutoProcessor.from_pretrained("IDEA-Research/grounding-dino-base")
196
+ AutoModelForZeroShotObjectDetection.from_pretrained("IDEA-Research/grounding-dino-base")
197
+ print(" Grounding DINO Base ✅")
198
+ except Exception as e:
199
+ print(f" ⚠️ Will download on first run: {e}")
200
+
201
+ print(" Downloading SAM 2.1 Base+ (~350MB)...")
202
+ try:
203
+ hf_hub_download(repo_id="facebook/sam2.1-hiera-base-plus", filename="sam2.1_hiera_base_plus.pt")
204
+ print(" SAM 2.1 ✅")
205
+ except Exception as e:
206
+ print(f" ⚠️ Will download on first run: {e}")
207
+
208
+ print(" Models are cached in ~/.cache/huggingface — next launch will be instant.")
209
+ PYEOF
210
+
211
+ # ── Verify ──
212
+ echo ""
213
+ echo "🔧 Verifying..."
214
+ python3 -c "
215
+ import torch, cv2, gradio, numpy
216
+ print(f' PyTorch {torch.__version__}', '(CUDA)' if torch.cuda.is_available() else '(CPU)')
217
+ print(f' OpenCV {cv2.__version__} | Gradio {gradio.__version__} | NumPy {numpy.__version__}')
218
+ try:
219
+ import sam2
220
+ print(f' SAM 2 ✅')
221
+ except ImportError:
222
+ print(' ⚠️ SAM 2 import failed - check installation')
223
+ try:
224
+ from transformers import AutoModelForZeroShotObjectDetection
225
+ print(f' Grounding DINO (via transformers) ✅')
226
+ except ImportError:
227
+ print(' ⚠️ Transformers import failed')
228
+ print(' All OK ✅')
229
+ "
230
+
231
+ # ── Done ──
232
+ echo ""
233
+ echo "╔══════════════════════════════════════════════════╗"
234
+ echo "║ ✅ Setup Complete! ║"
235
+ echo "╠══════════════════════════════════════════════════╣"
236
+
237
+ if [ "$CAN_SYMLINK" = false ]; then
238
+ echo "║ ║"
239
+ echo "║ 🚀 Start the app: ║"
240
+ echo "║ ./run.sh ║"
241
+ echo "║ ║"
242
+ echo "║ 🖥️ CLI mode: ║"
243
+ echo "║ ./run_cli.sh video.mp4 -p \"face.\" ║"
244
+ echo "║ ║"
245
+ echo "║ Or manually: ║"
246
+ echo "║ source activate.sh ║"
247
+ echo "║ python app.py ║"
248
+ else
249
+ echo "║ ║"
250
+ echo "║ 🚀 Start the app: ║"
251
+ echo "║ source venv/bin/activate ║"
252
+ echo "║ python app.py ║"
253
+ fi
254
+
255
+ echo "║ ║"
256
+ echo "║ 🌐 Open: http://localhost:7860 ║"
257
+ echo "║ 📡 Public: python app.py --share ║"
258
+ echo "║ ║"
259
+ echo "╚══════════════════════════════════════════════════╝"
utils/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # Video Blur AI - Utilities
utils/blur_engine.py ADDED
@@ -0,0 +1,173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Blur Engine - Applies various blur effects using segmentation masks
3
+ Supports Gaussian blur, pixelation, and solid color overlay
4
+ """
5
+ import cv2
6
+ import numpy as np
7
+ from typing import Optional
8
+
9
+
10
+ class BlurEngine:
11
+ """Applies blur effects to images using segmentation masks"""
12
+
13
+ def __init__(self, config):
14
+ self.config = config
15
+
16
+ def apply_blur(
17
+ self,
18
+ frame: np.ndarray,
19
+ mask: np.ndarray,
20
+ blur_type: Optional[str] = None,
21
+ blur_strength: Optional[int] = None,
22
+ edge_feather: Optional[int] = None,
23
+ ) -> np.ndarray:
24
+ """
25
+ Apply blur to masked region of frame
26
+
27
+ Args:
28
+ frame: BGR image (H, W, 3) uint8
29
+ mask: Binary mask (H, W) uint8, 255 = blur region
30
+ blur_type: "gaussian", "pixelate", or "black"
31
+ blur_strength: Kernel size for Gaussian (must be odd)
32
+ edge_feather: Edge softening kernel (must be odd)
33
+
34
+ Returns:
35
+ Blurred frame (H, W, 3) uint8
36
+ """
37
+ blur_type = blur_type or self.config.blur.blur_type
38
+ blur_strength = blur_strength or self.config.blur.blur_strength
39
+ edge_feather = edge_feather or self.config.blur.edge_feather
40
+
41
+ # Ensure odd kernel sizes
42
+ blur_strength = blur_strength | 1
43
+ edge_feather = edge_feather | 1
44
+
45
+ # Skip if no mask
46
+ if mask is None or mask.max() == 0:
47
+ return frame.copy()
48
+
49
+ # Resize mask to match frame if needed
50
+ if mask.shape[:2] != frame.shape[:2]:
51
+ mask = cv2.resize(mask, (frame.shape[1], frame.shape[0]),
52
+ interpolation=cv2.INTER_LINEAR)
53
+ mask = (mask > 127).astype(np.uint8) * 255
54
+
55
+ # Generate the blurred/modified version
56
+ if blur_type == "gaussian":
57
+ modified = self._gaussian_blur(frame, blur_strength)
58
+ elif blur_type == "pixelate":
59
+ modified = self._pixelate(frame, self.config.blur.pixelate_size)
60
+ elif blur_type == "black":
61
+ modified = np.zeros_like(frame)
62
+ else:
63
+ modified = self._gaussian_blur(frame, blur_strength)
64
+
65
+ # Apply edge feathering for smooth transitions
66
+ mask_soft = self._feather_mask(mask, edge_feather)
67
+
68
+ # Composite: blend original and modified using soft mask
69
+ result = self._composite(frame, modified, mask_soft)
70
+
71
+ return result
72
+
73
+ def _gaussian_blur(self, frame: np.ndarray, kernel_size: int) -> np.ndarray:
74
+ """Apply Gaussian blur to entire frame"""
75
+ return cv2.GaussianBlur(frame, (kernel_size, kernel_size), 0)
76
+
77
+ def _pixelate(self, frame: np.ndarray, block_size: int) -> np.ndarray:
78
+ """Apply pixelation effect"""
79
+ h, w = frame.shape[:2]
80
+ small = cv2.resize(frame, (w // block_size, h // block_size),
81
+ interpolation=cv2.INTER_LINEAR)
82
+ pixelated = cv2.resize(small, (w, h), interpolation=cv2.INTER_NEAREST)
83
+ return pixelated
84
+
85
+ def _feather_mask(self, mask: np.ndarray, kernel_size: int) -> np.ndarray:
86
+ """
87
+ Create soft-edged mask for smooth blur transitions
88
+
89
+ Returns:
90
+ Float mask (H, W) in range [0, 1]
91
+ """
92
+ # Optional: dilate slightly to ensure full coverage
93
+ dilate_kernel = np.ones((3, 3), np.uint8)
94
+ mask_dilated = cv2.dilate(mask, dilate_kernel, iterations=1)
95
+
96
+ # Apply Gaussian blur to create soft edges
97
+ mask_soft = cv2.GaussianBlur(
98
+ mask_dilated.astype(np.float32),
99
+ (kernel_size, kernel_size),
100
+ 0
101
+ )
102
+
103
+ # Normalize to [0, 1]
104
+ mask_max = mask_soft.max()
105
+ if mask_max > 0:
106
+ mask_soft = mask_soft / mask_max
107
+
108
+ return mask_soft
109
+
110
+ def _composite(
111
+ self,
112
+ original: np.ndarray,
113
+ modified: np.ndarray,
114
+ mask_soft: np.ndarray
115
+ ) -> np.ndarray:
116
+ """
117
+ Alpha-blend original and modified frames using soft mask
118
+
119
+ result = original * (1 - mask) + modified * mask
120
+ """
121
+ # Expand mask to 3 channels
122
+ mask_3ch = np.stack([mask_soft] * 3, axis=-1)
123
+
124
+ # Blend
125
+ result = (
126
+ original.astype(np.float32) * (1.0 - mask_3ch) +
127
+ modified.astype(np.float32) * mask_3ch
128
+ )
129
+
130
+ return np.clip(result, 0, 255).astype(np.uint8)
131
+
132
+ def visualize_mask(
133
+ self,
134
+ frame: np.ndarray,
135
+ mask: np.ndarray,
136
+ color: tuple = (0, 255, 0),
137
+ alpha: float = 0.4
138
+ ) -> np.ndarray:
139
+ """
140
+ Overlay colored mask on frame for visualization
141
+
142
+ Args:
143
+ frame: BGR image
144
+ mask: Binary mask (H, W) uint8
145
+ color: BGR color tuple
146
+ alpha: Overlay transparency
147
+
148
+ Returns:
149
+ Visualization image
150
+ """
151
+ vis = frame.copy()
152
+
153
+ if mask is None or mask.max() == 0:
154
+ return vis
155
+
156
+ # Resize mask if needed
157
+ if mask.shape[:2] != frame.shape[:2]:
158
+ mask = cv2.resize(mask, (frame.shape[1], frame.shape[0]))
159
+
160
+ # Create colored overlay
161
+ overlay = np.zeros_like(frame)
162
+ overlay[mask > 127] = color
163
+
164
+ # Blend
165
+ vis = cv2.addWeighted(vis, 1.0, overlay, alpha, 0)
166
+
167
+ # Draw contours
168
+ contours, _ = cv2.findContours(
169
+ mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
170
+ )
171
+ cv2.drawContours(vis, contours, -1, color, 2)
172
+
173
+ return vis
utils/video_processor.py ADDED
@@ -0,0 +1,281 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Video Processor - Handles video I/O using FFmpeg
3
+ Supports frame extraction, video assembly, and audio preservation
4
+ """
5
+ import os
6
+ import cv2
7
+ import json
8
+ import shutil
9
+ import subprocess
10
+ import numpy as np
11
+ from pathlib import Path
12
+ from typing import Generator, Tuple, Optional, Dict
13
+ from dataclasses import dataclass
14
+
15
+
16
+ @dataclass
17
+ class VideoInfo:
18
+ """Video metadata"""
19
+ width: int
20
+ height: int
21
+ fps: float
22
+ total_frames: int
23
+ duration: float
24
+ has_audio: bool
25
+ codec: str
26
+ filepath: str
27
+
28
+
29
+ class VideoProcessor:
30
+ """Handles all video I/O operations"""
31
+
32
+ def __init__(self, config):
33
+ self.config = config
34
+ self._verify_ffmpeg()
35
+
36
+ def _verify_ffmpeg(self):
37
+ """Verify FFmpeg is installed"""
38
+ try:
39
+ result = subprocess.run(
40
+ ["ffmpeg", "-version"],
41
+ capture_output=True, text=True, timeout=5
42
+ )
43
+ if result.returncode != 0:
44
+ raise RuntimeError("FFmpeg not working properly")
45
+ except FileNotFoundError:
46
+ raise RuntimeError(
47
+ "FFmpeg not found! Install it:\n"
48
+ " Ubuntu: sudo apt install ffmpeg\n"
49
+ " Mac: brew install ffmpeg\n"
50
+ " Windows: choco install ffmpeg"
51
+ )
52
+
53
+ def get_video_info(self, video_path: str) -> VideoInfo:
54
+ """Extract video metadata using ffprobe"""
55
+ cmd = [
56
+ "ffprobe", "-v", "quiet",
57
+ "-print_format", "json",
58
+ "-show_format", "-show_streams",
59
+ video_path
60
+ ]
61
+
62
+ result = subprocess.run(cmd, capture_output=True, text=True)
63
+ if result.returncode != 0:
64
+ raise ValueError(f"Cannot read video: {video_path}")
65
+
66
+ probe = json.loads(result.stdout)
67
+
68
+ # Find video stream
69
+ video_stream = None
70
+ has_audio = False
71
+ for stream in probe.get("streams", []):
72
+ if stream["codec_type"] == "video" and video_stream is None:
73
+ video_stream = stream
74
+ elif stream["codec_type"] == "audio":
75
+ has_audio = True
76
+
77
+ if not video_stream:
78
+ raise ValueError("No video stream found")
79
+
80
+ # Parse FPS
81
+ fps_parts = video_stream.get("r_frame_rate", "30/1").split("/")
82
+ fps = float(fps_parts[0]) / float(fps_parts[1]) if len(fps_parts) == 2 else 30.0
83
+
84
+ # Parse frame count
85
+ nb_frames = int(video_stream.get("nb_frames", 0))
86
+ duration = float(probe.get("format", {}).get("duration", 0))
87
+
88
+ if nb_frames == 0 and duration > 0:
89
+ nb_frames = int(duration * fps)
90
+
91
+ return VideoInfo(
92
+ width=int(video_stream["width"]),
93
+ height=int(video_stream["height"]),
94
+ fps=fps,
95
+ total_frames=nb_frames,
96
+ duration=duration,
97
+ has_audio=has_audio,
98
+ codec=video_stream.get("codec_name", "unknown"),
99
+ filepath=video_path
100
+ )
101
+
102
+ def extract_frames_to_dir(
103
+ self,
104
+ video_path: str,
105
+ output_dir: str,
106
+ max_height: Optional[int] = None,
107
+ progress_callback=None
108
+ ) -> Tuple[str, VideoInfo]:
109
+ """
110
+ Extract all frames as numbered JPEGs (required by SAM 2)
111
+
112
+ Args:
113
+ video_path: Path to input video
114
+ output_dir: Directory to save frames
115
+ max_height: Optional max height for resizing
116
+ progress_callback: Optional callback(current, total)
117
+
118
+ Returns:
119
+ (frames_dir, video_info)
120
+ """
121
+ os.makedirs(output_dir, exist_ok=True)
122
+ info = self.get_video_info(video_path)
123
+
124
+ # Build FFmpeg command
125
+ cmd = ["ffmpeg", "-y", "-i", video_path]
126
+
127
+ # Add scaling if needed
128
+ if max_height and info.height > max_height:
129
+ cmd.extend(["-vf", f"scale=-2:{max_height}"])
130
+
131
+ # Output as numbered JPEGs
132
+ cmd.extend([
133
+ "-qscale:v", "2", # High quality JPEG
134
+ os.path.join(output_dir, "%06d.jpg")
135
+ ])
136
+
137
+ process = subprocess.Popen(
138
+ cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE
139
+ )
140
+ _, stderr = process.communicate()
141
+
142
+ if process.returncode != 0:
143
+ raise RuntimeError(f"Frame extraction failed: {stderr.decode()}")
144
+
145
+ # Count extracted frames
146
+ frame_files = sorted([
147
+ f for f in os.listdir(output_dir)
148
+ if f.endswith('.jpg')
149
+ ])
150
+ info.total_frames = len(frame_files)
151
+
152
+ # Update dimensions if resized
153
+ if frame_files:
154
+ sample = cv2.imread(os.path.join(output_dir, frame_files[0]))
155
+ if sample is not None:
156
+ info.height, info.width = sample.shape[:2]
157
+
158
+ print(f"📹 Extracted {info.total_frames} frames ({info.width}x{info.height} @ {info.fps:.1f} FPS)")
159
+ return output_dir, info
160
+
161
+ def read_frames_generator(
162
+ self,
163
+ video_path: str,
164
+ max_height: Optional[int] = None
165
+ ) -> Generator[Tuple[int, np.ndarray], None, None]:
166
+ """
167
+ Stream frames from video using OpenCV (memory efficient)
168
+
169
+ Yields:
170
+ (frame_index, frame_bgr)
171
+ """
172
+ cap = cv2.VideoCapture(video_path)
173
+ if not cap.isOpened():
174
+ raise ValueError(f"Cannot open video: {video_path}")
175
+
176
+ frame_idx = 0
177
+ while True:
178
+ ret, frame = cap.read()
179
+ if not ret:
180
+ break
181
+
182
+ # Resize if needed
183
+ if max_height and frame.shape[0] > max_height:
184
+ scale = max_height / frame.shape[0]
185
+ new_w = int(frame.shape[1] * scale)
186
+ frame = cv2.resize(frame, (new_w, max_height))
187
+
188
+ yield frame_idx, frame
189
+ frame_idx += 1
190
+
191
+ cap.release()
192
+
193
+ def assemble_video(
194
+ self,
195
+ frames_dir: str,
196
+ output_path: str,
197
+ fps: float,
198
+ original_video: Optional[str] = None,
199
+ progress_callback=None
200
+ ) -> str:
201
+ """
202
+ Assemble processed frames back into video with optional audio
203
+
204
+ Args:
205
+ frames_dir: Directory with numbered JPEG frames
206
+ output_path: Output video path
207
+ fps: Frame rate
208
+ original_video: Original video to copy audio from
209
+ progress_callback: Optional callback
210
+
211
+ Returns:
212
+ Path to output video
213
+ """
214
+ temp_video = output_path + ".temp.mp4"
215
+
216
+ # Step 1: Encode frames to video
217
+ cmd = [
218
+ "ffmpeg", "-y",
219
+ "-framerate", str(fps),
220
+ "-i", os.path.join(frames_dir, "%06d.jpg"),
221
+ "-c:v", self.config.video.output_codec,
222
+ "-crf", str(self.config.video.output_crf),
223
+ "-preset", self.config.video.output_preset,
224
+ "-pix_fmt", self.config.video.pixel_format,
225
+ "-movflags", "+faststart",
226
+ ]
227
+
228
+ if original_video:
229
+ cmd.append(temp_video)
230
+ else:
231
+ cmd.append(output_path)
232
+
233
+ process = subprocess.Popen(
234
+ cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE
235
+ )
236
+ _, stderr = process.communicate()
237
+
238
+ if process.returncode != 0:
239
+ raise RuntimeError(f"Video encoding failed: {stderr.decode()}")
240
+
241
+ # Step 2: Mux audio from original video
242
+ if original_video:
243
+ info = self.get_video_info(original_video)
244
+
245
+ if info.has_audio:
246
+ mux_cmd = [
247
+ "ffmpeg", "-y",
248
+ "-i", temp_video,
249
+ "-i", original_video,
250
+ "-c:v", "copy",
251
+ "-c:a", "aac",
252
+ "-map", "0:v:0",
253
+ "-map", "1:a:0?",
254
+ "-shortest",
255
+ output_path
256
+ ]
257
+
258
+ process = subprocess.Popen(
259
+ mux_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE
260
+ )
261
+ _, stderr = process.communicate()
262
+
263
+ if process.returncode != 0:
264
+ # Fallback: just use video without audio
265
+ shutil.move(temp_video, output_path)
266
+ print("⚠️ Audio muxing failed, output has no audio")
267
+ else:
268
+ os.remove(temp_video)
269
+ print("🔊 Audio preserved from original video")
270
+ else:
271
+ shutil.move(temp_video, output_path)
272
+
273
+ file_size_mb = os.path.getsize(output_path) / (1024 * 1024)
274
+ print(f"✅ Output video: {output_path} ({file_size_mb:.1f} MB)")
275
+
276
+ return output_path
277
+
278
+ def cleanup_temp(self, temp_dir: str):
279
+ """Remove temporary files"""
280
+ if os.path.exists(temp_dir):
281
+ shutil.rmtree(temp_dir, ignore_errors=True)