File size: 3,860 Bytes
c41ab7b 3afce52 c7f8992 70ed446 4d84cf4 9aa4b4e c7f8992 c41ab7b 76580be c41ab7b 76580be 84ac472 c41ab7b 6391dff 53cb751 6391dff c41ab7b 76580be c41ab7b 76580be 53cb751 c41ab7b 53cb751 c41ab7b 8e32139 76580be c41ab7b 6391dff c41ab7b 6391dff c41ab7b 84ac472 76580be c41ab7b 84ac472 c41ab7b 6391dff 53cb751 6391dff 53cb751 84ac472 c41ab7b 6391dff c41ab7b 84ac472 53cb751 8e32139 6391dff c41ab7b 84ac472 c41ab7b 84ac472 53cb751 84ac472 c41ab7b 84ac472 c41ab7b 84ac472 c41ab7b 84ac472 c41ab7b 6391dff c41ab7b 6391dff c41ab7b 53cb751 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 |
---
title: 🎬 BackgroundFX Pro - SAM2 + MatAnyone
emoji: 🎥
colorFrom: indigo
colorTo: purple
sdk: streamlit
sdk_version: 1.32.0
app_file: streamlit_app.py
pinned: false
license: mit
tags:
- video
- background-removal
- segmentation
- matting
- SAM2
- MatAnyone
---
# 🎬 BackgroundFX Pro — Professional Video Background Replacement
BackgroundFX Pro is a GPU-accelerated app for Hugging Face Spaces (Docker) that replaces video backgrounds using:
- **SAM2** — high-quality object segmentation
- **MatAnyone** — temporal video matting for stable alpha over time
Built on: **CUDA 12.1.1**, **PyTorch 2.5.1 (cu121)**, **torchvision 0.20.1**, **Streamlit 1.49.1**.
---
## ✨ Features
- Replace backgrounds with: **solid color**, **AI-generated** image (procedural), **custom uploaded image**, or **professional backgrounds**
- Optimized for **T4 GPUs** on Hugging Face
- Two-stage pipeline: SAM2 segmentation → MatAnyone refinement → compositing
- Caching & logs stored in the repo volume:
- HF cache → `./.hf`
- Torch cache → `./.torch`
- App data & logs → `./data` (see `data/run.log`)
- **FFmpeg** — video format conversion and frame extraction
---
## 🚀 Try It
Open the Space in your browser (GPU required):
https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2
---
## 🖱️ How to Use
1. **Upload a video** (`.mp4`, `.mov`, `.avi`, `.mkv`).
2. Choose a **Background Type**: Image, Color, Blur, Professional Backgrounds, or AI Generated.
3. If using custom background, upload your image or select from professional options.
4. Click **🚀 Process Video**.
5. Preview and **💾 Download Result**.
> Tip: Start with 720p/1080p on T4; 4K can exceed memory limits.
---
## 🗂️ Project Structure (key files)
- `Dockerfile` — CUDA 12.1.1 + PyTorch 2.5.1 container
- `requirements.txt` — Python dependencies
- `app.py` — Main Streamlit application
- `integrated_pipeline.py` — Two-stage processing pipeline
- `models/sam2_loader.py` — SAM2 model loader with HF Hub integration
- `models/matanyone_loader.py` — MatAnyone model loader
- `utils/` — Utility functions
- `data/` — Created at runtime for logs/outputs
- `tmp/` — Created at runtime for processing jobs - `video_pipeline.py` — Core video processing logic (SAM2 + MatAnyone integration)
- `video_pipeline.py` — Core video processing logic (SAM2 + MatAnyone integration)
---
## ⚙️ Runtime Notes
- Binds to `PORT` / `STREAMLIT_SERVER_PORT` (defaults to **7860**)
- File upload limit: 200MB via `--server.maxUploadSize=200`
- CORS disabled for Docker compatibility: `--server.enableCORS=false`
- Memory management with automatic cleanup between stages
- If processing fails, check Space logs for detailed error information
---
## 🧪 Local Development (Docker)
Requires an NVIDIA GPU with CUDA drivers.
```bash
git clone https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2
cd VideoBackgroundReplacer2
# Build (Ubuntu 22.04, CUDA 12.1.1; installs Torch 2.5.1+cu121)
docker build -t backgroundfx-pro .
# Run
docker run --gpus all -p 7860:7860 backgroundfx-pro
```
Access at: http://localhost:7860
---
## 🔧 Technical Details
### Pipeline Architecture
1. **Stage 1**: SAM2 generates object masks using click points
2. **Stage 2**: MatAnyone refines masks for temporal consistency
3. **Stage 3**: Composite foreground with new background
### Model Loading
- SAM2 models downloaded from Hugging Face Hub automatically
- Supports small/base/large variants (small recommended for T4)
- MatAnyone loaded from official repository
### Performance Optimizations
- T4-specific optimizations (fp16, channels_last)
- Memory pruning during long video processing
- Automatic model unloading between stages
---
## 📝 License
MIT License - See LICENSE file for details. |