|
|
--- |
|
|
title: BackgroundFX Pro - SAM2 + MatAnyone |
|
|
emoji: 🎬 |
|
|
colorFrom: blue |
|
|
colorTo: purple |
|
|
sdk: gradio |
|
|
sdk_version: 5.42.0 |
|
|
app_file: app.py |
|
|
pinned: false |
|
|
license: mit |
|
|
suggested_hardware: t4-small |
|
|
suggested_storage: small |
|
|
--- |
|
|
|
|
|
# 🎬 BackgroundFX Pro - SAM2 + MatAnyone |
|
|
|
|
|
**Professional AI video background replacement with state-of-the-art segmentation and alpha matting** |
|
|
|
|
|
Powered by **SAM2** (Meta's Segment Anything Model 2) and **MatAnyone** (CVPR 2025) for Hollywood-quality video background replacement with professional alpha matting. |
|
|
|
|
|
## ✨ Key Features |
|
|
|
|
|
- **🤖 SAM2 Integration**: State-of-the-art object segmentation from Meta |
|
|
- **🎭 MatAnyone Professional Matting**: CVPR 2025 video matting with memory propagation |
|
|
- **⚡ Smart Pipeline**: SAM2 segmentation → MatAnyone refinement → Perfect compositing |
|
|
- **🎨 Background Options**: 8 built-in presets + custom image upload |
|
|
- **💾 Memory Optimized**: Lazy loading with automatic cleanup |
|
|
- **📱 Professional UI**: Clean interface with real-time progress tracking |
|
|
|
|
|
## 🚀 Quick Start |
|
|
|
|
|
1. **Upload Video**: Drag & drop your video (MP4, AVI, MOV, MKV, WebM - max 5 minutes) |
|
|
2. **Choose Background**: Select from presets or upload custom image |
|
|
3. **Select Quality**: Choose SAM2 model size and MatAnyone settings |
|
|
4. **Process**: Click "Create Professional Video" and wait for magic |
|
|
5. **Download**: Get your Hollywood-quality result! |
|
|
|
|
|
## 🎯 Professional Pipeline |
|
|
|
|
|
### **Stage 1: SAM2 Segmentation** |
|
|
- Meta's Segment Anything Model 2 detects and segments the main subject |
|
|
- Multiple model sizes: Tiny (38MB), Small (185MB), Base (320MB) |
|
|
- Smart point selection for optimal segmentation |
|
|
|
|
|
### **Stage 2: MatAnyone Alpha Matting** |
|
|
- CVPR 2025 state-of-the-art video matting |
|
|
- Memory-based propagation for stable boundaries |
|
|
- Professional hair and edge detail preservation |
|
|
- Target-assigned matting using SAM2 masks |
|
|
|
|
|
### **Stage 3: Professional Compositing** |
|
|
- Seamless background replacement |
|
|
- Advanced edge smoothing and blending |
|
|
- Cinema-quality output |
|
|
|
|
|
## 🔧 Technical Specifications |
|
|
|
|
|
### **AI Models** |
|
|
- **SAM2**: Meta's Segment Anything Model 2 for object detection |
|
|
- **MatAnyone**: CVPR 2025 video matting with consistent memory propagation |
|
|
- **Processing**: Lazy loading, CUDA optimization, memory management |
|
|
|
|
|
### **Supported Formats** |
|
|
- **Input**: MP4, AVI, MOV, MKV, WebM |
|
|
- **Resolution**: Up to 1920x1080 (Full HD) |
|
|
- **Duration**: Max 5 minutes on free tier |
|
|
- **Output**: High-quality MP4 with professional alpha compositing |
|
|
|
|
|
## ⚡ Performance Guide |
|
|
|
|
|
| Hardware | SAM2 Only | SAM2 + MatAnyone | Recommended Use | |
|
|
|----------|-----------|------------------|-----------------| |
|
|
| CPU | 3-5 min/video min | 6-10 min/video min | Quick tests only | |
|
|
| T4-small | 45-90s/video min | 90-180s/video min | ⭐ **Recommended** | |
|
|
| T4-medium+ | 30-60s/video min | 60-120s/video min | **Professional** | |
|
|
|
|
|
## 🎨 Background Presets |
|
|
|
|
|
Choose from 8 professional presets: |
|
|
- **🌊 Ocean Blue** - Clean professional gradient |
|
|
- **🌅 Sunset Orange** - Warm and vibrant atmosphere |
|
|
- **🌲 Forest Green** - Natural and calming backdrop |
|
|
- **💜 Purple Haze** - Creative and modern aesthetic |
|
|
- **⚪ Pure White** - Minimal and clean |
|
|
- **⚫ Pure Black** - Dramatic high-contrast |
|
|
- **💚 Chroma Green** - For further video editing |
|
|
- **💙 Chroma Blue** - Alternative chroma key |
|
|
|
|
|
## 🎬 Professional Use Cases |
|
|
|
|
|
- **🎯 Content Creation**: Remove distracting backgrounds for professional videos |
|
|
- **📹 Virtual Production**: Custom backgrounds for video calls and streaming |
|
|
- **🎓 Education**: Clean, professional backgrounds for instructional content |
|
|
- **📱 Social Media**: Eye-catching backgrounds that increase engagement |
|
|
- **🎪 Entertainment**: Creative backgrounds for artistic and commercial projects |
|
|
|
|
|
## 🛠️ Advanced Settings |
|
|
|
|
|
### **SAM2 Model Selection** |
|
|
- **Tiny**: Fastest processing, good quality (recommended for quick previews) |
|
|
- **Small**: Balanced speed and quality (recommended for most use cases) |
|
|
- **Base**: Best quality, slower processing (recommended for final videos) |
|
|
|
|
|
### **MatAnyone Options** |
|
|
- **Professional Mode**: Full MatAnyone processing for cinema-quality results |
|
|
- **Fast Mode**: SAM2-only processing for quick turnaround |
|
|
|
|
|
### **Edge Enhancement** |
|
|
- **Edge Smoothing**: 0-5 scale for softer subject boundaries |
|
|
- **Memory Propagation**: Automatic temporal consistency across frames |
|
|
|
|
|
## 🏗️ Built With |
|
|
|
|
|
- [**SAM2**](https://github.com/facebookresearch/segment-anything-2) - Meta's Segment Anything Model 2 |
|
|
- [**MatAnyone**](https://github.com/pq-yang/MatAnyone) - CVPR 2025 Video Matting Framework |
|
|
- [**Gradio**](https://gradio.app/) - Machine learning web interface framework |
|
|
- [**OpenCV**](https://opencv.org/) - Computer vision and video processing |
|
|
- [**PyTorch**](https://pytorch.org/) - Deep learning framework |
|
|
|
|
|
## 📋 System Requirements |
|
|
|
|
|
- **Recommended**: GPU-enabled Space (T4-small or better) |
|
|
- **Minimum**: CPU-only mode supported but significantly slower |
|
|
- **Memory**: Automatic management with CUDA optimization |
|
|
- **Storage**: No persistent storage needed (true lazy loading) |
|
|
|
|
|
## 🔬 Research Citations |
|
|
|
|
|
```bibtex |
|
|
@InProceedings{yang2025matanyone, |
|
|
title = {{MatAnyone}: Stable Video Matting with Consistent Memory Propagation}, |
|
|
author = {Yang, Peiqing and Zhou, Shangchen and Zhao, Jixin and Tao, Qingyi and Loy, Chen Change}, |
|
|
booktitle = {CVPR}, |
|
|
year = {2025} |
|
|
} |
|
|
|
|
|
@article{ravi2024sam2, |
|
|
title = {SAM 2: Segment Anything in Images and Videos}, |
|
|
author = {Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and Rädle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Dollár, Piotr and Feichtenhofer, Christoph}, |
|
|
journal = {arXiv preprint arXiv:2408.00714}, |
|
|
year = {2024} |
|
|
} |
|
|
``` |
|
|
|
|
|
## 🔗 Links |
|
|
|
|
|
- **Project Website**: [BackgroundFX Pro Demo](https://huggingface.co/spaces/your-username/backgroundfx-pro) |
|
|
- **SAM2 Repository**: [facebookresearch/segment-anything-2](https://github.com/facebookresearch/segment-anything-2) |
|
|
- **MatAnyone Repository**: [pq-yang/MatAnyone](https://github.com/pq-yang/MatAnyone) |
|
|
- **Configuration Reference**: [HF Spaces Config](https://huggingface.co/docs/hub/spaces-config-reference) |
|
|
|
|
|
--- |
|
|
|
|
|
**Ready to create professional videos with AI?** Upload your video and experience the future of background replacement! 🚀 |
