Update README.md
Browse files
README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
-
title: BackgroundFX Pro - SAM2
|
| 3 |
-
emoji:
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
| 6 |
sdk: gradio
|
|
@@ -12,85 +12,143 @@ suggested_hardware: t4-small
|
|
| 12 |
suggested_storage: small
|
| 13 |
---
|
| 14 |
|
| 15 |
-
#
|
| 16 |
|
| 17 |
-
**Professional AI video background replacement with
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
## β¨ Features
|
| 22 |
|
| 23 |
-
- **π€ SAM2 Integration**: State-of-the-art segmentation
|
| 24 |
-
-
|
|
|
|
| 25 |
- **π¨ Background Options**: 8 built-in presets + custom image upload
|
| 26 |
-
-
|
| 27 |
-
-
|
| 28 |
-
- **π± Professional UI**: Clean, intuitive interface with real-time progress
|
| 29 |
|
| 30 |
## π Quick Start
|
| 31 |
|
| 32 |
-
1. Upload
|
| 33 |
-
2. Choose
|
| 34 |
-
3. Select
|
| 35 |
-
4. Click "
|
| 36 |
-
5. Download your
|
| 37 |
|
| 38 |
-
##
|
| 39 |
|
| 40 |
-
|
| 41 |
-
-
|
| 42 |
-
-
|
| 43 |
-
-
|
| 44 |
-
- **GPU acceleration**: Automatically uses available GPU for faster processing
|
| 45 |
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
-
|
| 49 |
-
-
|
| 50 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
- **Duration**: Max 5 minutes on free tier
|
| 52 |
-
- **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
-
## π¬ Use Cases
|
| 55 |
|
| 56 |
-
-
|
| 57 |
-
-
|
| 58 |
-
-
|
| 59 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
## ποΈ Built With
|
| 62 |
|
| 63 |
-
- [SAM2](https://github.com/facebookresearch/segment-anything-2) - Meta's Segment Anything Model 2
|
| 64 |
-
- [
|
| 65 |
-
- [
|
| 66 |
-
- [
|
|
|
|
| 67 |
|
| 68 |
## π System Requirements
|
| 69 |
|
| 70 |
- **Recommended**: GPU-enabled Space (T4-small or better)
|
| 71 |
-
- **Minimum**: CPU-only mode supported but slower
|
| 72 |
- **Memory**: Automatic management with CUDA optimization
|
| 73 |
-
- **Storage**: No persistent storage needed (lazy loading)
|
| 74 |
|
| 75 |
-
##
|
| 76 |
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
- **Chroma Green** - For further editing
|
| 85 |
-
- **Chroma Blue** - Alternative chroma key
|
| 86 |
|
| 87 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
|
| 96 |
-
|
|
|
|
| 1 |
---
|
| 2 |
+
title: BackgroundFX Pro - SAM2 + MatAnyone
|
| 3 |
+
emoji: π¬
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
| 6 |
sdk: gradio
|
|
|
|
| 12 |
suggested_storage: small
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# π¬ BackgroundFX Pro - SAM2 + MatAnyone
|
| 16 |
|
| 17 |
+
**Professional AI video background replacement with state-of-the-art segmentation and alpha matting**
|
| 18 |
|
| 19 |
+
Powered by **SAM2** (Meta's Segment Anything Model 2) and **MatAnyone** (CVPR 2025) for Hollywood-quality video background replacement with professional alpha matting.
|
| 20 |
|
| 21 |
+
## β¨ Key Features
|
| 22 |
|
| 23 |
+
- **π€ SAM2 Integration**: State-of-the-art object segmentation from Meta
|
| 24 |
+
- **π MatAnyone Professional Matting**: CVPR 2025 video matting with memory propagation
|
| 25 |
+
- **β‘ Smart Pipeline**: SAM2 segmentation β MatAnyone refinement β Perfect compositing
|
| 26 |
- **π¨ Background Options**: 8 built-in presets + custom image upload
|
| 27 |
+
- **πΎ Memory Optimized**: Lazy loading with automatic cleanup
|
| 28 |
+
- **π± Professional UI**: Clean interface with real-time progress tracking
|
|
|
|
| 29 |
|
| 30 |
## π Quick Start
|
| 31 |
|
| 32 |
+
1. **Upload Video**: Drag & drop your video (MP4, AVI, MOV, MKV, WebM - max 5 minutes)
|
| 33 |
+
2. **Choose Background**: Select from presets or upload custom image
|
| 34 |
+
3. **Select Quality**: Choose SAM2 model size and MatAnyone settings
|
| 35 |
+
4. **Process**: Click "Create Professional Video" and wait for magic
|
| 36 |
+
5. **Download**: Get your Hollywood-quality result!
|
| 37 |
|
| 38 |
+
## π― Professional Pipeline
|
| 39 |
|
| 40 |
+
### **Stage 1: SAM2 Segmentation**
|
| 41 |
+
- Meta's Segment Anything Model 2 detects and segments the main subject
|
| 42 |
+
- Multiple model sizes: Tiny (38MB), Small (185MB), Base (320MB)
|
| 43 |
+
- Smart point selection for optimal segmentation
|
|
|
|
| 44 |
|
| 45 |
+
### **Stage 2: MatAnyone Alpha Matting**
|
| 46 |
+
- CVPR 2025 state-of-the-art video matting
|
| 47 |
+
- Memory-based propagation for stable boundaries
|
| 48 |
+
- Professional hair and edge detail preservation
|
| 49 |
+
- Target-assigned matting using SAM2 masks
|
| 50 |
|
| 51 |
+
### **Stage 3: Professional Compositing**
|
| 52 |
+
- Seamless background replacement
|
| 53 |
+
- Advanced edge smoothing and blending
|
| 54 |
+
- Cinema-quality output
|
| 55 |
+
|
| 56 |
+
## π§ Technical Specifications
|
| 57 |
+
|
| 58 |
+
### **AI Models**
|
| 59 |
+
- **SAM2**: Meta's Segment Anything Model 2 for object detection
|
| 60 |
+
- **MatAnyone**: CVPR 2025 video matting with consistent memory propagation
|
| 61 |
+
- **Processing**: Lazy loading, CUDA optimization, memory management
|
| 62 |
+
|
| 63 |
+
### **Supported Formats**
|
| 64 |
+
- **Input**: MP4, AVI, MOV, MKV, WebM
|
| 65 |
+
- **Resolution**: Up to 1920x1080 (Full HD)
|
| 66 |
- **Duration**: Max 5 minutes on free tier
|
| 67 |
+
- **Output**: High-quality MP4 with professional alpha compositing
|
| 68 |
+
|
| 69 |
+
## β‘ Performance Guide
|
| 70 |
+
|
| 71 |
+
| Hardware | SAM2 Only | SAM2 + MatAnyone | Recommended Use |
|
| 72 |
+
|----------|-----------|------------------|-----------------|
|
| 73 |
+
| CPU | 3-5 min/video min | 6-10 min/video min | Quick tests only |
|
| 74 |
+
| T4-small | 45-90s/video min | 90-180s/video min | β **Recommended** |
|
| 75 |
+
| T4-medium+ | 30-60s/video min | 60-120s/video min | **Professional** |
|
| 76 |
+
|
| 77 |
+
## π¨ Background Presets
|
| 78 |
+
|
| 79 |
+
Choose from 8 professional presets:
|
| 80 |
+
- **π Ocean Blue** - Clean professional gradient
|
| 81 |
+
- **π
Sunset Orange** - Warm and vibrant atmosphere
|
| 82 |
+
- **π² Forest Green** - Natural and calming backdrop
|
| 83 |
+
- **π Purple Haze** - Creative and modern aesthetic
|
| 84 |
+
- **βͺ Pure White** - Minimal and clean
|
| 85 |
+
- **β« Pure Black** - Dramatic high-contrast
|
| 86 |
+
- **π Chroma Green** - For further video editing
|
| 87 |
+
- **π Chroma Blue** - Alternative chroma key
|
| 88 |
|
| 89 |
+
## π¬ Professional Use Cases
|
| 90 |
|
| 91 |
+
- **π― Content Creation**: Remove distracting backgrounds for professional videos
|
| 92 |
+
- **πΉ Virtual Production**: Custom backgrounds for video calls and streaming
|
| 93 |
+
- **π Education**: Clean, professional backgrounds for instructional content
|
| 94 |
+
- **π± Social Media**: Eye-catching backgrounds that increase engagement
|
| 95 |
+
- **πͺ Entertainment**: Creative backgrounds for artistic and commercial projects
|
| 96 |
+
|
| 97 |
+
## π οΈ Advanced Settings
|
| 98 |
+
|
| 99 |
+
### **SAM2 Model Selection**
|
| 100 |
+
- **Tiny**: Fastest processing, good quality (recommended for quick previews)
|
| 101 |
+
- **Small**: Balanced speed and quality (recommended for most use cases)
|
| 102 |
+
- **Base**: Best quality, slower processing (recommended for final videos)
|
| 103 |
+
|
| 104 |
+
### **MatAnyone Options**
|
| 105 |
+
- **Professional Mode**: Full MatAnyone processing for cinema-quality results
|
| 106 |
+
- **Fast Mode**: SAM2-only processing for quick turnaround
|
| 107 |
+
|
| 108 |
+
### **Edge Enhancement**
|
| 109 |
+
- **Edge Smoothing**: 0-5 scale for softer subject boundaries
|
| 110 |
+
- **Memory Propagation**: Automatic temporal consistency across frames
|
| 111 |
|
| 112 |
## ποΈ Built With
|
| 113 |
|
| 114 |
+
- [**SAM2**](https://github.com/facebookresearch/segment-anything-2) - Meta's Segment Anything Model 2
|
| 115 |
+
- [**MatAnyone**](https://github.com/pq-yang/MatAnyone) - CVPR 2025 Video Matting Framework
|
| 116 |
+
- [**Gradio**](https://gradio.app/) - Machine learning web interface framework
|
| 117 |
+
- [**OpenCV**](https://opencv.org/) - Computer vision and video processing
|
| 118 |
+
- [**PyTorch**](https://pytorch.org/) - Deep learning framework
|
| 119 |
|
| 120 |
## π System Requirements
|
| 121 |
|
| 122 |
- **Recommended**: GPU-enabled Space (T4-small or better)
|
| 123 |
+
- **Minimum**: CPU-only mode supported but significantly slower
|
| 124 |
- **Memory**: Automatic management with CUDA optimization
|
| 125 |
+
- **Storage**: No persistent storage needed (true lazy loading)
|
| 126 |
|
| 127 |
+
## π¬ Research Citations
|
| 128 |
|
| 129 |
+
```bibtex
|
| 130 |
+
@InProceedings{yang2025matanyone,
|
| 131 |
+
title = {{MatAnyone}: Stable Video Matting with Consistent Memory Propagation},
|
| 132 |
+
author = {Yang, Peiqing and Zhou, Shangchen and Zhao, Jixin and Tao, Qingyi and Loy, Chen Change},
|
| 133 |
+
booktitle = {CVPR},
|
| 134 |
+
year = {2025}
|
| 135 |
+
}
|
|
|
|
|
|
|
| 136 |
|
| 137 |
+
@article{ravi2024sam2,
|
| 138 |
+
title = {SAM 2: Segment Anything in Images and Videos},
|
| 139 |
+
author = {Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and RΓ€dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and DollΓ‘r, Piotr and Feichtenhofer, Christoph},
|
| 140 |
+
journal = {arXiv preprint arXiv:2408.00714},
|
| 141 |
+
year = {2024}
|
| 142 |
+
}
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
## π Links
|
| 146 |
|
| 147 |
+
- **Project Website**: [BackgroundFX Pro Demo](https://huggingface.co/spaces/your-username/backgroundfx-pro)
|
| 148 |
+
- **SAM2 Repository**: [facebookresearch/segment-anything-2](https://github.com/facebookresearch/segment-anything-2)
|
| 149 |
+
- **MatAnyone Repository**: [pq-yang/MatAnyone](https://github.com/pq-yang/MatAnyone)
|
| 150 |
+
- **Configuration Reference**: [HF Spaces Config](https://huggingface.co/docs/hub/spaces-config-reference)
|
| 151 |
+
|
| 152 |
+
---
|
| 153 |
|
| 154 |
+
**Ready to create professional videos with AI?** Upload your video and experience the future of background replacement! π
|