MogensR commited on
Commit
0ee293e
Β·
1 Parent(s): 9f26dc2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +114 -56
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
- title: BackgroundFX Pro - SAM2 Powered
3
- emoji: πŸŽ₯
4
  colorFrom: blue
5
  colorTo: purple
6
  sdk: gradio
@@ -12,85 +12,143 @@ suggested_hardware: t4-small
12
  suggested_storage: small
13
  ---
14
 
15
- # πŸŽ₯ BackgroundFX Pro - SAM2 Powered
16
 
17
- **Professional AI video background replacement with advanced segmentation**
18
 
19
- Upload your video and let SAM2 AI automatically detect and replace the background with precision. Optimized for Hugging Face Spaces with smart memory management and lazy loading.
20
 
21
- ## ✨ Features
22
 
23
- - **πŸ€– SAM2 Integration**: State-of-the-art segmentation with Meta's SAM2
24
- - **⚑ Smart Loading**: True lazy loading - models download only when needed
 
25
  - **🎨 Background Options**: 8 built-in presets + custom image upload
26
- - **πŸ”§ Advanced Settings**: Model size selection and edge smoothing
27
- - **πŸ’Ύ Memory Optimized**: Automatic cleanup and CUDA cache management
28
- - **πŸ“± Professional UI**: Clean, intuitive interface with real-time progress
29
 
30
  ## πŸš€ Quick Start
31
 
32
- 1. Upload a video (MP4, AVI, MOV, MKV, WebM - max 5 minutes)
33
- 2. Choose a background preset or upload custom image
34
- 3. Select AI model size (Tiny/Small/Base)
35
- 4. Click "Replace Background" and wait for processing
36
- 5. Download your professional video!
37
 
38
- ## πŸ’‘ Pro Tips
39
 
40
- - **Best results**: Clear subject separation from background
41
- - **Lighting**: Even lighting works best for accurate segmentation
42
- - **Movement**: Minimal camera shake recommended
43
- - **Processing time**: ~30-60 seconds per minute of video
44
- - **GPU acceleration**: Automatically uses available GPU for faster processing
45
 
46
- ## πŸ”§ Technical Details
 
 
 
 
47
 
48
- - **Models**: SAM2 Tiny (38MB), Small (185MB), Base (320MB)
49
- - **Formats**: Supports all major video formats
50
- - **Resolution**: Up to 1920x1080 (Full HD)
 
 
 
 
 
 
 
 
 
 
 
 
51
  - **Duration**: Max 5 minutes on free tier
52
- - **Memory**: True lazy loading with automatic cleanup
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
 
54
- ## 🎬 Use Cases
55
 
56
- - **Content Creation**: Remove messy backgrounds for professional videos
57
- - **Virtual Meetings**: Create custom backgrounds for video calls
58
- - **Education**: Clean backgrounds for instructional videos
59
- - **Social Media**: Eye-catching backgrounds for posts and stories
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
 
61
  ## πŸ—οΈ Built With
62
 
63
- - [SAM2](https://github.com/facebookresearch/segment-anything-2) - Meta's Segment Anything Model 2
64
- - [Gradio](https://gradio.app/) - Machine learning web interface framework
65
- - [OpenCV](https://opencv.org/) - Computer vision library
66
- - [PyTorch](https://pytorch.org/) - Deep learning framework
 
67
 
68
  ## πŸ“‹ System Requirements
69
 
70
  - **Recommended**: GPU-enabled Space (T4-small or better)
71
- - **Minimum**: CPU-only mode supported but slower
72
  - **Memory**: Automatic management with CUDA optimization
73
- - **Storage**: No persistent storage needed (lazy loading)
74
 
75
- ## 🎯 Background Presets
76
 
77
- Choose from 8 beautiful presets:
78
- - **Ocean Blue** - Professional gradient
79
- - **Sunset Orange** - Warm and vibrant
80
- - **Forest Green** - Natural and calm
81
- - **Purple Haze** - Creative and modern
82
- - **Pure White** - Clean and minimal
83
- - **Pure Black** - Dramatic effect
84
- - **Chroma Green** - For further editing
85
- - **Chroma Blue** - Alternative chroma key
86
 
87
- ## ⚑ Performance Guide
 
 
 
 
 
 
 
 
88
 
89
- | Hardware | Processing Speed | Best Model | Concurrent Users |
90
- |----------|------------------|------------|------------------|
91
- | CPU | 2-3 min/video min | Tiny | 1 |
92
- | T4-small | 30-60s/video min | Small | 1-2 |
93
- | T4-medium | 20-40s/video min | Base | 2-3 |
94
- | A10G+ | 15-30s/video min | Base | 3-5 |
95
 
96
- Check out the [configuration reference](https://huggingface.co/docs/hub/spaces-config-reference) for more details.
 
1
  ---
2
+ title: BackgroundFX Pro - SAM2 + MatAnyone
3
+ emoji: 🎬
4
  colorFrom: blue
5
  colorTo: purple
6
  sdk: gradio
 
12
  suggested_storage: small
13
  ---
14
 
15
+ # 🎬 BackgroundFX Pro - SAM2 + MatAnyone
16
 
17
+ **Professional AI video background replacement with state-of-the-art segmentation and alpha matting**
18
 
19
+ Powered by **SAM2** (Meta's Segment Anything Model 2) and **MatAnyone** (CVPR 2025) for Hollywood-quality video background replacement with professional alpha matting.
20
 
21
+ ## ✨ Key Features
22
 
23
+ - **πŸ€– SAM2 Integration**: State-of-the-art object segmentation from Meta
24
+ - **🎭 MatAnyone Professional Matting**: CVPR 2025 video matting with memory propagation
25
+ - **⚑ Smart Pipeline**: SAM2 segmentation β†’ MatAnyone refinement β†’ Perfect compositing
26
  - **🎨 Background Options**: 8 built-in presets + custom image upload
27
+ - **πŸ’Ύ Memory Optimized**: Lazy loading with automatic cleanup
28
+ - **πŸ“± Professional UI**: Clean interface with real-time progress tracking
 
29
 
30
  ## πŸš€ Quick Start
31
 
32
+ 1. **Upload Video**: Drag & drop your video (MP4, AVI, MOV, MKV, WebM - max 5 minutes)
33
+ 2. **Choose Background**: Select from presets or upload custom image
34
+ 3. **Select Quality**: Choose SAM2 model size and MatAnyone settings
35
+ 4. **Process**: Click "Create Professional Video" and wait for magic
36
+ 5. **Download**: Get your Hollywood-quality result!
37
 
38
+ ## 🎯 Professional Pipeline
39
 
40
+ ### **Stage 1: SAM2 Segmentation**
41
+ - Meta's Segment Anything Model 2 detects and segments the main subject
42
+ - Multiple model sizes: Tiny (38MB), Small (185MB), Base (320MB)
43
+ - Smart point selection for optimal segmentation
 
44
 
45
+ ### **Stage 2: MatAnyone Alpha Matting**
46
+ - CVPR 2025 state-of-the-art video matting
47
+ - Memory-based propagation for stable boundaries
48
+ - Professional hair and edge detail preservation
49
+ - Target-assigned matting using SAM2 masks
50
 
51
+ ### **Stage 3: Professional Compositing**
52
+ - Seamless background replacement
53
+ - Advanced edge smoothing and blending
54
+ - Cinema-quality output
55
+
56
+ ## πŸ”§ Technical Specifications
57
+
58
+ ### **AI Models**
59
+ - **SAM2**: Meta's Segment Anything Model 2 for object detection
60
+ - **MatAnyone**: CVPR 2025 video matting with consistent memory propagation
61
+ - **Processing**: Lazy loading, CUDA optimization, memory management
62
+
63
+ ### **Supported Formats**
64
+ - **Input**: MP4, AVI, MOV, MKV, WebM
65
+ - **Resolution**: Up to 1920x1080 (Full HD)
66
  - **Duration**: Max 5 minutes on free tier
67
+ - **Output**: High-quality MP4 with professional alpha compositing
68
+
69
+ ## ⚑ Performance Guide
70
+
71
+ | Hardware | SAM2 Only | SAM2 + MatAnyone | Recommended Use |
72
+ |----------|-----------|------------------|-----------------|
73
+ | CPU | 3-5 min/video min | 6-10 min/video min | Quick tests only |
74
+ | T4-small | 45-90s/video min | 90-180s/video min | ⭐ **Recommended** |
75
+ | T4-medium+ | 30-60s/video min | 60-120s/video min | **Professional** |
76
+
77
+ ## 🎨 Background Presets
78
+
79
+ Choose from 8 professional presets:
80
+ - **🌊 Ocean Blue** - Clean professional gradient
81
+ - **πŸŒ… Sunset Orange** - Warm and vibrant atmosphere
82
+ - **🌲 Forest Green** - Natural and calming backdrop
83
+ - **πŸ’œ Purple Haze** - Creative and modern aesthetic
84
+ - **βšͺ Pure White** - Minimal and clean
85
+ - **⚫ Pure Black** - Dramatic high-contrast
86
+ - **πŸ’š Chroma Green** - For further video editing
87
+ - **πŸ’™ Chroma Blue** - Alternative chroma key
88
 
89
+ ## 🎬 Professional Use Cases
90
 
91
+ - **🎯 Content Creation**: Remove distracting backgrounds for professional videos
92
+ - **πŸ“Ή Virtual Production**: Custom backgrounds for video calls and streaming
93
+ - **πŸŽ“ Education**: Clean, professional backgrounds for instructional content
94
+ - **πŸ“± Social Media**: Eye-catching backgrounds that increase engagement
95
+ - **πŸŽͺ Entertainment**: Creative backgrounds for artistic and commercial projects
96
+
97
+ ## πŸ› οΈ Advanced Settings
98
+
99
+ ### **SAM2 Model Selection**
100
+ - **Tiny**: Fastest processing, good quality (recommended for quick previews)
101
+ - **Small**: Balanced speed and quality (recommended for most use cases)
102
+ - **Base**: Best quality, slower processing (recommended for final videos)
103
+
104
+ ### **MatAnyone Options**
105
+ - **Professional Mode**: Full MatAnyone processing for cinema-quality results
106
+ - **Fast Mode**: SAM2-only processing for quick turnaround
107
+
108
+ ### **Edge Enhancement**
109
+ - **Edge Smoothing**: 0-5 scale for softer subject boundaries
110
+ - **Memory Propagation**: Automatic temporal consistency across frames
111
 
112
  ## πŸ—οΈ Built With
113
 
114
+ - [**SAM2**](https://github.com/facebookresearch/segment-anything-2) - Meta's Segment Anything Model 2
115
+ - [**MatAnyone**](https://github.com/pq-yang/MatAnyone) - CVPR 2025 Video Matting Framework
116
+ - [**Gradio**](https://gradio.app/) - Machine learning web interface framework
117
+ - [**OpenCV**](https://opencv.org/) - Computer vision and video processing
118
+ - [**PyTorch**](https://pytorch.org/) - Deep learning framework
119
 
120
  ## πŸ“‹ System Requirements
121
 
122
  - **Recommended**: GPU-enabled Space (T4-small or better)
123
+ - **Minimum**: CPU-only mode supported but significantly slower
124
  - **Memory**: Automatic management with CUDA optimization
125
+ - **Storage**: No persistent storage needed (true lazy loading)
126
 
127
+ ## πŸ”¬ Research Citations
128
 
129
+ ```bibtex
130
+ @InProceedings{yang2025matanyone,
131
+ title = {{MatAnyone}: Stable Video Matting with Consistent Memory Propagation},
132
+ author = {Yang, Peiqing and Zhou, Shangchen and Zhao, Jixin and Tao, Qingyi and Loy, Chen Change},
133
+ booktitle = {CVPR},
134
+ year = {2025}
135
+ }
 
 
136
 
137
+ @article{ravi2024sam2,
138
+ title = {SAM 2: Segment Anything in Images and Videos},
139
+ author = {Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and RΓ€dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and DollΓ‘r, Piotr and Feichtenhofer, Christoph},
140
+ journal = {arXiv preprint arXiv:2408.00714},
141
+ year = {2024}
142
+ }
143
+ ```
144
+
145
+ ## πŸ”— Links
146
 
147
+ - **Project Website**: [BackgroundFX Pro Demo](https://huggingface.co/spaces/your-username/backgroundfx-pro)
148
+ - **SAM2 Repository**: [facebookresearch/segment-anything-2](https://github.com/facebookresearch/segment-anything-2)
149
+ - **MatAnyone Repository**: [pq-yang/MatAnyone](https://github.com/pq-yang/MatAnyone)
150
+ - **Configuration Reference**: [HF Spaces Config](https://huggingface.co/docs/hub/spaces-config-reference)
151
+
152
+ ---
153
 
154
+ **Ready to create professional videos with AI?** Upload your video and experience the future of background replacement! πŸš€