Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,7 @@ colorFrom: indigo
|
|
| 5 |
colorTo: purple
|
| 6 |
sdk: docker
|
| 7 |
app_port: 7860
|
|
|
|
| 8 |
license: mit
|
| 9 |
tags:
|
| 10 |
- video
|
|
@@ -21,14 +22,15 @@ BackgroundFX Pro is a GPU-accelerated app for Hugging Face Spaces (Docker) that
|
|
| 21 |
- **SAM2** β high-quality object segmentation
|
| 22 |
- **MatAnyone** β temporal video matting for stable alpha over time
|
| 23 |
|
| 24 |
-
Built on: **CUDA 12.1.1**, **PyTorch 2.5.1 (cu121)**, **torchvision 0.20.1**, **
|
| 25 |
|
| 26 |
---
|
| 27 |
|
| 28 |
## β¨ Features
|
| 29 |
|
| 30 |
-
- Replace backgrounds with: **solid color**, **AI-generated** image (procedural), **custom uploaded image**, or **
|
| 31 |
- Optimized for **T4 GPUs** on Hugging Face
|
|
|
|
| 32 |
- Caching & logs stored in the repo volume:
|
| 33 |
- HF cache β `./.hf`
|
| 34 |
- Torch cache β `./.torch`
|
|
@@ -45,37 +47,37 @@ https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2
|
|
| 45 |
|
| 46 |
## π±οΈ How to Use
|
| 47 |
|
| 48 |
-
1. **Upload a video** (`.mp4`, `.
|
| 49 |
-
2. Choose a **Background Type**:
|
| 50 |
-
3. If
|
| 51 |
-
4. Click
|
| 52 |
-
5. Preview and
|
| 53 |
|
| 54 |
-
> Tip: Start with 720p/1080p on T4; 4K can exceed memory.
|
| 55 |
|
| 56 |
---
|
| 57 |
|
| 58 |
## ποΈ Project Structure (key files)
|
| 59 |
|
| 60 |
-
- `Dockerfile`
|
| 61 |
-
- `requirements.txt`
|
| 62 |
-
- `
|
| 63 |
-
- `
|
| 64 |
-
- `
|
| 65 |
-
- `
|
| 66 |
-
- `
|
| 67 |
-
- `
|
| 68 |
-
- `
|
| 69 |
-
- `data/` (created at runtime for logs/outputs)
|
| 70 |
-
- `tmp/` (created at runtime for jobs/temp files)
|
| 71 |
|
| 72 |
---
|
| 73 |
|
| 74 |
## βοΈ Runtime Notes
|
| 75 |
|
| 76 |
-
- Binds to `PORT` / `
|
| 77 |
-
-
|
| 78 |
-
-
|
|
|
|
|
|
|
| 79 |
|
| 80 |
---
|
| 81 |
|
|
@@ -92,3 +94,31 @@ docker build -t backgroundfx-pro .
|
|
| 92 |
|
| 93 |
# Run
|
| 94 |
docker run --gpus all -p 7860:7860 backgroundfx-pro
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
colorTo: purple
|
| 6 |
sdk: docker
|
| 7 |
app_port: 7860
|
| 8 |
+
sdk_version: 1.49.1
|
| 9 |
license: mit
|
| 10 |
tags:
|
| 11 |
- video
|
|
|
|
| 22 |
- **SAM2** β high-quality object segmentation
|
| 23 |
- **MatAnyone** β temporal video matting for stable alpha over time
|
| 24 |
|
| 25 |
+
Built on: **CUDA 12.1.1**, **PyTorch 2.5.1 (cu121)**, **torchvision 0.20.1**, **Streamlit 1.49.1**.
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
## β¨ Features
|
| 30 |
|
| 31 |
+
- Replace backgrounds with: **solid color**, **AI-generated** image (procedural), **custom uploaded image**, or **professional backgrounds**
|
| 32 |
- Optimized for **T4 GPUs** on Hugging Face
|
| 33 |
+
- Two-stage pipeline: SAM2 segmentation β MatAnyone refinement β compositing
|
| 34 |
- Caching & logs stored in the repo volume:
|
| 35 |
- HF cache β `./.hf`
|
| 36 |
- Torch cache β `./.torch`
|
|
|
|
| 47 |
|
| 48 |
## π±οΈ How to Use
|
| 49 |
|
| 50 |
+
1. **Upload a video** (`.mp4`, `.mov`, `.avi`, `.mkv`).
|
| 51 |
+
2. Choose a **Background Type**: Image, Color, Blur, Professional Backgrounds, or AI Generated.
|
| 52 |
+
3. If using custom background, upload your image or select from professional options.
|
| 53 |
+
4. Click **π Process Video**.
|
| 54 |
+
5. Preview and **πΎ Download Result**.
|
| 55 |
|
| 56 |
+
> Tip: Start with 720p/1080p on T4; 4K can exceed memory limits.
|
| 57 |
|
| 58 |
---
|
| 59 |
|
| 60 |
## ποΈ Project Structure (key files)
|
| 61 |
|
| 62 |
+
- `Dockerfile` β CUDA 12.1.1 + PyTorch 2.5.1 container
|
| 63 |
+
- `requirements.txt` β Python dependencies
|
| 64 |
+
- `app.py` β Main Streamlit application
|
| 65 |
+
- `integrated_pipeline.py` β Two-stage processing pipeline
|
| 66 |
+
- `models/sam2_loader.py` β SAM2 model loader with HF Hub integration
|
| 67 |
+
- `models/matanyone_loader.py` β MatAnyone model loader
|
| 68 |
+
- `utils/` β Utility functions
|
| 69 |
+
- `data/` β Created at runtime for logs/outputs
|
| 70 |
+
- `tmp/` β Created at runtime for processing jobs
|
|
|
|
|
|
|
| 71 |
|
| 72 |
---
|
| 73 |
|
| 74 |
## βοΈ Runtime Notes
|
| 75 |
|
| 76 |
+
- Binds to `PORT` / `STREAMLIT_SERVER_PORT` (defaults to **7860**)
|
| 77 |
+
- File upload limit: 200MB via `--server.maxUploadSize=200`
|
| 78 |
+
- CORS disabled for Docker compatibility: `--server.enableCORS=false`
|
| 79 |
+
- Memory management with automatic cleanup between stages
|
| 80 |
+
- If processing fails, check Space logs for detailed error information
|
| 81 |
|
| 82 |
---
|
| 83 |
|
|
|
|
| 94 |
|
| 95 |
# Run
|
| 96 |
docker run --gpus all -p 7860:7860 backgroundfx-pro
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
Access at: http://localhost:7860
|
| 100 |
+
|
| 101 |
+
---
|
| 102 |
+
|
| 103 |
+
## π§ Technical Details
|
| 104 |
+
|
| 105 |
+
### Pipeline Architecture
|
| 106 |
+
1. **Stage 1**: SAM2 generates object masks using click points
|
| 107 |
+
2. **Stage 2**: MatAnyone refines masks for temporal consistency
|
| 108 |
+
3. **Stage 3**: Composite foreground with new background
|
| 109 |
+
|
| 110 |
+
### Model Loading
|
| 111 |
+
- SAM2 models downloaded from Hugging Face Hub automatically
|
| 112 |
+
- Supports small/base/large variants (small recommended for T4)
|
| 113 |
+
- MatAnyone loaded from official repository
|
| 114 |
+
|
| 115 |
+
### Performance Optimizations
|
| 116 |
+
- T4-specific optimizations (fp16, channels_last)
|
| 117 |
+
- Memory pruning during long video processing
|
| 118 |
+
- Automatic model unloading between stages
|
| 119 |
+
|
| 120 |
+
---
|
| 121 |
+
|
| 122 |
+
## π License
|
| 123 |
+
|
| 124 |
+
MIT License - See LICENSE file for details.
|