Update README.md
Browse files
README.md
CHANGED
|
@@ -1,73 +1,105 @@
|
|
| 1 |
-
|
| 2 |
-
title: Video Background Replacement
|
| 3 |
-
emoji: πΉ
|
| 4 |
-
colorFrom: blue
|
| 5 |
-
colorTo: purple
|
| 6 |
-
sdk: streamlit
|
| 7 |
-
sdk_version: 1.47.1
|
| 8 |
-
app_file: app.py
|
| 9 |
-
pinned: false
|
| 10 |
-
license: mit
|
| 11 |
-
---
|
| 12 |
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
A simple Streamlit app that replaces video backgrounds using MatAnyone AI with direct MyAvatar integration.
|
| 16 |
-
|
| 17 |
-
## Features
|
| 18 |
-
- Upload video files (MP4, AVI, MOV, MKV)
|
| 19 |
-
- Upload background images (PNG, JPG, JPEG)
|
| 20 |
-
- AI-powered background replacement using MatAnyone
|
| 21 |
-
- Preserves original audio from input videos
|
| 22 |
-
- Web-optimized MP4 output with instant streaming capability
|
| 23 |
-
- Download processed video
|
| 24 |
-
- πΎ Save to MyAvatar - Direct integration with MyAvatar library
|
| 25 |
-
|
| 26 |
-
## Usage
|
| 27 |
-
1. Upload your video file
|
| 28 |
-
2. Upload your desired background image
|
| 29 |
-
3. Click "πΉ PROCESS VIDEO"
|
| 30 |
-
4. Choose your action:
|
| 31 |
-
- β¬οΈ Download - Save to your device
|
| 32 |
-
- πΎ Save to My Videos - Add directly to your MyAvatar library
|
| 33 |
-
|
| 34 |
-
## π¬ Video Output Quality
|
| 35 |
-
This app automatically creates professional-quality, web-compatible MP4 videos:
|
| 36 |
-
|
| 37 |
-
- **H.264 Encoding**: Industry-standard codec for maximum browser compatibility
|
| 38 |
-
- **Audio Preservation**: Automatically extracts and maintains original audio tracks
|
| 39 |
-
- **Moov Atom Optimization**: Fixes MP4 metadata positioning using qtfaststart for instant web streaming
|
| 40 |
-
- **Smart Fallbacks**: Gracefully handles different system configurations (FFmpeg/moviepy available or not)
|
| 41 |
-
|
| 42 |
-
### Why This Matters
|
| 43 |
-
Input videos display perfectly in browsers, but AI-generated videos often have metadata and audio issues. Our solution ensures output videos have the same streaming compatibility and audio quality as professional camera recordings.
|
| 44 |
-
|
| 45 |
-
## π MyAvatar Integration
|
| 46 |
-
- **One-Click Save**: Processed videos can be saved directly to your MyAvatar library
|
| 47 |
-
- **No Manual Steps**: Skip downloading and re-uploading - save happens automatically
|
| 48 |
-
- **Seamless Workflow**: Process β Save β Use in other MyAvatar features
|
| 49 |
-
- **Library Management**: Videos appear in "My Videos" instantly
|
| 50 |
-
|
| 51 |
-
## Technology
|
| 52 |
-
- **MatAnyone**: Professional video matting AI model
|
| 53 |
-
- **Streamlit 1.47.1**: Simple web interface
|
| 54 |
-
- **GPU Accelerated**: Runs on NVIDIA hardware
|
| 55 |
-
- **qtfaststart**: Pure Python moov atom optimizer for web compatibility
|
| 56 |
-
- **moviepy**: Audio preservation and video editing capabilities
|
| 57 |
-
- **Multiple Codec Support**: H.264, XVID fallbacks for maximum compatibility
|
| 58 |
-
- **MyAvatar API**: Direct integration for seamless library management
|
| 59 |
|
| 60 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
-
##
|
| 63 |
|
| 64 |
-
**
|
| 65 |
-
**
|
|
|
|
|
|
|
|
|
|
| 66 |
|
| 67 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
|
| 69 |
-
|
| 70 |
-
- MatAnyone AI for ultimate quality segmentation
|
| 71 |
-
- MediaPipe fallback for good quality processing
|
| 72 |
-
- Advanced chroma key algorithm with edge smoothing
|
| 73 |
-
- Optimized for iframe embedding in MyAvatar
|
|
|
|
| 1 |
+
# πΉ Video Background Replacer
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
+
AI-powered video background replacement with automatic person segmentation and audio preservation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
+
## π Features
|
| 6 |
+
|
| 7 |
+
- **π€ AI-Powered Segmentation**: Uses MatAnyone from Hugging Face for high-quality person extraction
|
| 8 |
+
- **π Automatic Masking**: MediaPipe automatically detects and segments people - no manual masking needed
|
| 9 |
+
- **π΅ Audio Preservation**: Maintains original audio using FFmpeg with moviepy fallback
|
| 10 |
+
- **πΎ Direct Save**: Integrates with MyAvatar for easy video library management
|
| 11 |
+
- **π± Mobile Friendly**: Responsive design works on all devices
|
| 12 |
+
- **π¬ Professional Quality**: Optimized video output with proper encoding
|
| 13 |
+
|
| 14 |
+
## π οΈ How It Works
|
| 15 |
+
|
| 16 |
+
1. **Upload** your video and background image
|
| 17 |
+
2. **AI Processing**: MatAnyone extracts the person from the video
|
| 18 |
+
3. **Background Replacement**: Your new background is seamlessly applied
|
| 19 |
+
4. **Audio Preservation**: Original audio is maintained
|
| 20 |
+
5. **Download or Save**: Get your processed video or save to MyAvatar
|
| 21 |
+
|
| 22 |
+
## π― Technology Stack
|
| 23 |
+
|
| 24 |
+
- **Frontend**: Streamlit with custom CSS
|
| 25 |
+
- **AI Model**: MatAnyone (PeiqingYang/MatAnyone from Hugging Face)
|
| 26 |
+
- **Segmentation**: MediaPipe for automatic person detection
|
| 27 |
+
- **Video Processing**: OpenCV + FFmpeg + MoviePy
|
| 28 |
+
- **Audio**: FFmpeg with moviepy fallback for compatibility
|
| 29 |
+
|
| 30 |
+
## π Requirements
|
| 31 |
+
|
| 32 |
+
- Python 3.10+
|
| 33 |
+
- CUDA support (for GPU acceleration)
|
| 34 |
+
- FFmpeg (for audio processing)
|
| 35 |
+
|
| 36 |
+
## π§ Installation
|
| 37 |
+
|
| 38 |
+
The app automatically installs all dependencies from `requirements.txt`, including:
|
| 39 |
+
|
| 40 |
+
- **MatAnyone**: Loaded directly from Hugging Face Hub
|
| 41 |
+
- **MediaPipe**: For automatic segmentation
|
| 42 |
+
- **Streamlit**: Web interface
|
| 43 |
+
- **OpenCV**: Video processing
|
| 44 |
+
- **FFmpeg**: Audio handling
|
| 45 |
|
| 46 |
+
## π¨ Usage
|
| 47 |
|
| 48 |
+
1. **Upload Video**: Choose any MP4, AVI, or MOV file
|
| 49 |
+
2. **Upload Background**: Select a PNG, JPG, or JPEG image
|
| 50 |
+
3. **Click Process**: The AI handles everything automatically
|
| 51 |
+
4. **Download**: Get your processed video with preserved audio
|
| 52 |
+
5. **Save to MyAvatar**: Optional direct integration with MyAvatar library
|
| 53 |
|
| 54 |
+
## β¨ Key Improvements
|
| 55 |
+
|
| 56 |
+
### Automatic Segmentation
|
| 57 |
+
- No manual mask creation needed
|
| 58 |
+
- MediaPipe detects people automatically
|
| 59 |
+
- Works with any video containing people
|
| 60 |
+
|
| 61 |
+
### HuggingFace Integration
|
| 62 |
+
```python
|
| 63 |
+
processor = InferenceCore("PeiqingYang/MatAnyone")
|
| 64 |
+
```
|
| 65 |
+
- Direct model loading from HF Hub
|
| 66 |
+
- No manual model downloads
|
| 67 |
+
- Always latest version
|
| 68 |
+
|
| 69 |
+
### Audio Preservation
|
| 70 |
+
- Primary: FFmpeg for high-quality audio
|
| 71 |
+
- Fallback: MoviePy for compatibility
|
| 72 |
+
- Maintains original audio quality
|
| 73 |
+
|
| 74 |
+
### MyAvatar Integration
|
| 75 |
+
- Direct API connection
|
| 76 |
+
- Base64 encoding for secure transfer
|
| 77 |
+
- Automatic filename generation
|
| 78 |
+
|
| 79 |
+
## π¬ Supported Formats
|
| 80 |
+
|
| 81 |
+
**Input Video**: MP4, AVI, MOV
|
| 82 |
+
**Background Image**: PNG, JPG, JPEG
|
| 83 |
+
**Output**: MP4 with H.264 encoding
|
| 84 |
+
|
| 85 |
+
## π Perfect For
|
| 86 |
+
|
| 87 |
+
- **Content Creators**: Professional video backgrounds
|
| 88 |
+
- **Remote Work**: Clean video call backgrounds
|
| 89 |
+
- **Social Media**: Creative video content
|
| 90 |
+
- **Marketing**: Branded video backgrounds
|
| 91 |
+
- **Education**: Consistent presentation backgrounds
|
| 92 |
+
|
| 93 |
+
## π Privacy
|
| 94 |
+
|
| 95 |
+
- All processing happens on secure servers
|
| 96 |
+
- No data stored permanently
|
| 97 |
+
- Optional MyAvatar integration for your personal library
|
| 98 |
+
|
| 99 |
+
## π€ Support
|
| 100 |
+
|
| 101 |
+
Built with modern AI technology for reliable, high-quality results. Optimized for both desktop and mobile use.
|
| 102 |
+
|
| 103 |
+
---
|
| 104 |
|
| 105 |
+
*πΉ Powered by MatAnyone AI | Connected to MyAvatar*
|
|
|
|
|
|
|
|
|
|
|
|