MogensR commited on
Commit
e9f466f
Β·
1 Parent(s): ac197e4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +99 -67
README.md CHANGED
@@ -1,73 +1,105 @@
1
- ---
2
- title: Video Background Replacement
3
- emoji: 🍹
4
- colorFrom: blue
5
- colorTo: purple
6
- sdk: streamlit
7
- sdk_version: 1.47.1
8
- app_file: app.py
9
- pinned: false
10
- license: mit
11
- ---
12
 
13
- # Video Background Replacement
14
-
15
- A simple Streamlit app that replaces video backgrounds using MatAnyone AI with direct MyAvatar integration.
16
-
17
- ## Features
18
- - Upload video files (MP4, AVI, MOV, MKV)
19
- - Upload background images (PNG, JPG, JPEG)
20
- - AI-powered background replacement using MatAnyone
21
- - Preserves original audio from input videos
22
- - Web-optimized MP4 output with instant streaming capability
23
- - Download processed video
24
- - πŸ’Ύ Save to MyAvatar - Direct integration with MyAvatar library
25
-
26
- ## Usage
27
- 1. Upload your video file
28
- 2. Upload your desired background image
29
- 3. Click "🍹 PROCESS VIDEO"
30
- 4. Choose your action:
31
- - ⬇️ Download - Save to your device
32
- - πŸ’Ύ Save to My Videos - Add directly to your MyAvatar library
33
-
34
- ## 🎬 Video Output Quality
35
- This app automatically creates professional-quality, web-compatible MP4 videos:
36
-
37
- - **H.264 Encoding**: Industry-standard codec for maximum browser compatibility
38
- - **Audio Preservation**: Automatically extracts and maintains original audio tracks
39
- - **Moov Atom Optimization**: Fixes MP4 metadata positioning using qtfaststart for instant web streaming
40
- - **Smart Fallbacks**: Gracefully handles different system configurations (FFmpeg/moviepy available or not)
41
-
42
- ### Why This Matters
43
- Input videos display perfectly in browsers, but AI-generated videos often have metadata and audio issues. Our solution ensures output videos have the same streaming compatibility and audio quality as professional camera recordings.
44
-
45
- ## πŸ”— MyAvatar Integration
46
- - **One-Click Save**: Processed videos can be saved directly to your MyAvatar library
47
- - **No Manual Steps**: Skip downloading and re-uploading - save happens automatically
48
- - **Seamless Workflow**: Process β†’ Save β†’ Use in other MyAvatar features
49
- - **Library Management**: Videos appear in "My Videos" instantly
50
-
51
- ## Technology
52
- - **MatAnyone**: Professional video matting AI model
53
- - **Streamlit 1.47.1**: Simple web interface
54
- - **GPU Accelerated**: Runs on NVIDIA hardware
55
- - **qtfaststart**: Pure Python moov atom optimizer for web compatibility
56
- - **moviepy**: Audio preservation and video editing capabilities
57
- - **Multiple Codec Support**: H.264, XVID fallbacks for maximum compatibility
58
- - **MyAvatar API**: Direct integration for seamless library management
59
 
60
- # 🍹 Video Background Replacer
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
 
62
- ## 2-Stage Processing Pipeline
63
 
64
- **Stage 1:** AI-powered person segmentation (MatAnyone/MediaPipe) β†’ Clean green screen video
65
- **Stage 2:** Advanced chroma key replacement β†’ Green screen becomes your custom background
 
 
 
66
 
67
- This approach ensures maximum quality and reliability compared to direct background replacement.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
 
69
- ## Features
70
- - MatAnyone AI for ultimate quality segmentation
71
- - MediaPipe fallback for good quality processing
72
- - Advanced chroma key algorithm with edge smoothing
73
- - Optimized for iframe embedding in MyAvatar
 
1
+ # 🍹 Video Background Replacer
 
 
 
 
 
 
 
 
 
 
2
 
3
+ AI-powered video background replacement with automatic person segmentation and audio preservation.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
+ ## πŸš€ Features
6
+
7
+ - **πŸ€– AI-Powered Segmentation**: Uses MatAnyone from Hugging Face for high-quality person extraction
8
+ - **🎭 Automatic Masking**: MediaPipe automatically detects and segments people - no manual masking needed
9
+ - **🎡 Audio Preservation**: Maintains original audio using FFmpeg with moviepy fallback
10
+ - **πŸ’Ύ Direct Save**: Integrates with MyAvatar for easy video library management
11
+ - **πŸ“± Mobile Friendly**: Responsive design works on all devices
12
+ - **🎬 Professional Quality**: Optimized video output with proper encoding
13
+
14
+ ## πŸ› οΈ How It Works
15
+
16
+ 1. **Upload** your video and background image
17
+ 2. **AI Processing**: MatAnyone extracts the person from the video
18
+ 3. **Background Replacement**: Your new background is seamlessly applied
19
+ 4. **Audio Preservation**: Original audio is maintained
20
+ 5. **Download or Save**: Get your processed video or save to MyAvatar
21
+
22
+ ## 🎯 Technology Stack
23
+
24
+ - **Frontend**: Streamlit with custom CSS
25
+ - **AI Model**: MatAnyone (PeiqingYang/MatAnyone from Hugging Face)
26
+ - **Segmentation**: MediaPipe for automatic person detection
27
+ - **Video Processing**: OpenCV + FFmpeg + MoviePy
28
+ - **Audio**: FFmpeg with moviepy fallback for compatibility
29
+
30
+ ## πŸ“‹ Requirements
31
+
32
+ - Python 3.10+
33
+ - CUDA support (for GPU acceleration)
34
+ - FFmpeg (for audio processing)
35
+
36
+ ## πŸ”§ Installation
37
+
38
+ The app automatically installs all dependencies from `requirements.txt`, including:
39
+
40
+ - **MatAnyone**: Loaded directly from Hugging Face Hub
41
+ - **MediaPipe**: For automatic segmentation
42
+ - **Streamlit**: Web interface
43
+ - **OpenCV**: Video processing
44
+ - **FFmpeg**: Audio handling
45
 
46
+ ## 🎨 Usage
47
 
48
+ 1. **Upload Video**: Choose any MP4, AVI, or MOV file
49
+ 2. **Upload Background**: Select a PNG, JPG, or JPEG image
50
+ 3. **Click Process**: The AI handles everything automatically
51
+ 4. **Download**: Get your processed video with preserved audio
52
+ 5. **Save to MyAvatar**: Optional direct integration with MyAvatar library
53
 
54
+ ## ✨ Key Improvements
55
+
56
+ ### Automatic Segmentation
57
+ - No manual mask creation needed
58
+ - MediaPipe detects people automatically
59
+ - Works with any video containing people
60
+
61
+ ### HuggingFace Integration
62
+ ```python
63
+ processor = InferenceCore("PeiqingYang/MatAnyone")
64
+ ```
65
+ - Direct model loading from HF Hub
66
+ - No manual model downloads
67
+ - Always latest version
68
+
69
+ ### Audio Preservation
70
+ - Primary: FFmpeg for high-quality audio
71
+ - Fallback: MoviePy for compatibility
72
+ - Maintains original audio quality
73
+
74
+ ### MyAvatar Integration
75
+ - Direct API connection
76
+ - Base64 encoding for secure transfer
77
+ - Automatic filename generation
78
+
79
+ ## 🎬 Supported Formats
80
+
81
+ **Input Video**: MP4, AVI, MOV
82
+ **Background Image**: PNG, JPG, JPEG
83
+ **Output**: MP4 with H.264 encoding
84
+
85
+ ## 🌟 Perfect For
86
+
87
+ - **Content Creators**: Professional video backgrounds
88
+ - **Remote Work**: Clean video call backgrounds
89
+ - **Social Media**: Creative video content
90
+ - **Marketing**: Branded video backgrounds
91
+ - **Education**: Consistent presentation backgrounds
92
+
93
+ ## πŸ”’ Privacy
94
+
95
+ - All processing happens on secure servers
96
+ - No data stored permanently
97
+ - Optional MyAvatar integration for your personal library
98
+
99
+ ## 🀝 Support
100
+
101
+ Built with modern AI technology for reliable, high-quality results. Optimized for both desktop and mobile use.
102
+
103
+ ---
104
 
105
+ *🍹 Powered by MatAnyone AI | Connected to MyAvatar*