Upload README.md

README.md

@@ -0,0 +1,169 @@
+# WarpTuber
+
+A real-time face animation tool that brings static images and videos to life using your webcam.
+
+Tutorial: https://www.youtube.com/watch?v=w1_KY5E7Aig
+
+## Overview
+
+WarpTuber allows you to animate portraits or character images by mapping your facial expressions and movements to them in real-time. It supports both static images and video animations, making it perfect for virtual avatars, content creation, and streaming.
+
+## Quick Start
+
+1. Connect your webcam
+2. Choose one of the following launch options:
+   - `camera_image.bat` - For animating a static image
+   - `camera_video.bat` - For using a video animation with facial expressions
+
+## Setup Requirements
+
+- Windows 10/11
+- Webcam
+- Python environment (included in the `venv` folder)
+- NVIDIA GPU with CUDA support (recommended)
+- Python local install
+
+## Usage Instructions
+
+### Using Static Images (`camera_image.bat`)
+
+Use this mode when you want to animate a single static portrait or character image with your facial movements.
+
+1. Place your image in `assets/examples/source/` and name it `main.jpg`
+2. Run `camera_image.bat`
+3. When prompted, select your webcam index (usually 0 for the default webcam)
+4. The application will open with your animated image
+
+### Using Video Animations (`camera_video.bat`)
+
+Use this mode when you want to combine a pre-recorded animation/idle loop with your facial expressions.
+
+1. Place your video in `assets/examples/source/` and name it `main.mp4`
+2. Run `camera_video.bat`
+3. When prompted, select your webcam index (usually 0 for the default webcam)
+4. The application will open with your video animation that responds to your facial movements
+
+### File Requirements
+
+- **Static Image**: Must be named `main.jpg` and placed in `assets/examples/source/`
+- **Animation Video**: Must be named `main.mp4` and placed in `assets/examples/source/`
+
+For best results:
+- Images should be clear portraits with visible facial features
+- Videos should be looping animations with consistent lighting
+- Both should have dimensions of at least 512x512 pixels
+
+### Image Compatibility and Facial Landmark Detection
+
+WarpTuber relies on facial landmark detection to animate images. Not all images will work properly, especially:
+
+- Highly stylized anime or cartoon characters
+- Images with unusual facial proportions
+- Artwork with abstract facial features
+- Images with poor lighting or low contrast
+- Side-profile portraits (faces should be mostly front-facing)
+
+If you encounter an error message like `"no face in [image path]! exit!"`, this means the system failed to detect facial landmarks in your image.
+
+**Solutions for facial landmark detection issues:**
+
+1. **Try a different image**: Use photographs or realistic illustrations with clear facial features
+2. **Enable animal mode**: For cartoon characters, try running with the `--animal` parameter:
+   ```
+   camera_image.bat --animal
+   ```
+3. **Adjust the image**: Edit your image to make facial features more prominent:
+   - Increase contrast around eyes, nose, and mouth
+   - Ensure the face is well-lit and centered
+   - Crop the image to focus more on the face
+   - Convert stylized characters to a more realistic style using AI tools
+
+4. **Pre-process with face enhancement**: Use photo editing software to enhance facial features before using with WarpTuber
+
+Note that even with these adjustments, some highly stylized images may not work with the current facial landmark detection system.
+
+## Advanced Configuration
+
+### Command Line Parameters
+
+Both batch files support additional parameters:
+
+- `--src_image [path]` - Specify a custom source image/video path
+- `--animal` - Enable animal face mode
+- `--paste_back` - Enable background preservation (enabled by default)
+- `--interactive` - Enable interactive controls (enabled by default)
+- `--advanced_ui` - Enable advanced UI controls
+
+Example: `camera_image.bat --src_image assets/my_custom_folder/portrait.jpg --animal`
+
+### TensorRT Optimization
+
+This repository includes pre-compiled TensorRT models for optimal performance. If you encounter issues with the included models, you may need to compile your own:
+
+1. Navigate to the `checkpoints` directory and locate any existing `.trt` files:
+   - Human model files: `checkpoints/liveportrait_onnx/*.trt`
+   - Animal model files: `checkpoints/liveportrait_animal_onnx/*.trt`
+   - Example path: `C:\FLivePort\WarpTuber\checkpoints\liveportrait_onnx\stitching_lip.trt`
+
+2. Delete these `.trt` files if you're experiencing compatibility issues
+
+3. Run `scripts/all_onnx2trt.bat` to recompile all models
+   - This will convert all ONNX models to TensorRT format optimized for your specific GPU
+   - The conversion process may take several minutes to complete
+
+Note: Compiling your own TensorRT models is only necessary if the pre-compiled models don't work on your system. The compilation process creates optimized models specifically for your GPU hardware.
+
+## Troubleshooting
+
+- If no webcams are detected, ensure your camera is properly connected and not in use by another application
+- If the animation appears laggy, try closing other GPU-intensive applications
+- If you encounter model loading errors, try running the `scripts/all_onnx2trt.bat` script to compile models for your specific GPU
+- If you see the error `"no face in driving frame"`, ensure your webcam can clearly see your face with good lighting
+
+## License
+
+MIT License
+
+Copyright (c) 2024 Kuaishou Visual Generation and Interaction Center
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+**Important Note**: 
+- The code of InsightFace is released under the MIT License.
+- The models of InsightFace are for non-commercial research purposes only.
+- If you want to use the WarpTuber project for commercial purposes, you should remove and replace InsightFace's detection models to fully comply with the MIT license.
+
+## Acknowledgements
+
+This project is based on [FasterLivePortrait](https://github.com/warmshao/FasterLivePortrait) by [warmshao](https://github.com/warmshao), which is an optimized implementation of the original [LivePortrait](https://github.com/KwaiVGI/LivePortrait) technology developed by the Kuaishou Technology team.
+
+The original LivePortrait technology was created by Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang from Kuaishou Technology and academic institutions.
+
+WarpTuber is a customized implementation that simplifies the usage of this technology with a focus on ease of use for content creators and streamers.
+
+Key technologies used:
+- Face landmark detection and tracking
+- Neural rendering for portrait animation
+- TensorRT optimization for real-time performance
+
+Special thanks to the original developers for making this technology accessible and open source.
+
+For more information about the original research, please visit [liveportrait.github.io](https://liveportrait.github.io/).