Spaces:
Runtime error
Runtime error
| README_WIP.md | |
| --- | |
| title: Video Model Studio | |
| emoji: 🎥 | |
| colorFrom: gray | |
| colorTo: gray | |
| sdk: gradio | |
| sdk_version: 5.15.0 | |
| app_file: app.py | |
| pinned: true | |
| license: apache-2.0 | |
| short_description: All-in-one tool for AI video training | |
| --- | |
| # 🎥 Video Model Studio (VMS) | |
| ## Presentation | |
| VMS is an all-in-one tool to train LoRA models for various open-source AI video models: | |
| - Data collection from various sources | |
| - Splitting videos into short single camera shots | |
| - Automatic captioning | |
| - Training HunyuanVideo or LTX-Video | |
| ## Similar projects | |
| I wasn't aware of it when I started this project, | |
| but there is also this: https://github.com/alisson-anjos/diffusion-pipe-ui | |
| ## Installation | |
| VMS is built on top of Finetrainers and Gradio, and designed to run as a Hugging Face Space (but you can deploy it elsewhere if you want to). | |
| ### Full installation at Hugging Face | |
| Easy peasy: create a Space (make sure to use the `Gradio` type/template), and push the repo. No Docker needed! | |
| ### Dev mode on Hugging Face | |
| Enable dev mode in the space, then open VSCode in local or remote and run: | |
| ``` | |
| pip install -r requirements.txt | |
| ``` | |
| As this is not automatic, then click on "Restart" in the space dev mode UI widget. | |
| ### Full installation somewhere else | |
| I haven't tested it, but you can try to provided Dockerfile | |
| ### Full installation in local | |
| the full installation requires: | |
| - Linux | |
| - CUDA 12 | |
| - Python 3.10 | |
| This is because of flash attention, which is defined in the `requirements.txt` using an URL to download a prebuilt wheel (python bindings for a native library) | |
| ```bash | |
| ./setup.sh | |
| ``` | |
| ### Degraded installation in local | |
| If you cannot meet the requirements, you can: | |
| - solution 1: fix requirements.txt to use another prebuilt wheel | |
| - solution 2: manually build/install flash attention | |
| - solution 3: don't use clip captioning | |
| Here is how to do solution 3: | |
| ```bash | |
| ./setup_no_captions.sh | |
| ``` | |
| ## Run | |
| ### Running the Gradio app | |
| Note: please make sure you properly define the environment variables for `STORAGE_PATH` (eg. `/data/`) and `HF_HOME` (eg. `/data/huggingface/`) | |
| ```bash | |
| python app.py | |
| ``` | |
| ### Running locally | |
| See above remarks about the environment variable. | |
| By default `run.sh` will store stuff in `.data/` (located inside the current working directory): | |
| ```bash | |
| ./run.sh | |
| ``` |