xlt99 commited on
Commit
77ea2c6
Β·
verified Β·
1 Parent(s): c72d79d

Document custom music inference

Browse files
Files changed (1) hide show
  1. README.md +51 -1
README.md CHANGED
@@ -21,6 +21,9 @@ tags:
21
  <a href="#code">
22
  <img src="https://img.shields.io/badge/Conference-ECCV%202026-orange" alt="Conference">
23
  </a>
 
 
 
24
  </p>
25
 
26
  <p align="center">
@@ -32,7 +35,7 @@ tags:
32
  > FlowerDance combines MeanFlow with Physical Consistency Constraints for high-quality few-step sampling, and uses a lightweight non-autoregressive BiMamba backbone with Channel-Level Fusion for long-horizon music-to-dance synthesis. It also supports motion editing through time-decayed soft masking, enabling users to refine generated dance sequences interactively.
33
 
34
  πŸŽ‰ **FlowerDance has been accepted to ECCV 2026!**
35
- ✨ Training code release! ✨
36
 
37
  ---
38
 
@@ -71,6 +74,8 @@ To set up the necessary environment for running this project, follow the steps b
71
  - Download the complete **preprocessed data archive** from [Hugging Face](https://huggingface.co/datasets/xlt99/FlowerDance-Preprocessed/resolve/main/data.7z?download=true) and extract it in the project root. The archive contains the required `./data/` directory.
72
  - Download the **evaluation checkpoint** from [Hugging Face](https://huggingface.co/xlt99/FlowerDance/resolve/main/train-3700.pt?download=true) and place it at `./runs/train/uniform2/weights/train-3700.pt`.
73
 
 
 
74
  ---
75
 
76
  ## 🧩 Directory Structure
@@ -87,6 +92,7 @@ FlowerDance/
87
  β”œβ”€β”€ requirements.txt
88
  β”œβ”€β”€ args.py
89
  β”œβ”€β”€ EDGE.py
 
90
  β”œβ”€β”€ inpaint.py
91
  β”œβ”€β”€ test.py
92
  └── vis.py
@@ -101,6 +107,50 @@ accelerate launch train.py --batch_size 128 --epochs 4000 --feature_type baseli
101
  ```
102
  ---
103
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
  ## πŸ“ Evaluation
105
 
106
  ### πŸ§ͺ Evaluate the Model
 
21
  <a href="#code">
22
  <img src="https://img.shields.io/badge/Conference-ECCV%202026-orange" alt="Conference">
23
  </a>
24
+ <a href="#inference">
25
+ <img src="https://img.shields.io/badge/Inference-Custom_Music-8a2be2" alt="Inference">
26
+ </a>
27
  </p>
28
 
29
  <p align="center">
 
35
  > FlowerDance combines MeanFlow with Physical Consistency Constraints for high-quality few-step sampling, and uses a lightweight non-autoregressive BiMamba backbone with Channel-Level Fusion for long-horizon music-to-dance synthesis. It also supports motion editing through time-decayed soft masking, enabling users to refine generated dance sequences interactively.
36
 
37
  πŸŽ‰ **FlowerDance has been accepted to ECCV 2026!**
38
+ ✨ Training and inference code released! ✨
39
 
40
  ---
41
 
 
74
  - Download the complete **preprocessed data archive** from [Hugging Face](https://huggingface.co/datasets/xlt99/FlowerDance-Preprocessed/resolve/main/data.7z?download=true) and extract it in the project root. The archive contains the required `./data/` directory.
75
  - Download the **evaluation checkpoint** from [Hugging Face](https://huggingface.co/xlt99/FlowerDance/resolve/main/train-3700.pt?download=true) and place it at `./runs/train/uniform2/weights/train-3700.pt`.
76
 
77
+ The preprocessed data archive is only required for training and evaluation. The custom-music inference script can download the checkpoint automatically and does not require the dataset archive.
78
+
79
  ---
80
 
81
  ## 🧩 Directory Structure
 
92
  β”œβ”€β”€ requirements.txt
93
  β”œβ”€β”€ args.py
94
  β”œβ”€β”€ EDGE.py
95
+ β”œβ”€β”€ inference.py
96
  β”œβ”€β”€ inpaint.py
97
  β”œβ”€β”€ test.py
98
  └── vis.py
 
107
  ```
108
  ---
109
 
110
+ <a id="inference"></a>
111
+
112
+ ## 🎡 Inference
113
+
114
+ Generate a 3D dance sequence directly from your own music:
115
+
116
+ ```bash
117
+ python inference.py path/to/music.wav \
118
+ --genre Hiphop \
119
+ --duration 32 \
120
+ --output-dir inference_outputs
121
+ ```
122
+
123
+ The input can be a WAV, MP3, FLAC, or OGG file. The script extracts the same 35-dimensional baseline music features used during training, supports genre-conditioned generation, and automatically downloads `train-3700.pt` from [Hugging Face](https://huggingface.co/xlt99/FlowerDance) when `--checkpoint` is omitted.
124
+
125
+ List all 16 supported dance genres:
126
+
127
+ ```bash
128
+ python inference.py --list-genres
129
+ ```
130
+
131
+ Generate a 60-second sequence from a selected part of a longer track:
132
+
133
+ ```bash
134
+ python inference.py path/to/music.mp3 \
135
+ --genre Popping \
136
+ --start 10 \
137
+ --duration 60
138
+ ```
139
+
140
+ The default duration is 32 seconds and the maximum supported duration is 60 seconds. A CUDA GPU is required. Each run saves:
141
+
142
+ ```text
143
+ inference_outputs/
144
+ β”œβ”€β”€ music_Hiphop.wav
145
+ β”œβ”€β”€ music_Hiphop_features.npy
146
+ └── 0/
147
+ └── music_Hiphop.pkl
148
+ ```
149
+
150
+ The PKL file contains `smpl_poses`, `smpl_trans`, and `full_pose` for downstream visualization or rendering.
151
+
152
+ ---
153
+
154
  ## πŸ“ Evaluation
155
 
156
  ### πŸ§ͺ Evaluate the Model