Describe the current release without archive or migration history
Browse files
README.md
CHANGED
|
@@ -31,23 +31,22 @@ AudioLDM-ControlNet is a dance-conditioned music generation model. It combines:
|
|
| 31 |
3. a ControlNet-style conditioning module built on an AudioLDM backbone.
|
| 32 |
|
| 33 |
This repository includes the **full model**, the **w/o MotionBERT** checkpoint,
|
| 34 |
-
|
| 35 |
-
|
| 36 |
|
| 37 |
## Repository Contents
|
| 38 |
|
| 39 |
-
|
| 40 |
-
`testsets_video_results/`.
|
| 41 |
|
| 42 |
| Path | Contents |
|
| 43 |
| --- | --- |
|
| 44 |
-
| [`models/full/`](https://huggingface.co/kmraven/AudioLDM-ControlNet/tree/main/models/full) | Full-model checkpoint, YAML configuration, 36 generated WAV files and `metrics.txt` in `300ksteps/`, and the
|
| 45 |
| [`models/without_motioinbert/`](https://huggingface.co/kmraven/AudioLDM-ControlNet/tree/main/models/without_motioinbert) | w/o MotionBERT checkpoint, YAML configuration, and 36 generated WAV files and `metrics.txt` in `300ksteps/` |
|
| 46 |
| [`testsets_video_results/`](https://huggingface.co/kmraven/AudioLDM-ControlNet/tree/main/testsets_video_results) | 216 MP4 comparison videos: 36 clips for each of the six conditions listed below |
|
| 47 |
-
| `checkpoint-global_step=299999.ckpt` |
|
| 48 |
-
| `audioldm_original_medium_stretch_pretrained.yaml` |
|
| 49 |
|
| 50 |
-
The spelling `without_motioinbert` is retained from the original archive.
|
| 51 |
The root-level checkpoint and configuration are identical to their counterparts
|
| 52 |
under `models/full/`.
|
| 53 |
|
|
@@ -61,7 +60,7 @@ configurations are:
|
|
| 61 |
The six video folders are `AudioLDM (default)`, `Ground Truth`, `Li et al.`,
|
| 62 |
`Ours`, `Ours (w-o Contrastive Pretraining)`, and `Ours (w-o MotionBERT)`.
|
| 63 |
The w/o Contrastive Pretraining variant is represented by comparison videos;
|
| 64 |
-
a checkpoint for that variant
|
| 65 |
|
| 66 |
## Download and Use
|
| 67 |
|
|
@@ -71,8 +70,7 @@ Install the Hugging Face CLI if needed:
|
|
| 71 |
pip install -U huggingface_hub
|
| 72 |
```
|
| 73 |
|
| 74 |
-
Download
|
| 75 |
-
its directory structure and avoiding a second local copy of the root checkpoint:
|
| 76 |
|
| 77 |
```bash
|
| 78 |
hf download kmraven/AudioLDM-ControlNet \
|
|
@@ -92,13 +90,13 @@ hf download kmraven/AudioLDM-ControlNet \
|
|
| 92 |
Run preprocessing and evaluation with the
|
| 93 |
[official source code and setup instructions](https://github.com/kmraven/AudioLDM-ControlNet#data-and-checkpoints).
|
| 94 |
These are project-specific PyTorch Lightning checkpoints; use the repository's
|
| 95 |
-
evaluation scripts. The
|
| 96 |
-
|
| 97 |
-
|
| 98 |
does not include a standalone Stage 1 `model_best.pth` checkpoint.
|
| 99 |
|
| 100 |
Exact training resume, including optimizer state and global step, may be
|
| 101 |
-
incompatible with the
|
| 102 |
repository recommends commit `0c1f4e3197a7c339586ce5769c6a4510dfbc6f30`.
|
| 103 |
|
| 104 |
## Checkpoint Details
|
|
|
|
| 31 |
3. a ControlNet-style conditioning module built on an AudioLDM backbone.
|
| 32 |
|
| 33 |
This repository includes the **full model**, the **w/o MotionBERT** checkpoint,
|
| 34 |
+
along with generated audio, evaluation metrics, training logs, and comparison
|
| 35 |
+
videos.
|
| 36 |
|
| 37 |
## Repository Contents
|
| 38 |
|
| 39 |
+
Model checkpoints and evaluation outputs are organized under `models/` and
|
| 40 |
+
`testsets_video_results/`.
|
| 41 |
|
| 42 |
| Path | Contents |
|
| 43 |
| --- | --- |
|
| 44 |
+
| [`models/full/`](https://huggingface.co/kmraven/AudioLDM-ControlNet/tree/main/models/full) | Full-model checkpoint, YAML configuration, 36 generated WAV files and `metrics.txt` in `300ksteps/`, and the W&B training run in `wandb/` |
|
| 45 |
| [`models/without_motioinbert/`](https://huggingface.co/kmraven/AudioLDM-ControlNet/tree/main/models/without_motioinbert) | w/o MotionBERT checkpoint, YAML configuration, and 36 generated WAV files and `metrics.txt` in `300ksteps/` |
|
| 46 |
| [`testsets_video_results/`](https://huggingface.co/kmraven/AudioLDM-ControlNet/tree/main/testsets_video_results) | 216 MP4 comparison videos: 36 clips for each of the six conditions listed below |
|
| 47 |
+
| `checkpoint-global_step=299999.ckpt` | Full-model checkpoint, also available under `models/full/checkpoints/` |
|
| 48 |
+
| `audioldm_original_medium_stretch_pretrained.yaml` | Full-model configuration, also available under `models/full/` |
|
| 49 |
|
|
|
|
| 50 |
The root-level checkpoint and configuration are identical to their counterparts
|
| 51 |
under `models/full/`.
|
| 52 |
|
|
|
|
| 60 |
The six video folders are `AudioLDM (default)`, `Ground Truth`, `Li et al.`,
|
| 61 |
`Ours`, `Ours (w-o Contrastive Pretraining)`, and `Ours (w-o MotionBERT)`.
|
| 62 |
The w/o Contrastive Pretraining variant is represented by comparison videos;
|
| 63 |
+
a checkpoint for that variant is not included in this release.
|
| 64 |
|
| 65 |
## Download and Use
|
| 66 |
|
|
|
|
| 70 |
pip install -U huggingface_hub
|
| 71 |
```
|
| 72 |
|
| 73 |
+
Download all model and evaluation files (approximately 16.5 GB):
|
|
|
|
| 74 |
|
| 75 |
```bash
|
| 76 |
hf download kmraven/AudioLDM-ControlNet \
|
|
|
|
| 90 |
Run preprocessing and evaluation with the
|
| 91 |
[official source code and setup instructions](https://github.com/kmraven/AudioLDM-ControlNet#data-and-checkpoints).
|
| 92 |
These are project-specific PyTorch Lightning checkpoints; use the repository's
|
| 93 |
+
evaluation scripts. The YAML files specify the training settings and use
|
| 94 |
+
relative paths. Dataset inputs and separately referenced upstream weights must
|
| 95 |
+
be prepared as described in the source repository. This release
|
| 96 |
does not include a standalone Stage 1 `model_best.pth` checkpoint.
|
| 97 |
|
| 98 |
Exact training resume, including optimizer state and global step, may be
|
| 99 |
+
incompatible with the current training code. If exact resume fails, the source
|
| 100 |
repository recommends commit `0c1f4e3197a7c339586ce5769c6a4510dfbc6f30`.
|
| 101 |
|
| 102 |
## Checkpoint Details
|