CocoBro commited on
Commit
1ebd877
Β·
verified Β·
1 Parent(s): f0b101f

Add files using upload-large-folder tool

Browse files
Files changed (2) hide show
  1. README.md +26 -2
  2. ckpts/Foley-Omni/v2st.pth +3 -0
README.md CHANGED
@@ -24,7 +24,7 @@ The main model checkpoint in this release is an inference-only export from:
24
  ```text
25
  ckpts/
26
  β”œβ”€β”€ Foley-Omni/
27
- β”‚ └── model_checkpoint.pth
28
  β”œβ”€β”€ Wan2.2-TI2V-5B/
29
  β”‚ β”œβ”€β”€ models_t5_umt5-xxl-enc-bf16.pth
30
  β”‚ └── google/
@@ -42,7 +42,7 @@ ckpts/
42
 
43
  What each part is used for:
44
 
45
- - `ckpts/Foley-Omni/model_checkpoint.pth`: released inference-only Foley-Omni weights
46
  - `ckpts/Wan2.2-TI2V-5B/*`: text encoder and tokenizer for text conditioning
47
  - `ckpts/mmaudio/ext_weights/v1-16.pth`: audio VAE for the 16 kHz inference path
48
  - `ckpts/mmaudio/ext_weights/best_netG.pt`: vocoder for waveform decoding
@@ -68,4 +68,28 @@ This repository redistributes a small subset of files from the following upstrea
68
  - **MMAudio**: audio VAE, vocoder, and Synchformer files
69
 
70
  Please refer to the original upstream repositories for their licenses, usage terms, and project details.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  ```
 
24
  ```text
25
  ckpts/
26
  β”œβ”€β”€ Foley-Omni/
27
+ β”‚ └── v2st.pth
28
  β”œβ”€β”€ Wan2.2-TI2V-5B/
29
  β”‚ β”œβ”€β”€ models_t5_umt5-xxl-enc-bf16.pth
30
  β”‚ └── google/
 
42
 
43
  What each part is used for:
44
 
45
+ - `ckpts/Foley-Omni/v2st.pth`: released inference-only Foley-Omni weights
46
  - `ckpts/Wan2.2-TI2V-5B/*`: text encoder and tokenizer for text conditioning
47
  - `ckpts/mmaudio/ext_weights/v1-16.pth`: audio VAE for the 16 kHz inference path
48
  - `ckpts/mmaudio/ext_weights/best_netG.pt`: vocoder for waveform decoding
 
68
  - **MMAudio**: audio VAE, vocoder, and Synchformer files
69
 
70
  Please refer to the original upstream repositories for their licenses, usage terms, and project details.
71
+
72
+ ## Quick Start
73
+
74
+ Use the code repository for inference scripts, configs, examples, and feature extraction tools:
75
+
76
+ - `inference_v2st.py`
77
+ - `inference_v2st.yaml`
78
+ - `examples/video_text_example.json`
79
+ - `data_process/convert_memmap_to_npy.py`
80
+
81
+ Download the packaged checkpoints with:
82
+
83
+ ```bash
84
+ hf download CocoBro/Foley-Omni \
85
+ ckpts/Foley-Omni/v2st.pth \
86
+ ckpts/Wan2.2-TI2V-5B/models_t5_umt5-xxl-enc-bf16.pth \
87
+ ckpts/Wan2.2-TI2V-5B/google/umt5-xxl/special_tokens_map.json \
88
+ ckpts/Wan2.2-TI2V-5B/google/umt5-xxl/spiece.model \
89
+ ckpts/Wan2.2-TI2V-5B/google/umt5-xxl/tokenizer.json \
90
+ ckpts/Wan2.2-TI2V-5B/google/umt5-xxl/tokenizer_config.json \
91
+ ckpts/mmaudio/ext_weights/v1-16.pth \
92
+ ckpts/mmaudio/ext_weights/best_netG.pt \
93
+ ckpts/mmaudio/ext_weights/synchformer_state_dict.pth \
94
+ --local-dir .
95
  ```
ckpts/Foley-Omni/v2st.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8dfcebc33b4848b3639cea815000c8b2c9e02de2ffc655a763c03f3e4232d941
3
+ size 22214978751