Image-to-Video
Diffusers
Safetensors
StableVideoDiffusionPipeline
TaiMingLu commited on
Commit
09b6f01
Β·
verified Β·
1 Parent(s): 521d9a2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -48
README.md CHANGED
@@ -1,49 +1,50 @@
1
- ---
2
- library_name: diffusers
3
- pipeline_tag: video-to-video
4
- ---
5
-
6
- # GenEx-World-Initializer πŸš€πŸŒ
7
-
8
- **GenEx World Explorer** is a video generation pipeline built on top of [Stable Video Diffusion (SVD)](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1).
9
- . It takes a keyframe, and generates a temporally consistent video. This explorer version builds on SVD with a custom `UNetSpatioTemporalConditionModel`.
10
-
11
- The diffuser generate a forward moving path of a panoramic input image, to explore a given scene.
12
-
13
-
14
- ![](examples.png)
15
-
16
- ## πŸ“¦ Usage
17
-
18
- ```python
19
- from diffusers import UNetSpatioTemporalConditionModel, StableVideoDiffusionPipeline
20
- import torch
21
-
22
- model_id = 'TaiMingLu/GenEx-World-Explorer'
23
-
24
- # Load the custom UNet
25
- unet = UNetSpatioTemporalConditionModel.from_pretrained(
26
- model_id,
27
- subfolder='unet',
28
- torch_dtype=torch.float16,
29
- low_cpu_mem_usage=True,
30
- )
31
-
32
- # Load the full pipeline with custom UNet
33
- pipe = StableVideoDiffusionPipeline.from_pretrained(
34
- model_id,
35
- unet=unet,
36
- low_cpu_mem_usage=True,
37
- torch_dtype=torch.float16,
38
- local_files_only=True,
39
- ).to('cuda')
40
- ```
41
-
42
- ## πŸ”§ Requirements
43
-
44
- ```
45
- diffusers>=0.33.1
46
- transformers
47
- numpy
48
- pillow
 
49
  ```
 
1
+ ---
2
+ base_model:
3
+ - stabilityai/stable-video-diffusion-img2vid-xt-1-1
4
+ pipeline_tag: image-to-video
5
+ ---
6
+
7
+ # GenEx-World-Initializer πŸš€πŸŒ
8
+
9
+ **GenEx World Explorer** is a video generation pipeline built on top of [Stable Video Diffusion (SVD)](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1).
10
+ . It takes a keyframe, and generates a temporally consistent video. This explorer version builds on SVD with a custom `UNetSpatioTemporalConditionModel`.
11
+
12
+ The diffuser generate a forward moving path of a panoramic input image, to explore a given scene.
13
+
14
+
15
+ ![](examples.png)
16
+
17
+ ## πŸ“¦ Usage
18
+
19
+ ```python
20
+ from diffusers import UNetSpatioTemporalConditionModel, StableVideoDiffusionPipeline
21
+ import torch
22
+
23
+ model_id = 'TaiMingLu/GenEx-World-Explorer'
24
+
25
+ # Load the custom UNet
26
+ unet = UNetSpatioTemporalConditionModel.from_pretrained(
27
+ model_id,
28
+ subfolder='unet',
29
+ torch_dtype=torch.float16,
30
+ low_cpu_mem_usage=True,
31
+ )
32
+
33
+ # Load the full pipeline with custom UNet
34
+ pipe = StableVideoDiffusionPipeline.from_pretrained(
35
+ model_id,
36
+ unet=unet,
37
+ low_cpu_mem_usage=True,
38
+ torch_dtype=torch.float16,
39
+ local_files_only=True,
40
+ ).to('cuda')
41
+ ```
42
+
43
+ ## πŸ”§ Requirements
44
+
45
+ ```
46
+ diffusers>=0.33.1
47
+ transformers
48
+ numpy
49
+ pillow
50
  ```