kelseye commited on
Commit
ebefe25
·
verified ·
1 Parent(s): 6fa1899

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ _cover_images_/cover_video.mp4 filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+ # Wanxiao 2.1-1.3B-LoRA-Length-Extension-v1
5
+
6
+ ## Model Introduction
7
+
8
+ This LoRA model is trained based on the [Wanxiao 2.1-1.3B](https://www.modelscope.cn/models/Wan-AI/Wan2.1-T2V-1.3B) model and the [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) framework. It extends the video length capability, enabling generation of videos twice as long when using this LoRA model.
9
+
10
+ Recommended parameter setting: `num_frames=161`
11
+
12
+ ## Model Performance
13
+
14
+ Prompt: Documentary photography style, a lively puppy wearing black sunglasses running quickly across a lush green grass field. The puppy has golden-brown fur and black sunglasses, with upright ears and an expression full of focus and joy. Sunlight shines on its body, making its coat appear exceptionally soft and shiny. The background features an open grassland dotted with occasional wildflowers, with a faint view of blue sky and scattered white clouds in the distance. Strong perspective conveys the dynamic motion of the running puppy and the vitality of the surrounding grassland. Medium shot with a side-moving viewpoint.
15
+
16
+ <div align="center"><video width="80%" controls><source src="_cover_images_/cover_video.mp4" type="video/mp4">Your browser does not support the video tag.</video></div>
17
+
18
+ ## Usage Instructions
19
+
20
+ This model is built upon the [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) framework. Please install it first:
21
+
22
+ ```
23
+ pip install diffsynth
24
+ ```
25
+
26
+ ```python
27
+ import torch
28
+ from diffsynth import ModelManager, WanVideoPipeline, save_video
29
+ from modelscope import snapshot_download
30
+
31
+
32
+ snapshot_download(
33
+ model_id="DiffSynth-Studio/Wan2.1-1.3b-lora-exvideo-v1",
34
+ local_dir="models/DiffSynth-Studio/Wan2.1-1.3b-lora-exvideo-v1",
35
+ allow_file_pattern="*.safetensors"
36
+ )
37
+ model_manager = ModelManager(device="cpu")
38
+ model_manager.load_models(
39
+ [
40
+ "models/Wan-AI/Wan2.1-T2V-1.3B/diffusion_pytorch_model.safetensors",
41
+ "models/Wan-AI/Wan2.1-T2V-1.3B/models_t5_umt5-xxl-enc-bf16.pth",
42
+ "models/Wan-AI/Wan2.1-T2V-1.3B/Wan2.1_VAE.pth",
43
+ ],
44
+ torch_dtype=torch.bfloat16,
45
+ )
46
+ model_manager.load_lora("models/DiffSynth-Studio/Wan2.1-1.3b-lora-exvideo-v1/model.safetensors", lora_alpha=1)
47
+ pipe = WanVideoPipeline.from_model_manager(model_manager, torch_dtype=torch.bfloat16, device="cuda")
48
+ pipe.enable_vram_management(num_persistent_param_in_dit=None)
49
+ ```
50
+
51
+ video = pipe(
52
+ prompt="Documentary photography style scene, a lively little dog running quickly on a green grassy field. The dog has brownish-yellow fur, upright ears, and an expression that is focused yet joyful. Sunlight shines on its body, making the fur appear exceptionally soft and shiny. The background features an open grassland dotted with occasional wildflowers, with a faint view of blue sky and a few white clouds in the distance. Strong sense of perspective captures the dynamic motion of the running dog and the vitality of the surrounding grass. Mid-shot side-moving viewpoint.",
53
+ negative_prompt="Vivid colors, overexposed, static, blurry details, subtitles, style, artwork, painting, frame, still, overall grayish tone, worst quality, low quality, JPEG compression artifacts, ugly, defective, extra fingers, poorly drawn hands, poorly drawn face, malformed limbs, fused fingers, motionless scene, cluttered background, three legs, crowded background, walking backwards",
54
+ num_inference_steps=50, num_frames=161,
55
+ seed=0, tiled=True
56
+ )
57
+ save_video(video, "video1.mp4", fps=15, quality=5)
58
+ ```
README_from_modelscope.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: MusePublic/wan2.1-1.3b@v1
3
+ cover_images:
4
+ - _cover_images_/cover_video.mp4
5
+ frameworks:
6
+ - Pytorch
7
+ license: Apache License 2.0
8
+ tags:
9
+ - LoRA
10
+ - text2video generation
11
+ tasks:
12
+ - text-to-video-synthesis
13
+
14
+ trigger_words:
15
+ - ""
16
+
17
+ vision_foundation: WAN_VIDEO_2_1_T2V_1_3_B
18
+ ---
19
+
20
+ # 通义万相2.1-1.3B-LoRA-长度扩展-v1
21
+
22
+ ## 模型介绍
23
+
24
+ 本 LoRA 模型是基于模型[通义万相2.1-1.3B](https://www.modelscope.cn/models/Wan-AI/Wan2.1-T2V-1.3B)和框架 [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) 训练的LoRA。对视频长度进行了扩展,使用本 LoRA 模型后,能够生成两倍长度的视频。
25
+
26
+ 建议的参数设置:`num_frames=161`
27
+
28
+ ## 模型效果
29
+
30
+ 提示词:纪实摄影风格画面,一只活泼的小狗戴着黑色墨镜在绿茵茵的草地上迅速奔跑。小狗毛色棕黄,戴着黑色墨镜,两只耳朵立起,神情专注而欢快。阳光洒在它身上,使得毛发看上去格外柔软而闪亮。背景是一片开阔的草地,偶尔点缀着几朵野花,远处隐约可见蓝天和几片白云。透视感鲜明,捕捉小狗奔跑时的动感和四周草地的生机。中景侧面移动视角。
31
+
32
+ <div align="center"><video width="80%" controls><source src="_cover_images_/cover_video.mp4" type="video/mp4">Your browser does not support the video tag.</video></div>
33
+
34
+ ## 使用说明
35
+
36
+ 本模型基于框架 [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) 训练,请先安装
37
+
38
+ ```
39
+ pip install diffsynth
40
+ ```
41
+
42
+ ```python
43
+ import torch
44
+ from diffsynth import ModelManager, WanVideoPipeline, save_video
45
+ from modelscope import snapshot_download
46
+
47
+
48
+ snapshot_download(
49
+ model_id="DiffSynth-Studio/Wan2.1-1.3b-lora-exvideo-v1",
50
+ local_dir="models/DiffSynth-Studio/Wan2.1-1.3b-lora-exvideo-v1",
51
+ allow_file_pattern="*.safetensors"
52
+ )
53
+ model_manager = ModelManager(device="cpu")
54
+ model_manager.load_models(
55
+ [
56
+ "models/Wan-AI/Wan2.1-T2V-1.3B/diffusion_pytorch_model.safetensors",
57
+ "models/Wan-AI/Wan2.1-T2V-1.3B/models_t5_umt5-xxl-enc-bf16.pth",
58
+ "models/Wan-AI/Wan2.1-T2V-1.3B/Wan2.1_VAE.pth",
59
+ ],
60
+ torch_dtype=torch.bfloat16,
61
+ )
62
+ model_manager.load_lora("models/DiffSynth-Studio/Wan2.1-1.3b-lora-exvideo-v1/model.safetensors", lora_alpha=1)
63
+ pipe = WanVideoPipeline.from_model_manager(model_manager, torch_dtype=torch.bfloat16, device="cuda")
64
+ pipe.enable_vram_management(num_persistent_param_in_dit=None)
65
+
66
+ video = pipe(
67
+ prompt="纪实摄影风格画面,一只活泼的小狗在绿茵茵的草地上迅速奔跑。小狗毛色棕黄,两只耳朵立起,神情专注而欢快。阳光洒在它身上,使得毛发看上去格外柔软而闪亮。背景是一片开阔的草地,偶尔点缀着几朵野花,远处隐约可见蓝天和几片白云。透视感鲜明,捕捉小狗奔跑时的动感和四周草地的生机。中景侧面移动视角。",
68
+ negative_prompt="色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走",
69
+ num_inference_steps=50, num_frames=161,
70
+ seed=0, tiled=True
71
+ )
72
+ save_video(video, "video1.mp4", fps=15, quality=5)
73
+ ```
_cover_images_/cover_video.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b72c410f2b72a48a10b01a315643ac179a786da85bbf2413d8447bab178c9a9d
3
+ size 1474232
configuration.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"aigc_model":true,"framework":"pytorch","model_file_location":"model.safetensors","task":"text-to-image-synthesis"}
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4003327b62d7ad093b7d173a422a6f57c466f5436340cc70092d5d7a51649e6a
3
+ size 350031392