BestWishYsh commited on
Commit
96bf6ad
·
verified ·
1 Parent(s): dfe4d06

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +95 -1
README.md CHANGED
@@ -179,7 +179,101 @@ Install diffusers from source:
179
  pip install git+https://github.com/huggingface/diffusers.git
180
  ```
181
 
182
- For example, let's take Helios-Distilled.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
183
 
184
  <details>
185
  <summary>Click to expand the code</summary>
 
179
  pip install git+https://github.com/huggingface/diffusers.git
180
  ```
181
 
182
+ For example, let's take Helios-Distilled (**Standard Pipeline**).
183
+
184
+ <details>
185
+ <summary>Click to expand the code</summary>
186
+
187
+ ```bash
188
+ import torch
189
+ from diffusers import AutoModel, HeliosPyramidPipeline
190
+ from diffusers.utils import export_to_video, load_video, load_image
191
+
192
+ vae = AutoModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="vae", torch_dtype=torch.float32)
193
+
194
+ pipeline = HeliosPyramidPipeline.from_pretrained(
195
+ "BestWishYsh/Helios-Distilled",
196
+ vae=vae,
197
+ torch_dtype=torch.bfloat16
198
+ )
199
+ pipeline.to("cuda")
200
+
201
+ negative_prompt = """
202
+ Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality,
203
+ low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured,
204
+ misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards
205
+ """
206
+
207
+ # --- T2V ---
208
+ prompt = """
209
+ A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue
210
+ and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with
211
+ a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear,
212
+ allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades
213
+ of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and
214
+ the vivid colors of its surroundings. A close-up shot with dynamic movement.
215
+ """
216
+
217
+ output = pipeline(
218
+ prompt=prompt,
219
+ negative_prompt=negative_prompt,
220
+ num_frames=240,
221
+ pyramid_num_inference_steps_list=[2, 2, 2],
222
+ guidance_scale=1.0,
223
+ is_amplify_first_chunk=True,
224
+ generator=torch.Generator("cuda").manual_seed(42),
225
+ ).frames[0]
226
+ export_to_video(output, "helios_distilled_t2v_output.mp4", fps=24)
227
+
228
+ # --- I2V ---
229
+ i2v_prompt = """
230
+ A towering emerald wave surges forward, its crest curling with raw power and energy. Sunlight glints off the translucent water,
231
+ illuminating the intricate textures and deep green hues within the wave’s body. A thick spray erupts from the breaking crest,
232
+ casting a misty veil that dances above the churning surface. As the perspective widens, the immense scale of the wave becomes
233
+ apparent, revealing the restless expanse of the ocean stretching beyond. The scene captures the ocean’s untamed beauty and
234
+ relentless force, with every droplet and ripple shimmering in the light. The dynamic motion and vivid colors evoke both awe and
235
+ respect for nature’s might.
236
+ """
237
+ image_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/wave.jpg"
238
+
239
+ output = pipeline(
240
+ prompt=i2v_prompt,
241
+ negative_prompt=negative_prompt,
242
+ image=load_image(image_path).resize((640, 384)),
243
+ num_frames=240,
244
+ pyramid_num_inference_steps_list=[2, 2, 2],
245
+ guidance_scale=1.0,
246
+ is_amplify_first_chunk=True,
247
+ generator=torch.Generator("cuda").manual_seed(42),
248
+ ).frames[0]
249
+ export_to_video(output, "helios_distilled_i2v_output.mp4", fps=24)
250
+
251
+ # --- V2V ---
252
+ v2v_prompt = """
253
+ A bright yellow Lamborghini Huracn Tecnica speeds along a curving mountain road, surrounded by lush green trees
254
+ under a partly cloudy sky. The car's sleek design and vibrant color stand out against the natural backdrop,
255
+ emphasizing its dynamic movement. The road curves gently, with a guardrail visible on one side, adding depth to
256
+ the scene. The motion blur captures the sense of speed and energy, creating a thrilling and exhilarating atmosphere.
257
+ A front-facing shot from a slightly elevated angle, highlighting the car's aggressive stance and the surrounding greenery.
258
+ """
259
+ video_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/car.mp4"
260
+
261
+ output = pipeline(
262
+ prompt=v2v_prompt,
263
+ negative_prompt=negative_prompt,
264
+ video=load_video(video_path),
265
+ num_frames=240,
266
+ pyramid_num_inference_steps_list=[2, 2, 2],
267
+ guidance_scale=1.0,
268
+ is_amplify_first_chunk=True,
269
+ generator=torch.Generator("cuda").manual_seed(42),
270
+ ).frames[0]
271
+ export_to_video(output, "helios_distilled_v2v_output.mp4", fps=24)
272
+ ```
273
+
274
+ </details>
275
+
276
+ For example, let's take Helios-Distilled (**Modular Pipeline**).
277
 
278
  <details>
279
  <summary>Click to expand the code</summary>