Update README.md

README.md

@@ -1,3 +1,39 @@
----
-license: gpl-3.0
----
+---
+license: gpl-3.0
+datasets:
+- nkp37/OpenVid-1M
+- TempoFunk/webvid-10M
+base_model:
+- VideoCrafter/VideoCrafter2
+pipeline_tag: text-to-video
+---
+# Advanced text-to-video Diffusion Models
+
+
+⚡️ This repository provides training recipes for the AMD efficient text-to-video models, which are designed for high performance and efficiency. The training process includes two key steps:
+
+* Distillation and Pruning: We distill and prune the popular text-to-video model [VideoCrafter2](https://github.com/AILab-CVC/VideoCrafter), reducing the parameters to a compact 945M while maintaining competitive performance.
+
+* Optimization with T2V-Turbo: We apply the [T2V-Turbo](https://github.com/Ji4chenLi/t2v-turbo) method on the distilled model to reduce inference steps and further enhance model quality.
+
+This implementation is released to promote further research and innovation in the field of efficient text-to-video generation, optimized for AMD Instinct accelerators.
+
+
+![pic](GIFs/vbench.png "Vbench performance")
+
+
+**4-Steps Results**
+| The camera follows behind a white vintage SUV with a black roof rack as it speeds up a steep dirt road surrounded by pine trees on a steep mountain slope.            | Waves crashing against a lone lighthouse, ominous lighting.               | A vibrant underwater scene. A group of blue fish, with yellow fins, are swimming around a coral reef.                | A young man in his twenties sat on a cloud in the sky and flipped through the book he was reading.               |
+|------------------------|-----------------------------|-----------------------------|-----------------------------|
+| <img src="GIFs/The_camera_follows_behind_a_white_vintage_SUV_with.gif" width="320">  | <img src="GIFs/Waves_crashing_against_a_lone_lighthouse,_ominous_.gif" width="320"> | <img src="GIFs/A_vibrant_underwater_scene._A_group_of_blue_fish,_.gif" width="320"> | <img src="GIFs/A_young_man_in_his_twenties_sat_on_a_cloud_in_the_.gif" width="320"> |
+
+**8-Steps Results**
+| A serene underwater scene featuring a sea turtle swimming through a coral reef.            | Animated scene features a close-up of a short fluffy monster kneeling beside a melting red candle. The art style is 3D and realistic, with a focus on lighting and texture.                | A close up view of a glass sphere that has a zen garden within it. There is a small dwarf in the sphere who is raking the zen garden and creating patterns in the sand.                | A drone camera circles around a beautiful historic church built on a rocky outcropping along the Amalfi Coast, the view showcases historic and magnificent architectural details and tiered pathways and patios, waves are seen crashing against the rocks below as the view overlooks the horizon of the coastal waters and hilly landscapes of the Amalfi Coast Italy, several distant people are seen walking and enjoying vistas on patios of the dramatic ocean views, the warm glow of the afternoon sun creates a magical and romantic feeling to the scene, the view is stunning captured with beautiful photography.               |
+|------------------------|-----------------------------|-----------------------------|-----------------------------|
+| <img src="GIFs/A_serene_underwater_scene_featuring_a_sea_turtle_s.gif" width="320">  | <img src="GIFs/Animated_scene_features_a_close-up_of_a_short_fluf.gif" width="320"> | <img src="GIFs/A_close_up_view_of_a_glass_sphere_that_has_a_zen_g.gif" width="320"> | <img src="GIFs/A_drone_camera_circles_around_a_beautiful_historic.gif" width="320"> |
+
+
+
+
+# License
+Copyright (c) 2024 Advanced Micro Devices, Inc. All Rights Reserved.