kabachuha commited on
Commit
891f28b
·
verified ·
1 Parent(s): 9c0a887

fix details in readme

Browse files
Files changed (1) hide show
  1. README.md +55 -48
README.md CHANGED
@@ -1,63 +1,69 @@
1
  ---
2
  tags:
3
- - text-to-image
4
  - lora
5
  - diffusers
6
  - template:diffusion-lora
7
  widget:
8
- - output:
9
- url: images/LTX2IV_00236-audio.webp
10
- text: >-
11
  The video opens on a cake. A knife, held by a hand, is coming into frame and
12
  hovering over the cake. The knife then begins cutting into the cake to c4k3
13
  cakeify it. As the knife slices the cake open, the inside of the cake is
14
  revealed to be cake with chocolate layers. The knife cuts through and the
15
- contents of the cake are revealed.
16
- - output:
17
- url: images/LTX2IV_00233-audio.webp
18
- text: >-
19
- The video opens on a cake. A knife, held by a hand, is coming into frame and
20
- hovering over the cake. The knife then begins cutting into the cake to c4k3
21
- cakeify it. As the knife slices the cake open, the inside of the cake is
22
- revealed to be cake with chocolate layers. The knife cuts through and the
23
- contents of the cake are revealed.
24
- - output:
25
- url: images/LTX2IV_00229-audio.webp
26
- text: >-
27
- The video opens on a cake. A knife, held by a hand, is coming into frame and
28
- hovering over the cake. The knife then begins cutting into the cake to c4k3
29
- cakeify it. As the knife slices the cake open, the inside of the cake is
30
- revealed to be cake with chocolate layers. The knife cuts through and the
31
- contents of the cake are revealed.
32
- - output:
33
- url: images/LTX2IV_00221-audio.webp
34
- text: >-
35
- The video opens on a cake. A knife, held by a hand, is coming into frame and
36
- hovering over the cake. The knife then begins cutting into the cake to c4k3
37
- cakeify it. As the knife slices the cake open, the inside of the cake is
38
- revealed to be cake with chocolate layers. The knife cuts through and the
39
- contents of the cake are revealed.
40
- - output:
41
- url: images/LTX2IV_00215-audio.webp
42
- text: >-
43
- The video opens on a cake. A knife, held by a hand, is coming into frame and
44
- hovering over the cake. The knife then begins cutting into the cake to c4k3
45
- cakeify it. As the knife slices the cake open, the inside of the cake is
46
- revealed to be cake with chocolate layers. The knife cuts through and the
47
- contents of the cake are revealed.
48
- - output:
49
- url: images/LTX2IV_00210-audio.webp
50
- text: >-
51
- The video opens on a cake. A knife, held by a hand, is coming into frame and
52
- hovering over the cake. The knife then begins cutting into the cake to c4k3
53
- cakeify it. As the knife slices the cake open, the inside of the cake is
54
- revealed to be cake with chocolate layers. The knife cuts through and the
55
- contents of the cake are revealed.
 
 
 
 
56
  base_model: Lightricks/LTX-2
57
  instance_prompt: c4k3 cakeify it
58
  license: other
59
  license_name: ltx-2-community-license-agreement
60
  license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
 
 
 
 
61
  ---
62
  # LTX-2 Cakeify
63
 
@@ -69,20 +75,21 @@ license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
69
 
70
  We all know the potential of a model reveals itself only when LoRAs are trained. This is my *third* LTX-2 LoRA after [Hydraulic press](https://huggingface.co/kabachuha/ltx2-hydraulic-press) and [Inflate it](https://huggingface.co/kabachuha/ltx2-inflate-it). It follows the quest of porting some the *classic* VFX LoRAs to LTX-2 (which has sound!).
71
 
72
- This is the second attempt, the first one was with only the attention being tuned. Here, following *inflate-it*, in addition to CREPA and TREAD, I also unfroze **FFN** ("ff.net.0.proj", "ff.net.2"), however I compensated for its parameter growth by lowerting the overall LoRA rank and the size gain was not that large compared to *H-press* (800 vs 640 Mb, and 1 Gb on *inflate*). The LoRA was trained in **2 hours 41 minutes** (1400 steps) on a single 5090, which I find is medium complexity, and the LoRA is pretty robust to various object types. If the object is stubborn, replace its name in the prompt with "cake" and it will be easier for the model to understand it.
73
 
74
  The LoRA, in contrast to *H-press* or *inflate-it*, has been trained on **pure, real videos** of objects which have had turned out to be a cake after being halved.
75
 
76
  The thing I love the most about LTX-2 is that you are able to set or generate any background music, be it a song or a greek style tune.
77
 
78
- The SimpleTuner training and dataset configs are under config.json and ltx2-multiresolution-inflate-t2v.json respectively.
79
 
80
  ## Trigger words
81
 
82
  You should use `c4k3 cakeify it` to trigger the image generation.
83
 
 
84
 
85
  ## Download model
86
 
87
 
88
- [Download](/kabachuha/ltx2-cakeify/tree/main) them in the Files & versions tab.
 
1
  ---
2
  tags:
3
+ - image-to-video
4
  - lora
5
  - diffusers
6
  - template:diffusion-lora
7
  widget:
8
+ - text: >-
 
 
9
  The video opens on a cake. A knife, held by a hand, is coming into frame and
10
  hovering over the cake. The knife then begins cutting into the cake to c4k3
11
  cakeify it. As the knife slices the cake open, the inside of the cake is
12
  revealed to be cake with chocolate layers. The knife cuts through and the
13
+ contents of the cake are revealed. Greek music playing in the background.
14
+ output:
15
+ url: video_examples/greek.mp4
16
+ - text: >-
17
+ The video opens on a toy penguin. A knife, held by a hand, is coming into
18
+ frame and hovering over the toy penguin. The knife then begins cutting into
19
+ the toy penguin to c4k3 cakeify it. As the knife slices the toy penguin
20
+ open, the inside of the toy penguin is revealed to be cake with chocolate
21
+ layers. The knife cuts through and the contents of the toy penguin are
22
+ revealed. The song: "Happy, happy penguin!" is sung in the background.
23
+ output:
24
+ url: video_examples/penguin.mp4
25
+ - text: >-
26
+ The video opens on three game die. A knife, held by a hand, is coming into
27
+ frame and hovering over the three game die. The knife then begins cutting
28
+ into the three game die to c4k3 cakeify them. As the knife slices the three
29
+ game die open, the insides of the three game die are revealed to be cakes
30
+ with chocolate layers. The knife cuts through and the contents of the three
31
+ game die are revealed.
32
+ output:
33
+ url: video_examples/die.mp4
34
+ - text: >-
35
+ The video opens on a glass globe. A knife, held by a hand, is coming into
36
+ frame and hovering over glass globe. The knife then begins cutting into the
37
+ glass globe to c4k3 cakeify it. As the knife slices the glass globe open,
38
+ the inside of the glass globe is revealed to be cake with chocolate layers.
39
+ The knife cuts through and the contents of the glass globe are revealed.
40
+ output:
41
+ url: video_examples/ball.mp4
42
+ - text: >-
43
+ The video opens on an anime girl. A knife, held by a hand, is coming into
44
+ frame and hovering over the anime girl. The knife then begins cutting into
45
+ the anime girl to c4k3 cakeify it. As the knife slices the anime girl open,
46
+ the inside of the anime girl is revealed to be cake with chocolate layers.
47
+ The knife cuts through and the contents of the anime girl are revealed.
48
+ output:
49
+ url: video_examples/fox.mp4
50
+ - text: >-
51
+ The video opens on a castle. A knife, held by a hand, is coming into frame
52
+ and hovering over the castle. The knife then begins cutting into the castle
53
+ to c4k3 cakeify it. As the knife slices the castle open, the inside of the
54
+ castle is revealed to be cake with chocolate layers. The knife cuts through
55
+ and the contents of the castle are revealed.
56
+ output:
57
+ url: video_examples/castle.mp4
58
  base_model: Lightricks/LTX-2
59
  instance_prompt: c4k3 cakeify it
60
  license: other
61
  license_name: ltx-2-community-license-agreement
62
  license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
63
+ datasets:
64
+ - finetrainers/cakeify-smol
65
+ language:
66
+ - en
67
  ---
68
  # LTX-2 Cakeify
69
 
 
75
 
76
  We all know the potential of a model reveals itself only when LoRAs are trained. This is my *third* LTX-2 LoRA after [Hydraulic press](https://huggingface.co/kabachuha/ltx2-hydraulic-press) and [Inflate it](https://huggingface.co/kabachuha/ltx2-inflate-it). It follows the quest of porting some the *classic* VFX LoRAs to LTX-2 (which has sound!).
77
 
78
+ This is the second attempt, the first one was with only the attention being tuned. Here, following *inflate-it*, in addition to CREPA and TREAD, I also unfroze **FFN** ("ff.net.0.proj", "ff.net.2"), however I compensated for its parameter growth by lowerting the overall LoRA rank and the size gain was not that large compared to *H-press* (830 vs 640 Mb, and 1 Gb on *inflate*). The LoRA was trained in **2 hours 41 minutes** (1400 steps) on a single 5090, which I find is medium complexity, and the LoRA is pretty robust to various object types. If the object is stubborn, replace its name in the prompt with "cake" and it will be easier for the model to understand it.
79
 
80
  The LoRA, in contrast to *H-press* or *inflate-it*, has been trained on **pure, real videos** of objects which have had turned out to be a cake after being halved.
81
 
82
  The thing I love the most about LTX-2 is that you are able to set or generate any background music, be it a song or a greek style tune.
83
 
84
+ The SimpleTuner training and dataset configs are under config.json and ltx2-multiresolution-cakeify-t2v.json respectively.
85
 
86
  ## Trigger words
87
 
88
  You should use `c4k3 cakeify it` to trigger the image generation.
89
 
90
+ Also, turn on the sound or prompt or add your own!
91
 
92
  ## Download model
93
 
94
 
95
+ [Download](/kabachuha/ltx2-cakeify/tree/main) them in the Files & versions tab.