Update README.md
Browse files
README.md
CHANGED
|
@@ -5,7 +5,7 @@ base_model:
|
|
| 5 |
- bytedance-research/Phantom
|
| 6 |
---
|
| 7 |
|
| 8 |
-
Introduction
|
| 9 |
|
| 10 |
|
| 11 |
VACE is currently one of the most powerful tools currently available to be able to control AI video generations and Phantom is currently one of the best image reference to video for WAN. For the last few months I have worked on trying to combine the two together. I could have released earlier as even my earlier approaches were at least inconsistently working but I have spent time to find an approach that works most of the time.
|
|
@@ -15,7 +15,7 @@ I have split this article into two sections - one is a user guide for those of y
|
|
| 15 |
|
| 16 |
|
| 17 |
|
| 18 |
-
User Guide
|
| 19 |
|
| 20 |
|
| 21 |
There are several steps to using Phantom and this is not the be all and end all but an approach that seems to work.
|
|
@@ -29,7 +29,7 @@ There are several steps to using Phantom and this is not the be all and end all
|
|
| 29 |
4/It should be as simple as that - at least most of the time. The more you push the boundaries of what the model the more tweaking you might need to do.
|
| 30 |
|
| 31 |
|
| 32 |
-
Notes
|
| 33 |
|
| 34 |
I use the Load Video node to control the length of the animation and the dimensions - so depending on your wish you will need to change these to your needs.
|
| 35 |
Remember phantom takes at most 4 images to embed.
|
|
@@ -42,7 +42,7 @@ You likely can get away with as low as 6 steps but Phantom needs CFG > 1 and mor
|
|
| 42 |
Sometimes the results are a bit more blurry than base WAN - this is a result of the merge and if I find a solution I will update the model.
|
| 43 |
|
| 44 |
|
| 45 |
-
Troubleshooting
|
| 46 |
|
| 47 |
|
| 48 |
If you are not getting what you want it usually can be summarized as either the characters are not showing up or the motion is not being followed.
|
|
@@ -61,7 +61,7 @@ Try a different seed - some are much better than others.
|
|
| 61 |
Consider trying a different contolnet type like depth which will be much stronger and more consistent.
|
| 62 |
|
| 63 |
|
| 64 |
-
Technical Report
|
| 65 |
|
| 66 |
|
| 67 |
I am not alone in trying to merge VACE with things. VACE is well designed in that it is adding layers to WAN rather than being a fine tune which works in our favor here as otherwise we could not mix these models together.
|
|
@@ -74,7 +74,7 @@ Ill try to update this further as things come to mind.
|
|
| 74 |
|
| 75 |
|
| 76 |
|
| 77 |
-
In Closing
|
| 78 |
|
| 79 |
I hope you enjoyed this tutorial. If you did enjoy it please consider subscribing to my YouTube channel (https://www.youtube.com/@Inner-Reflections-AI) or my Instagram/Tiktok (https://linktr.ee/Inner_Reflections )
|
| 80 |
|
|
|
|
| 5 |
- bytedance-research/Phantom
|
| 6 |
---
|
| 7 |
|
| 8 |
+
**Introduction**
|
| 9 |
|
| 10 |
|
| 11 |
VACE is currently one of the most powerful tools currently available to be able to control AI video generations and Phantom is currently one of the best image reference to video for WAN. For the last few months I have worked on trying to combine the two together. I could have released earlier as even my earlier approaches were at least inconsistently working but I have spent time to find an approach that works most of the time.
|
|
|
|
| 15 |
|
| 16 |
|
| 17 |
|
| 18 |
+
**User Guide**
|
| 19 |
|
| 20 |
|
| 21 |
There are several steps to using Phantom and this is not the be all and end all but an approach that seems to work.
|
|
|
|
| 29 |
4/It should be as simple as that - at least most of the time. The more you push the boundaries of what the model the more tweaking you might need to do.
|
| 30 |
|
| 31 |
|
| 32 |
+
**Notes**
|
| 33 |
|
| 34 |
I use the Load Video node to control the length of the animation and the dimensions - so depending on your wish you will need to change these to your needs.
|
| 35 |
Remember phantom takes at most 4 images to embed.
|
|
|
|
| 42 |
Sometimes the results are a bit more blurry than base WAN - this is a result of the merge and if I find a solution I will update the model.
|
| 43 |
|
| 44 |
|
| 45 |
+
**Troubleshooting**
|
| 46 |
|
| 47 |
|
| 48 |
If you are not getting what you want it usually can be summarized as either the characters are not showing up or the motion is not being followed.
|
|
|
|
| 61 |
Consider trying a different contolnet type like depth which will be much stronger and more consistent.
|
| 62 |
|
| 63 |
|
| 64 |
+
**Technical Report**
|
| 65 |
|
| 66 |
|
| 67 |
I am not alone in trying to merge VACE with things. VACE is well designed in that it is adding layers to WAN rather than being a fine tune which works in our favor here as otherwise we could not mix these models together.
|
|
|
|
| 74 |
|
| 75 |
|
| 76 |
|
| 77 |
+
**In Closing**
|
| 78 |
|
| 79 |
I hope you enjoyed this tutorial. If you did enjoy it please consider subscribing to my YouTube channel (https://www.youtube.com/@Inner-Reflections-AI) or my Instagram/Tiktok (https://linktr.ee/Inner_Reflections )
|
| 80 |
|