Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -53,10 +53,37 @@ ffmpeg.
|
|
| 53 |
The LoRA is not a separate download: it is merged into the weights, so the file
|
| 54 |
IS the 4-step model.
|
| 55 |
|
| 56 |
-
**Text-to-video
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
## Running it
|
| 62 |
|
|
|
|
| 53 |
The LoRA is not a separate download: it is merged into the weights, so the file
|
| 54 |
IS the 4-step model.
|
| 55 |
|
| 56 |
+
**Text-to-video and keyframe-to-video.** Prompt in, video and audio out; or
|
| 57 |
+
give it a first and/or last frame and it continues from there. The release's
|
| 58 |
+
third path — `ref2va`, conditioning on reference images, clips and audio — is
|
| 59 |
+
not ported.
|
| 60 |
+
|
| 61 |
+
| file | size | what differs |
|
| 62 |
+
|---|---|---|
|
| 63 |
+
| `mmh3-turbo-fl2va-q4tp.cmf` | 23.94 GB | every projection at 4 bits |
|
| 64 |
+
| `mmh3-turbo-fl2va-q2tp.cmf` | 18.74 GB | the gate/up planes at 2 |
|
| 65 |
+
|
| 66 |
+
## Keyframe to video
|
| 67 |
+
|
| 68 |
+

|
| 69 |
+
|
| 70 |
+
```bash
|
| 71 |
+
cortiq animate mmh3-turbo-fl2va-q4tp.cmf \
|
| 72 |
+
--prompt "the corgi lifts the pan and flips the pancake high, sizzling" \
|
| 73 |
+
--first-frame keyframe.ppm --out flip.avi
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
One picture conditions the run twice, and both halves matter. Its VAE latent
|
| 77 |
+
becomes a row the DiT holds at a timestep of its own near 1 — a condition, not
|
| 78 |
+
noise being removed — and never denoises. The picture ITSELF goes to the prompt
|
| 79 |
+
encoder through Qwen3-VL's vision tower, as `"<Picture 1>: "` and a vision
|
| 80 |
+
block: at 512×288 that is 144 tokens of the 168 the prompt above carries.
|
| 81 |
+
Leave one out and the model is conditioned on something the reference never
|
| 82 |
+
conditions on.
|
| 83 |
+
|
| 84 |
+
`--last-frame` anchors the other end. The first frame is a geometry anchor and
|
| 85 |
+
is stretched to the canvas; the last one follows and is cover-cropped, which is
|
| 86 |
+
what the reference does with each. Frames come in as binary P6 PPM.
|
| 87 |
|
| 88 |
## Running it
|
| 89 |
|