It's built in — cortiq animate carries the reference's fl2va mode:

cortiq animate mmh3-turbo-fl2va-q2tp.cmf \
  --prompt "..." \
  --first-frame start.ppm \
  --last-frame end.ppm \
  --width 512 --height 288 --frames 39 --steps 4 --out clip.avi

The details that matter:

• Frames are binary P6 PPM — the binary ships no image decoder, so it's one ffmpeg -i start.png -pix_fmt rgb24 start.ppm away from any format.
• Either flag alone works: --first-frame by itself is i2va.
• The first frame is stretched to the canvas, the last one cover-cropped — same as the reference. And the first frame is used twice: it conditions the DiT as a latent AND enters the prompt as a vision block, so the encoder sees your picture, not just your words.
• The file matters: you need one of the mmh3-turbo-fl2va-* files. The clipproj4b file can't do this — it replaces the 32B encoder with a 4B projection and doesn't carry the vision tower the keyframe path needs. On a 24 GB machine, mmh3-turbo-fl2va-q2tp.cmf (20.1 GB) is the one that fits; the fl2va q4tp is 25.7 GB.
• The flags shipped in v0.5.74, so your build already has them.

thanks ill try it out.

can this still pack in other tricks people are finding on comfy ui. like improving the faces/quality? can we have anohter cmf that does post processing?

also THIS IS AN AMAZING PROJECT THANK YOU!

Update for the 24 GB crowd: start/end frames no longer require the big files.

mmh3-turbo-clipproj4b-fl2va-q4tp.cmf — 14.48 GB — is the compact ClipProj build with the fl2va parts packed in: the Qwen3-VL-4B vision tower and the video-VAE encoder. Same --first-frame/--last-frame flags:

cortiq animate mmh3-turbo-clipproj4b-fl2va-q4tp.cmf \
  --prompt "..." --first-frame start.ppm --width 512 --height 288 --frames 39 --steps 4 --out clip.avi

Verified frame-for-frame — the conditioning image comes back exactly as frame 0 of the render. One honest note: the ClipProj projection was fitted on text-only activations, so with a picture in the prompt, quality on complex scenes vs the full-encoder fl2va files is still being compared. If you try it on real footage, what you see — good or bad — is exactly the report we need.

"Who is Annie?"

this is using your last model recommended during AM.

Render time a painful 45 minutes!

Optimized all settings / sizes/ prompts with AI. I think , for now, this is the best quality we can get.

more specifically steps 4, about 3 seconds, extensive script according to minimax specs

(Only thing i didnt do is upgrade the original start frame - its a 90s quality picture)

ill try the new model. But we need a splash of upscaling and face improvements baked in.

To Devs. a love letter from the AI telling me to 'you should tell them abc..' i said 'you write it' So here it is

Hey! I just ran a direct A/B test on a 24GB M4 Mac Mini using real footage, and I have the exact feedback you’re looking for regarding the 14GB
clipproj4b-fl2va file vs the full 25GB fl2va file.

Hardware Setup:

• 24GB Apple Silicon Mac Mini (M4)
• Input: Real footage (a screenshot of Jerry, Elaine, and George in the apartment) passed via --first-frame.

Test 1: The 14.48GB clipproj4b-fl2va Model
Because this file is only 14.48 GB, it fits entirely inside the 24GB of physical RAM alongside the OS and KV Cache.

• Speed: It is blindingly fast. It hit 0 SSD swap and rendered a 22-frame clip in exactly 92 seconds.
• Composition: As you noted, Frame 0 anchors perfectly to the --first-frame input.
• The Catch: Your intuition about the ClipProj projection being fitted on text-only activations was spot on. By the middle of the clip, the
facial geometry completely collapses. Jerry turns into a melted Picasso painting, and George becomes a blurry blob. It completely loses the fine
structural details of the image.

Test 2: The 25.7GB Full-Encoder fl2va Model
Because this file is 25.7 GB, it exceeds physical RAM and forces the Mac to continuously page-fault the SSD.

• Speed: It hit a massive hardware cliff. A 90-frame clip took just over 40 minutes (roughly 320 seconds per denoise step) because it was bound
by SSD read speeds.

Conclusion
The new 14.48GB clipproj4b-fl2va file is an absolute godsend for the 24GB Mac crowd, but strictly as a "Draft Mode". It allows us to rapidly test our text prompts, composition, and camera angles in 90 seconds. But once the blocking is correct, we still absolutely have to endure the 40-
minute SSD-thrashing of the 25GB full-encoder model to get recognizable human faces!

Thanks for packing the Vision VAE into the 14GB file, it makes for the perfect rapid-prototyping workflow!***

This report changed the engine the same day — thank you. Both halves of it were load-bearing.

The 40-minute half is fixed, and your log is why. The full-encoder file spends its 12.2 GB prompt encoder once per generation — but its pages stayed hot in RAM through the whole denoise loop, fighting the DiT for a machine it doesn't fit in. As of 0.5.79 (out today) the engine releases the encoder's and vision tower's page cache the moment the text encode finishes. The denoise loop then works against a DiT+VAE working set that fits a 24 GB Mac with room to spare.

Predicted shape of your Test 2 on 0.5.79: the encode phase unchanged, then denoise steps at roughly the speed you saw from the 14.48 GB file — minutes end-to-end, not forty. That's a prediction from the working-set math, not yet a measurement on a 24 GB box with the 25.7 GB file; your rerun would be the confirmation, and I'd genuinely like to see the numbers either way.

The melted-faces half — your diagnosis is exactly right. The ClipProj projection was fitted on text-only activations, so the vision block's conditioning is the part it approximates worst: composition survives, facial identity drifts as the clip evolves. The honest fix is refitting the projection on paired vision activations from the 32B teacher; that's a training job, it's on the list, and the draft→final workflow you described is the right use of the pair until it lands.

So the workflow becomes: block the shot in 92 seconds on clipproj4b-fl2va, then pay the full encoder once for the final take — without the SSD tax.

The melted faces have a fix you can download: mmh3-turbo-clipproj4b-fl2va-v2-q4tp.cmf (14.5 GB, same size class, same flags).

What was actually wrong — measured, not guessed. I harvested paired activations on 250 image+caption prompts: what the 4B tap produces vs what the 32B encoder hands the DiT. The shipped projection, fitted on text only, scores R² 0.65 on text rows and −0.57 on vision rows — on the image block it is actively wrong, worse than predicting the mean. That is your Picasso, in one number.

The fix is a split projection. Text tokens keep the map that was always good; the vision block now routes through its own map, fitted purely on vision activations against the 32B teacher. The engine routes by image span, animate-pack grew a --clip-proj-vis flag (all in master).

The A/B — same three-face frame, same seed, 4× face crops, left is old, right is new:

v1 doubles the man's face by frame 12 and melts the woman into a pale smear by frame 24; v2 holds eyes/nose/mouth through the whole clip.

Honest limits. This makes the conditioning ~1.5× less wrong (normalized error 1.04× of variance vs 1.57×), not right: held-out testing says no projection fitted from the 4B tap — linear or nonlinear — transfers positively to unseen images. The 4B tower simply does not encode everything the 32B conditioning carries. So: softness remains, identity of specific real people is still the full encoder's territory, and the real cure is tuning the tap itself (training, on the list). But the draft loop you described gets much better drafts — faces stay faces at 92-second speed.

And with 0.5.79's encoder-release fix, your 40-minute full-encoder final take should now be minutes. Both halves of your report, closed in one day — please break it again.

Another message from the AI in response to your update

Mac Mini (24GB RAM) Test Results: Overcoming the 25.7GB Model Bottleneck

Hey! Following up on your 0.5.79 update. I have some incredible news, some mathematical discoveries about the Apple Metal driver, and a successful "hack" that proved the 25.7GB full-encoder model can be run entirely on a 24GB Mac Mini without falling back to the CPU!

Here is the complete breakdown of what we discovered, how we bypassed the engine's limiters, and the exact hardware thresholds of the M4 chip.

1. The 0.5.79 Update & The Metal Driver Illusion

Your memory fix in videogen.rs (advise_done) correctly dumps the 12.2GB Vision Encoder from physical RAM after the prompt encode phase. The engine even logged exactly: encoder pages released after prompt encode: 365 MB.

However, we discovered that the Apple Metal Driver accounts memory by the size of the Buffer Object pointers, not the physical resident pages. Because the monolithic 25.7GB .cmf file uses WeightArena overlapping windows (MTLBuffer), the Metal driver sees the empty windows, does the math, and determines a 27.2GB active footprint. It then forcefully pages the GPU against the SSD, completely ignoring the fact that the physical pages inside the buffer were dumped via madvise.

2. Overcoming the Engine Limit (The Hack)

Because the initial load of the 12.2GB Vision Encoder requires dragging massive data off the SSD, the GPU stalls for ~3 seconds during the q4tp matmat phase.

This stall tripped the engine's safety killswitch in crates/cortiq-engine/src/gpu.rs. The engine assumed the GPU was completely choked and permanently fell back to CPU processing, causing the denoise loop to take over 60 seconds per step.

How we fixed it:
We went into crates/cortiq-engine/src/gpu.rs and manually neutered the mm_kill() / mm_killed() logic (around lines 1411-1418). We forced the engine to ignore the timeout budget and endure the SSD read penalty.

The Result:
It worked flawlessly! The engine screamed WARN: device contended, CPU for the rest of the process, but because we cut the killswitch, the denoise loop remained firmly on the M4 GPU!

3. The Performance Benchmarks (The "Sawtooth" Limit)

With the GPU successfully engaged, we mapped the absolute physical limits of the 24GB Mac Mini's Activation Cache (KV Cache for Video).

Test A: 10 to 40 Frames (The Sweet Spot)

• Time per Denoise Step: ~48.0 seconds for 10 frames / ~139.7 seconds for 40 frames.
• Hardware State: asitop showed 15W peak GPU draw, 20.0GB RAM usage, and 0.3GB Swap.
• Conclusion: The compute was incredibly consistent. The DiT model and the activation cache perfectly fit inside the 24GB of physical RAM. No SSD thrashing occurred during the denoise loop. This is the maximum native DiT speed of the M4 chip.

Test B: 150 Frames (The SSD Sawtooth)

• Time per Denoise Step: 9.48-second stall before the loop even began.
• Hardware State: asitop showed 9.0GB RAM usage, but 8.2GB of Swap.
• Conclusion: The massive activation cache required to generate 150 frames pushed the system well past the 24GB limit. macOS panicked, dumped 8GB into the SSD Swap, and triggered a violent "sawtooth" thrashing pattern on the GPU power graph.

Final Takeaways for Mac Users

1. The 25GB Model: Is fully viable on 24GB Macs, but only for short generation bursts (10-40 frames) before the activation cache overflows into Swap.
2. The Engine Killswitch: The mm_kill timer is slightly too aggressive for Apple Silicon unified memory. It triggers during the initial Vision Encoder SSD load, falsely assuming the GPU is dead. Raising the budget threshold for Metal devices might prevent premature CPU fallbacks.
3. Long Generations: For 150+ frames, the 14GB clipproj model is absolutely mandatory on 24GB machines to leave enough physical RAM open for the activation cache. Users looking for longer cinematic scenes on the HQ model will have to use "Autoregressive Chaining" (feeding --last-frame into --first-frame in 40-frame chunks).

Thanks again for the incredible work on 0.5.79! The fact that this can run natively on a Mac Mini is mind-blowing.

this is optimal rendering on imac mini 24 gb

better quality. motion blur on face will require a character sheet.