Instructions to use mlx-community/FastFrames-S-bf16-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/FastFrames-S-bf16-mlx with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download mlx-community/FastFrames-S-bf16-mlx --local-dir FastFrames-S-bf16-mlx
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
FastFrames-S, MLX port (bf16)
An unofficial community port of FastFrames-S to MLX, for text-to-video on Apple silicon Macs. It is not made, reviewed or endorsed by Apple, and it has no relationship with Apple.
Research use only, non-commercial. The KV refinement network and the pixel decoder are released under the Apple Machine Learning Research Model License, which allows use, modification and redistribution only for non-commercial scientific research. That applies to this port as a whole. See LICENSE-APPLE-AMLR before you use it.
Apple Machine Learning Research Model is licensed under the Apple Machine Learning Research Model License Agreement.
From a text prompt it makes a 5 second clip: 81 frames at 832x480, 16 fps. The pipeline has four parts:
- the umT5-XXL text encoder from Wan2.1, run once per prompt;
- the frame-wise causal Wan2.1-1.3B DiT, the two-step Causal Forcing++ generator, which denoises one latent frame (4 video frames) at a time in 2 steps against a KV cache of the frames before it;
- the FastFrames KV refinement network, which repairs each finished frame's cache entry instead of running a second clean DiT pass;
- the FastFrames causal pixel decoder, which turns the DiT features straight into RGB frames in place of the Wan VAE.
Samples
All were made with run.py at its default settings on an M3 Max. Click a preview for the
full-quality mp4. The previews are 416 px wide at 12 fps.
Prompts and seeds (also in the .json file next to each clip):
corgi-surfing_s0, seed 0: "A corgi surfing a wave at sunset"big-sur-coast_s75000225, seed 75000225: "A drone view of waves crashing against the rugged cliffs along Big Sur's Garay Point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore, casting long shadows. In the distance, a small island with a lighthouse stands tall, its beam piercing the twilight. Green shrubbery covers the cliff's edge, and the steep drop from the road down to the beach is a dramatic feat, with the cliff's edges jutting out over the sea. The camera angle provides a bird's-eye view, capturing the raw beauty of the coast and the rugged landscape of the Pacific Coast Highway. The scene is bathed in a warm, golden hue, highlighting the textures and details of the rocky terrain."llama-disco_s94000282, seed 94000282: "A realistic photo of a llama wearing colorful pajamas dancing energetically on a stage under vibrant disco lighting. The llama has large floppy ears and a playful expression, moving its legs in a lively dance. It wears a red and yellow striped pajama top and matching pajama pants, with a fluffy tail swaying behind it. The stage is adorned with glittering disco balls and colorful lights, casting a lively and joyful atmosphere. The background features blurred audience members and a backdrop with disco-themed decorations. A dynamic shot capturing the llama mid-dance from a slightly elevated angle."red-fox-snow_s3, seed 3: "A red fox trotting through fresh snow in a pine forest, morning light"
The corgi is a fair example of the model with a short prompt: the scene is right, but the dog is swimming rather than surfing. Long, descriptive prompts like the Big Sur and llama ones follow the text more closely.
Quick start
Needs an Apple silicon Mac with enough free memory for an 11 GB peak (tested on a 36 GB M3 Max),
Python 3 (tested with 3.13) and ffmpeg on the PATH. The download is about 15 GB.
pip install -U huggingface_hub
hf download mlx-community/FastFrames-S-bf16-mlx --local-dir FastFrames-S-bf16-mlx
cd FastFrames-S-bf16-mlx
pip install -r requirements.txt # brew install ffmpeg, if you don't have it
# one clip: prompt, seed, output file
python run.py "A red fox trotting through fresh snow in a pine forest, morning light" 3 fox.mp4
run.py looks for the weights next to itself, so it works from the downloaded folder as is. It
also takes --weights DIR, or FASTFRAMES_MLX_WEIGHTS, and if it finds no weights it fetches this
repo with snapshot_download. The umT5 tokenizer is downloaded from
Wan-AI/Wan2.1-T2V-1.3B on first use (a few MB). A
JSON line with the timings is printed and saved next to the mp4.
Useful options:
--frames Nsets the number of latent frames (default 21, which gives 81 video frames).Nlatent frames give4N - 3video frames, so--frames 5makes a 17-frame test clip.--referenceruns the plain port without the speed changes listed below, for A/B checks.--opts xcollapse=0turns off the cross-attention change, which makes the DiT bit-identical to the plain port.--opts attn=flash2uses the custom Metal attention kernel (about 5 % slower).
Several clips at once: batch.py loads the text encoder once for the whole batch, caches each
prompt's embedding, and writes each mp4 on a CPU thread while the GPU starts the next clip.
cat > prompts.txt <<'EOF'
A corgi surfing a wave at sunset
{"prompt": "A red fox trotting through fresh snow in a pine forest, morning light", "seeds": [3]}
EOF
python batch.py prompts.txt out/ --seeds 0,1
Text embeddings are cached in ~/.cache/fastframes-mlx/emb (set FASTFRAMES_MLX_CACHE to move
it), so a repeated prompt skips the 11 GB text encoder.
Accuracy
- In fp32, each MLX component (DiT pass, KV refiner, pixel decoder) matches Apple's PyTorch modules to about 1e-5 relative error.
- The default bf16 path gives the same video as the plain bf16 port to within 1/255 per pixel
(91-92 dB PSNR) when
xcollapseis off. The DiT and refiner outputs are then bit-identical; the only difference is the fp32 rounding in the decoder kernel. - With
xcollapseon (the default), the clips are no longer identical to the plain port, but they are as close to an fp32 run as the plain port is. PSNR of the decoded frames against the same clip with the DiT in fp32 (same noise), with the relative error of the DiT features in brackets:
| clip | plain bf16 port | default (xcollapse) | flash2 kernel |
|---|---|---|---|
| corgi, 25 frames | 24.71 (0.116) | 24.23 (0.118) | 24.03 (0.123) |
| Big Sur, 25 frames | 14.70 (0.253) | 14.18 (0.259) | 14.65 (0.255) |
| llama, 25 frames | 14.98 (0.290) | 14.95 (0.293) | 14.58 (0.305) |
| corgi, 81 frames | 24.62 (0.123) | 24.33 (0.121) | 24.02 (0.127) |
| Big Sur, 81 frames | 19.30 (0.167) | 20.99 (0.147) | 19.50 (0.166) |
| llama, 81 frames | 14.41 (0.329) | 14.43 (0.329) | 13.93 (0.334) |
| mean | 18.79 dB (0.213) | 18.85 dB (0.211) | 18.45 dB (0.218) |
These PSNR values look low because generation is chaotic. Each frame is generated from the cached frames before it, so a rounding difference in one frame changes the next one, and the bf16 and fp32 runs drift apart in motion detail while showing the same scene and subject. The table measures that spread, not a quality loss: all three bf16 variants sit at the same distance from fp32, and side by side the clips are equally clean, with no artefacts, blur or colour drift.
Performance
Measured on an Apple M3 Max with 36 GB of unified memory, one clip of 81 frames:
- About 55-61 s per clip for the rollout on a quiet machine, and about 90 s when other GPU work is running. Run-to-run noise is about 20 %.
- Peak memory about 11 GB: 11.1 GB while the text encoder runs, 10.7 GB during generation. The text encoder is unloaded before generation starts.
- Text encoding about 7 s per new prompt. Embeddings are cached, so repeats and batches skip it.
- Decoding the 81 frames takes about 1.8 s.
- Self-attention over the growing KV cache is most of the late-frame cost. MLX's attention kernel
runs at 8.2 TFLOP/s here, 85-89 % of the 9.2-9.5 TFLOP/s matrix-multiply peak measured on this
GPU, so there is little left to gain from a faster kernel. A custom flash-attention kernel is
included (
--opts attn=flash2) but is about 5 % slower.
Limitations
- Fine detail is soft, and faces and hands are often weak. This matches Apple's S model; it is not an effect of the port.
- English prompts only were tested. Long, detailed prompts in the style of Wan's prompt examples work best; short prompts give looser results.
- Clips are 5 seconds (81 frames at 16 fps). Longer clips are not supported.
- Tested only on an M3 Max. Machines with 16 GB of memory or less may not have room for the 11 GB peak.
Model Derivative and modifications
This repository is a Model Derivative of the Apple Machine Learning Research Model FastFrames-S, as defined in its licence. It contains no retraining or fine-tuning: every weight value is the released one. The changes are:
- Format. The PyTorch checkpoints were converted to MLX safetensors. The DiT, the KV refiner and the text encoder are bf16, as released; the pixel decoder is fp32, as released.
- Inference code. The pipeline was re-implemented in MLX (
ff_ref.pyis the plain port,ff_mlx.pythe faster default path,ff_kernels.pythe Metal kernels). It follows the structure of Apple's reference code in ml-fastframes. Inference runs in bf16, with norms and RoPE computed in fp32, as in the reference. - Fused QKV. The DiT's three self-attention projections are concatenated at load time into one matrix multiply. The output is bit-identical.
- Cross-attention padding collapse (
xcollapse, on by default). The prompt embedding is zero-padded to 512 tokens, so every padding position has the same key and value. The port attends to the real tokens plus one padding key weighted by log of the padding count, in fp32. This is mathematically equal to the original softmax but changes the rounding, so clips differ slightly from the plain port (see Accuracy). - Skipped last cache repair (
skiplast). The KV refinement for the final frame is never read by a later frame, so it is skipped. The output is bit-identical. - Decoder kernel. The pixel decoder's depthwise 3x3x3 causal convolution runs as one Metal kernel instead of 27 shifted adds (at most 1/255 per pixel difference).
- Other speed changes that do not alter the output:
mx.compileof elementwise chains and a per-prompt embedding cache.
Files
| file | size | what |
|---|---|---|
dit.safetensors |
2.84 GB | Causal Forcing++ frame-wise 2-step generator (Wan2.1-1.3B DiT), bf16 |
kv_refiner.safetensors |
0.69 GB | FastFrames-S KV refinement network, bf16 |
pixel_decoder.safetensors |
0.26 GB | FastFrames-S causal pixel decoder, fp32 |
umt5_xxl.safetensors |
11.36 GB | Wan2.1 umT5-XXL text encoder, bf16 |
run.py |
one clip from the command line | |
batch.py |
several prompts and seeds, text encoder loaded once | |
ff_mlx.py |
the default (fast) MLX pipeline | |
ff_ref.py |
the plain MLX port; ff_mlx.py builds on it |
|
ff_kernels.py |
Metal kernels: decoder depthwise conv, flash attention | |
requirements.txt |
Python packages | |
LICENSE-APPLE-AMLR |
Apple Machine Learning Research Model License (KV refiner, pixel decoder, this derivative) | |
LICENSE-APACHE-2.0 |
Apache License 2.0 (Causal Forcing++ generator, Wan2.1 umT5 encoder and tokenizer) | |
samples/ |
the sample clips (mp4), previews (gif) and their prompts and seeds (json) |
Licences
- KV refinement network and pixel decoder: from apple/FastFrames-S, Apple Machine Learning Research Model License (LICENSE-APPLE-AMLR, included verbatim). Research purposes only; no commercial use.
- DiT generator: the Causal Forcing++ frame-wise 2-step checkpoint from zhuhz22/Causal-Forcing, Apache License 2.0 (LICENSE-APACHE-2.0). Converted to MLX safetensors without other changes.
- Text encoder and tokenizer: umT5-XXL from Wan-AI/Wan2.1-T2V-1.3B, Apache License 2.0 (LICENSE-APACHE-2.0). The encoder was converted to MLX safetensors without other changes; the tokenizer is downloaded from the Wan repo at run time.
Because the pipeline includes the Apple components, use of this repository as a whole is limited to the research purposes the Apple licence allows.
Credits
- FastFrames by Gene Chou, Guandao Yang, Fujun Luan, Lars Mescheder, Wei Dong, Stephan R. Richter and Vladlen Koltun (Apple). Project page, code, model.
- Causal Forcing by Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li and Jun Zhu. Paper (arXiv:2602.02214), project page, code, model.
- Wan2.1 by the Wan team. Code, model.
If you use this port in research, please cite the original work:
@article{chou2026fastframes,
title = {FastFrames: Video Generation at Interactive Latency},
author = {Chou, Gene and Yang, Guandao and Luan, Fujun and Mescheder, Lars and
Dong, Wei and Richter, Stephan R. and Koltun, Vladlen},
journal = {arXiv preprint},
year = {2026}
}
@article{zhu2026causal,
title = {Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality
Real-Time Interactive Video Generation},
author = {Zhu, Hongzhou and Zhao, Min and He, Guande and Su, Hang and Li, Chongxuan and Zhu, Jun},
journal = {arXiv preprint arXiv:2602.02214},
year = {2026}
}
@article{wan2.1,
title = {Wan: Open and Advanced Large-Scale Video Generative Models},
author = {Wan Team},
year = {2025}
}
Quantized



