| # Elastic Forcing — 1.3B |
|
|
| [Overview](../README.md) · [14B guide](../14B/README.md) · [Checkpoints](https://huggingface.co/Elastic-forcing/Elastic-forcing-checkpoint/tree/main/checkpoints/1.3B) |
|
|
| The 1.3B model uses Wan2.1-T2V-1.3B with DINOv3, V-JEPA 2, and VideoMAEv2 for distribution matching. It generates 81-frame videos at 832 × 480 resolution and 16 FPS, with four causal denoising steps per chunk. |
|
|
| ## Environment |
|
|
| Use Linux, Python 3.10, and NVIDIA GPUs with BF16 support. Run these commands from `1.3B/`: |
|
|
| ```bash |
| conda create -n elastic-forcing python=3.10 -y |
| conda activate elastic-forcing |
| python -m pip install torch==2.5.1 torchvision==0.20.1 \ |
| --index-url https://download.pytorch.org/whl/cu121 |
| python -m pip install -r requirements.txt |
| python -m pip install flash-attn==2.7.4.post1 --no-build-isolation |
| export PYTHON_BIN="$(command -v python)" |
| ``` |
|
|
| ## Models and assets |
|
|
| Inference requires the Wan2.1-T2V-1.3B base model, T5 text encoder, tokenizer, VAE, and the Elastic Forcing generator checkpoint. Place the base-model bundle under `$ASSET_ROOT/models/Wan2.1-T2V-1.3B/`. Training also requires the Self-Forcing ODE initialization, local DINOv3, V-JEPA 2, and VideoMAEv2 models, and precomputed reference features. |
|
|
| Download the generator checkpoint: |
|
|
| ```python |
| from pathlib import Path |
| from huggingface_hub import snapshot_download |
| |
| folder = Path(snapshot_download( |
| "Elastic-forcing/Elastic-forcing-checkpoint", |
| allow_patterns=["checkpoints/1.3B/*"], |
| )) |
| print(folder / "checkpoints/1.3B/model.pt") |
| ``` |
|
|
| Set the local asset root and generate the configuration: |
|
|
| ```bash |
| export ASSET_ROOT=/path/to/assets |
| python scripts/prepare_config.py --asset-root "$ASSET_ROOT" |
| ``` |
|
|
| ## Inference |
|
|
| Use the downloaded checkpoint and a text file containing one prompt per line: |
|
|
| ```bash |
| CHECKPOINT=/path/to/model.pt \ |
| GPUS=0 \ |
| PROMPTS=code/prompts/test_8.txt \ |
| OUTPUT_DIR=outputs/demo \ |
| bash scripts/infer.sh |
| ``` |
|
|
| The launcher produces one video per prompt with seed 0 and non-EMA generator weights. Prepare base-model assets before launching; model loading uses local files. |
|
|
| ## Training |
|
|
| Training requires an ODE initialization, all three frozen encoders, a matched prompt/video manifest, a reference feature bank, and one Nyström summary per encoder. Prepare the model and reference assets at the locations specified in [configs/three_encoder.yaml](configs/three_encoder.yaml), generate the local configuration, and start training: |
|
|
| ```bash |
| GPUS=0,1,2,3 LOGDIR=runs/train bash scripts/train.sh |
| ``` |
|
|
| | Setting | Value | |
| |---|---| |
| | Training updates | 150 | |
| | Distribution / differentiation / reference batch | 256 / 128 / 128 | |
| | Per-GPU generated / differentiated videos | 64 / 32 | |
| | Reference videos | 8,682 | |
| | Encoder weights | DINOv3 : V-JEPA 2 : VideoMAEv2 = 10 : 1 : 1 | |
| | Loss divisor | 3 | |
| | Nyström / Monte Carlo weights | 2 : 1 | |
| | Optimizer | AdamW, betas (0, 0.999), weight decay 0 | |
| | Learning rate | Cosine from 2.5e-6 to 5e-7 over 125 updates, then constant | |
| | Gradient clipping | Maximum norm 10 | |
| | Checkpoint interval | Every five updates | |
|
|
| The launcher uses FSDP and gradient checkpointing. Training settings are in [configs/three_encoder.yaml](configs/three_encoder.yaml). Use a distinct `LOGDIR` for each run; checkpoints include optimizer and random-state files for resuming training. |
|
|
| ## Evaluation |
|
|
| | VBench Total ↑ | Quality ↑ | Semantic ↑ | |
| |---:|---:|---:| |
| | 84.25 | 85.06 | 80.99 | |
|
|
| Use [VBench](https://github.com/Vchitect/VBench) to evaluate generated videos. Per-dimension scores are in the [reference results](results/reference/three-encoder-summary.json). Run the CPU numerical tests from `1.3B/`: |
|
|
| ```bash |
| python -m pip install pytest |
| python scripts/test_cpu.py |
| ``` |
|
|
