Image-to-Video
Wan2.2
English
remind
remind-5b
video-generation
world-model
video-world-model
causal-video-generation
dynamic-memory
out-of-sight-state-evolution
teaching-video-generators-to-remember
lora
Instructions to use AppliedIntuitionResearch/ReMind with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Wan2.2
How to use AppliedIntuitionResearch/ReMind with Wan2.2:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Document Qwen prompt alignment
Browse filesAdd the first-frame Qwen3-VL prompt-alignment workflow and mark its documentation complete.
README.md
CHANGED
|
@@ -48,6 +48,12 @@ world state through camera motion, occlusion, and illumination changes.
|
|
| 48 |
This repository currently releases only the validated ReMind 5B weights. It
|
| 49 |
does not contain ReMind 1.3B weights.
|
| 50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
## Files
|
| 52 |
|
| 53 |
| File | Size | Purpose |
|
|
@@ -102,6 +108,46 @@ three-latent-frame chunks, and the shifted four-step schedule
|
|
| 102 |
`[1000, 938, 833, 625]`. Camera examples use the bundled pair-fixed GT camera
|
| 103 |
trajectories.
|
| 104 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
## Checkpoint lineage and validation
|
| 106 |
|
| 107 |
- ReMind-5B generator: pair-fixed 480p checkpoint after an additional 2,000
|
|
|
|
| 48 |
This repository currently releases only the validated ReMind 5B weights. It
|
| 49 |
does not contain ReMind 1.3B weights.
|
| 50 |
|
| 51 |
+
Release checklist:
|
| 52 |
+
|
| 53 |
+
- [x] Release ReMind-5B checkpoints
|
| 54 |
+
- [x] Document Qwen3-VL prompt alignment for custom inputs
|
| 55 |
+
- [ ] Release ReMind-1.3B checkpoints
|
| 56 |
+
|
| 57 |
## Files
|
| 58 |
|
| 59 |
| File | Size | Purpose |
|
|
|
|
| 108 |
`[1000, 938, 833, 625]`. Camera examples use the bundled pair-fixed GT camera
|
| 109 |
trajectories.
|
| 110 |
|
| 111 |
+
## Qwen3-VL prompt alignment for custom inputs
|
| 112 |
+
|
| 113 |
+
For a new first frame and an ordinary user request, we recommend using a local
|
| 114 |
+
`Qwen/Qwen3-VL-8B-Instruct` call before ReMind inference. Give Qwen only the
|
| 115 |
+
initial image and the user's text; do not provide future frames, evaluation
|
| 116 |
+
criteria, or a hand-written target caption. Use greedy decoding and ask for a
|
| 117 |
+
JSON object containing:
|
| 118 |
+
|
| 119 |
+
- `scene_caption`: an English, content-only description of the initial state,
|
| 120 |
+
physical progression, and requested final state;
|
| 121 |
+
- `control_requests`: any explicit `camera`, `occluder`, or `light` request,
|
| 122 |
+
with an exact supporting substring from the user text; and
|
| 123 |
+
- `clarification_questions`: unresolved intent or conflicting controls.
|
| 124 |
+
|
| 125 |
+
Preserve actions, negations, identities, directions, and stated end goals.
|
| 126 |
+
Keep camera motion and reversible visibility events out of `scene_caption`:
|
| 127 |
+
camera geometry belongs in a bundled trajectory, while occluder/light timing
|
| 128 |
+
belongs in the preset's `control` block. Never ask Qwen to invent camera
|
| 129 |
+
matrices or replace a bundled pair-fixed trajectory with an analytic path.
|
| 130 |
+
|
| 131 |
+
Copy the accepted `scene_caption` into the closest preset, configure at most
|
| 132 |
+
one supported control, and validate the resolved seven-chunk schedule before
|
| 133 |
+
loading weights:
|
| 134 |
+
|
| 135 |
+
```bash
|
| 136 |
+
python inference.py --preset /path/to/custom_case.yaml --dry-run
|
| 137 |
+
|
| 138 |
+
python inference.py \
|
| 139 |
+
--preset /path/to/custom_case.yaml \
|
| 140 |
+
--model-folder checkpoints/Wan2.2-TI2V-5B \
|
| 141 |
+
--base-checkpoint checkpoints/ReMind-5B/ReMind-5B.safetensors \
|
| 142 |
+
--ema-checkpoint checkpoints/ReMind-5B/ReMind-5b-dmd-ema.safetensors \
|
| 143 |
+
--output outputs/custom_case.mp4
|
| 144 |
+
```
|
| 145 |
+
|
| 146 |
+
Treat Qwen's rewrite and self-review as heuristic. Reject or manually review
|
| 147 |
+
outputs that drop a negation, invent an event, mix control language into the
|
| 148 |
+
scene caption, request multiple controls, or leave the intent ambiguous.
|
| 149 |
+
Fixed project-page captions should be used verbatim rather than rewritten.
|
| 150 |
+
|
| 151 |
## Checkpoint lineage and validation
|
| 152 |
|
| 153 |
- ReMind-5B generator: pair-fixed 480p checkpoint after an additional 2,000
|