How to use from the
Use from the
Diffusers library
pip install -U diffusers transformers accelerate
import torch
from diffusers import DiffusionPipeline

# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("inclusionAI/invoker-render-1.0", dtype=torch.bfloat16, device_map="cuda")

prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]

Code World Model 2.0 overview: diverse proxies, action-driven branching, adaptive proxy granularity, long-horizon 360ยฐ orbit, subject and camera control, and real gameplay proxies rendered by Invoker-Render

Code World Model 2.0: Towards Infinite Controllable Video Generation

Invoker-Render: proxy-conditioned video rendering

arXiv | Homepage | GitHub

Code World Model 2.0 keeps the world state in an upstream engine. Visual proxies fix the camera path, scene layout, and object motion, and a video model generates the appearance.

This repository hosts the causal streaming DiT weights of Invoker-Render. The model takes a structural proxy, one appearance reference image, and text. The companion CodeWorldModel inference code runs causal four-step streaming generation and is built on MiniMax-H3, SGLang, and SGLang-Omni.

Highlights

  • Long-horizon streaming: a fixed appearance anchor and a bounded history window let a session run for as long as proxy frames arrive.
  • Multi-granularity control: accepts semantic maps, sparse 3D boxes, subject masks, shaded geometry, and boxes with skeletons.
  • Interactive deployment: the companion code provides a WebSocket session service, a file client, and a browser client.

Weights

  • Repository root: main causal streaming DiT weights. 33.12B parameters stored in FP32, 29 safetensors shards with 638 tensors, about 132.5 GB, not quantized, and loadable as is.
  • config.json keeps the MiniMaxH3Transformer3DModel architecture identifier; the shard index is diffusion_pytorch_model.safetensors.index.json.
  • bidirectional_lora/teacher-live.safetensors: LoRA weights of the bidirectional teacher (about 2.3 GB), with MiniMax-H3 as the base model. It is a different model from the causal weights in the repository root; do not apply it on top of them.
  • This repository does not include the VAE, Qwen, the causal decoder, silence tensors, or a prepared inference bundle. Prepare matching assets separately.

Inference

Use the companion CodeWorldModel repository. Follow its README.md and docs/INFERENCE.md to set up the environment and prepare a bundle, then start the service or run file inference with invoker_render.py. The default production profile uses eight NVIDIA GPUs on one node.

The inputs are a 1344ร—768 RGB first image, a 24 FPS proxy video, and a text prompt. File inference requires 17*N+5 proxy frames, at least 39. 24 FPS is the frame rate of the video timeline, not inference throughput.

License

This repository is subject to Apache-2.0 and CC BY-NC 4.0 (non-commercial) terms.

Downloads last month
1,989
Safetensors
Model size
33B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Space using inclusionAI/invoker-render-1.0 1

Paper for inclusionAI/invoker-render-1.0