PedroMarinhoDev/Hy-Nomas-ComfyUI

Hy-Nomas for ComfyUI

Nodes on GitHub · Samples · Quick start · The original model

Tencent Hy-Nomas, the video world model, packaged for the native ComfyUI nodes of ComfyUI-Hy-Nomas: a world generated one segment at a time, each segment continuing from the model's memory of the earlier ones, steered by text, timed actions and the camera. Start it from a prompt or from your own image, chain segments into long videos, or drive it with the keyboard turn by turn. These are Tencent's INT8 distilled DiT and FP8 decoder, unchanged, under ComfyUI file names; with the nodes they run on 12 GB cards.

Work in progress. The nodes work end to end and are checked against Tencent's code, but they are a first release: expect rough edges and changing node inputs. More features (Tencent's other checkpoints and LoRAs, a prompt helper) and optimizations (faster decoding, closer to real time drive mode) are planned. See the roadmap.

Overview

A 68 s tour of the nodes: start from an image, chained segments, camera keys, drive mode, speed.

Samples

20 s each at 1280x704, 24 fps: four 5 s segments chained, each with its own text. RTX 4090, about 3 minutes each.

From an image: Paris. Third-person follow: she walks down to the river road, stops at the wall to look across at the bridge, the sun breaks through behind the tower, she walks on.

From an image: a beach. First person: past the fire pit to the water's edge, a turn along the shore, onto the deck by the hut. The image's heavy color grade holds for the whole walk.

From text only: the fox. It walks the path, stops and turns to the camera, then trots off into the forest as sunbeams break through the fog.

Camera keys: an orbit around her. The same Paris picture; Camera Keys D+Left 3 on three segments (strafe right while turning left, so the camera circles her as she stands still), then S 4 as she walks toward it.

Drive mode. Six 1 s turns at 832x480 steered with W, W+Left, W, W+Right, W, W in the Drive node; each turn took about 4.3 s on the RTX 4090.

_{Every 2 s of each sample; the first frame of the top two rows is the start image. The worlds (prompts) are in the node pack's examples/.}

Files

File	Size	Folder	What
hy_nomas_distil_int8.safetensors	9.06 GB	models/diffusion_models	the 8-step distilled DiT, INT8 weights (Tencent's single-GPU checkpoint)
hy_nomas_pixelumm_fp8.safetensors	9.13 GB	models/vae	the fine-tuned PixelUMM video decoder, FP8, plus the DCAE-V encoder (used to start from an image). Non-commercial
qwen_2.5_vl_7b_fp8_scaled.safetensors (or the bf16 qwen_2.5_vl_7b)	9.4 GB	models/text_encoders	the text encoder, from Comfy-Org/Qwen-Image_ComfyUI; load it with Load CLIP, type qwen_image

Quick start

1. Install the nodes: clone ComfyUI-Hy-Nomas into ComfyUI/custom_nodes and restart ComfyUI. Nothing to pip install.

2. Download the files:

   hf download PedroMarinhoDev/Hy-Nomas-ComfyUI hy_nomas_distil_int8.safetensors --local-dir ComfyUI/models/diffusion_models
   hf download PedroMarinhoDev/Hy-Nomas-ComfyUI hy_nomas_pixelumm_fp8.safetensors --local-dir ComfyUI/models/vae
   hf download Comfy-Org/Qwen-Image_ComfyUI split_files/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors --local-dir ComfyUI/models/text_encoders
   

3. Open a workflow from the pack's workflows/ folder: a world one segment at a time, the same in one node, start from image, camera control, or the interactive drive mode. Each has a Read-me note.

Settings are the distilled model's: euler / simple, 8 steps, cfg 1, 24 fps, 1280x704 (Tencent's 720p) or 832x480. Write the world prompt in the format Tencent's prompt rewriter produces (<|Subject1|>, <Action1>...</Action1>, follow-ups starting "Compared to the previous segment, "): loosely written follow-up segments are mostly ignored. The node pack's README explains the format, the nodes, speed and memory.

What changed

Nothing in the tensors: the files are Tencent's, byte for byte, renamed for ComfyUI's model folders.

This repo	tencent/Hy-Nomas	sha256
hy_nomas_distil_int8.safetensors	DiT/distilled_int8/diffusion_pytorch_model_ema_0_995_int8.safetensors	973fa409f3b009232db3c83feddc89c5326fc3a5672feb791f0989ea8bf83252
hy_nomas_pixelumm_fp8.safetensors	VAE/PixelUMM_FT_FP8/diffusion_pytorch_model_fp8.safetensors	5ada04d7cedf9fe95bbaf87b4a8c0334944b312af5571568853b37d9b7256d78

So Tencent's files work with the nodes too, under any name.

Credits and license

Hy-Nomas is developed and released by Tencent (tencent/Hy-Nomas, code); all credit for the model goes to its authors. The DiT is under the Apache License 2.0 (LICENSE-APACHE-2.0).

The video decoder is Tencent's fine-tune of NVIDIA's PixelUMM. PixelUMM weights and their derivatives are under the NVIDIA One-Way Noncommercial License (LICENSE-NVIDIA-ONEWAY-NONCOMMERCIAL.txt): research and evaluation only, no commercial use. LICENSE lists which file is under which license; NOTICE has the attributions.