topabaem commited on
Commit
f05c126
·
verified ·
1 Parent(s): a92edcc

Document self-contained loading and the guided modes

Browse files
Files changed (1) hide show
  1. README.md +32 -6
README.md CHANGED
@@ -83,17 +83,23 @@ V100 — the encoder fits one consumer card for the first time.
83
  ### Standalone
84
 
85
  ```python
 
86
  from ltx_packed_codec import load_packed_model
87
  from transformers import AutoTokenizer
88
 
89
- model = load_packed_model("path/to/encoder-hf", "A3.packed.safetensors",
90
- resident=True)
91
- tokenizer = AutoTokenizer.from_pretrained("path/to/encoder-hf")
 
 
 
92
  ```
93
 
94
- `encoder-hf` is a normal HF model directory for this encoder config,
95
- tokenizer and the assets that ship inside the original safetensors. Extract it
96
- with `ltx_extract_encoder_assets.py` from the Lightricks file.
 
 
97
 
98
  ### ComfyUI
99
 
@@ -113,6 +119,26 @@ CONDITIONING.
113
  > nodes themselves have not been loaded in a live ComfyUI, and that is a
114
  > different claim.
115
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
  ## How it was built
117
 
118
  nvfp4 4.5 bpw (E2M1, group 16 with an fp8-e4m3 scale) on 320 projections;
 
83
  ### Standalone
84
 
85
  ```python
86
+ from huggingface_hub import hf_hub_download, snapshot_download
87
  from ltx_packed_codec import load_packed_model
88
  from transformers import AutoTokenizer
89
 
90
+ repo = "topabaem/LTX-2.5-TextEncoder-nvfp4-portable"
91
+ packed = hf_hub_download(repo, "A3.packed.safetensors")
92
+ encoder_dir = snapshot_download(repo, allow_patterns=["encoder-hf/*"]) + "/encoder-hf"
93
+
94
+ model = load_packed_model(encoder_dir, packed, resident=True)
95
+ tokenizer = AutoTokenizer.from_pretrained(encoder_dir)
96
  ```
97
 
98
+ `encoder-hf/` here is config and tokenizer only, 31 MB. **The 26 GB original is
99
+ not needed**: `load_packed_model` builds the skeleton with
100
+ `AutoModel.from_config` and feeds every parameter from the packed file, so
101
+ nothing reads the Lightricks weights. The asset blobs that ship inside the
102
+ original safetensors are already in the packed file as raw BF16.
103
 
104
  ### ComfyUI
105
 
 
119
  > nodes themselves have not been loaded in a live ComfyUI, and that is a
120
  > different claim.
121
 
122
+ ## Image- and video-to-video
123
+
124
+ Both work, and both needed a fix ComfyUI does not ship. `LTXVAddGuide` is the
125
+ only producer of guided LTX latents and it cannot take an LTX-2.5 one: it calls
126
+ `torch.cat` on what is a `NestedTensor` for this model. The `ValueError` in that
127
+ function saying AV guides are unsupported never fires — `NestedTensor.shape`
128
+ proxies to the video half, whose channel count is exactly the 128 it checks for
129
+ — so the real failure is a `TypeError`, and the message is stale.
130
+
131
+ Everything below the node already supports AV guides: the model routes
132
+ `keyframe_idxs` to the video branch, the sampler pads a video-only denoise mask
133
+ with ones for audio, and `model_base` splits the packed mask apart again. So
134
+ `ltx_av_guide.py` unwraps the pair, runs the stock node on the video half and
135
+ re-wraps; the guide arithmetic stays the vendor's.
136
+
137
+ Measured on a 16 GB V100 at 512x320, 25 frames: t2v 98.8 s / 5.84 GiB, i2v
138
+ 62.7 s / 6.80 GiB, v2v 86.5 s / 5.84 GiB. An i2v first frame lands **relL2
139
+ 0.0766** from its guide image against **0.7061** for the same seed and prompt
140
+ without the guide, so the guide is honoured rather than merely accepted.
141
+
142
  ## How it was built
143
 
144
  nvfp4 4.5 bpw (E2M1, group 16 with an fp8-e4m3 scale) on 320 projections;