ConstBob commited on
Commit
ffb36c2
·
verified ·
1 Parent(s): e6e06ab

docs: fix Diffusers FP8 modular restore

Browse files

Load the fp8 revision explicitly, enable ModelOpt checkpoint restoration, and preload the transformer from its full local path before loading the remaining modular components.

Files changed (1) hide show
  1. README.md +21 -3
README.md CHANGED
@@ -514,6 +514,7 @@ uv pip install \
514
  huggingface_hub \
515
  imageio \
516
  imageio-ffmpeg \
 
517
  torch \
518
  torchvision \
519
  transformers
@@ -523,18 +524,35 @@ The safety checker requires access to the gated [nvidia/Cosmos-1.0-Guardrail](ht
523
 
524
  ### Example: Text to Image Generation
525
 
 
 
526
  ```python
527
  import json
 
528
 
 
529
  import torch
530
- from diffusers import Cosmos3DistilledModularPipeline
 
 
531
 
532
  json_prompt = json.load(open("assets/example_caption.json"))
533
  json_prompt["resolution"] = {"H": 768, "W": 768}
534
  json_prompt["aspect_ratio"] = "1,1"
535
 
536
- pipe = Cosmos3DistilledModularPipeline.from_pretrained("nvidia/Cosmos3-Super-Text2Image-4Step")
537
- pipe.load_components(torch_dtype=torch.bfloat16)
 
 
 
 
 
 
 
 
 
 
 
538
  pipe.enable_safety_checker()
539
  pipe.to("cuda")
540
 
 
514
  huggingface_hub \
515
  imageio \
516
  imageio-ffmpeg \
517
+ nvidia-modelopt==0.44.0 \
518
  torch \
519
  torchvision \
520
  transformers
 
524
 
525
  ### Example: Text to Image Generation
526
 
527
+ Enable ModelOpt checkpoint restoration before loading the FP8 transformer. ModelOpt 0.44 does not use a Hub `subfolder` when locating `modelopt_state.pth`, so the example materializes the `fp8` revision, loads `transformer/` from its full local path, and pins it before loading the remaining modular components.
528
+
529
  ```python
530
  import json
531
+ from pathlib import Path
532
 
533
+ import modelopt.torch.quantization.backends.fp8_per_tensor_gemm # noqa: F401
534
  import torch
535
+ from diffusers import AutoModel, Cosmos3DistilledModularPipeline
536
+ from huggingface_hub import snapshot_download
537
+ from modelopt.torch.opt import enable_huggingface_checkpointing
538
 
539
  json_prompt = json.load(open("assets/example_caption.json"))
540
  json_prompt["resolution"] = {"H": 768, "W": 768}
541
  json_prompt["aspect_ratio"] = "1,1"
542
 
543
+ model_id = "nvidia/Cosmos3-Super-Text2Image-4Step"
544
+ revision = "fp8"
545
+ enable_huggingface_checkpointing()
546
+
547
+ pipe = Cosmos3DistilledModularPipeline.from_pretrained(model_id, revision=revision)
548
+ checkpoint_dir = Path(snapshot_download(model_id, revision=revision)).resolve()
549
+ transformer_dir = checkpoint_dir / "transformer"
550
+ modelopt_state = transformer_dir / "modelopt_state.pth"
551
+ if not modelopt_state.is_file():
552
+ raise RuntimeError(f"FP8 requested but no ModelOpt state at {modelopt_state}")
553
+ transformer = AutoModel.from_pretrained(str(transformer_dir), torch_dtype=torch.bfloat16)
554
+ pipe.update_components(transformer=transformer)
555
+ pipe.load_components(pretrained_model_name_or_path=str(checkpoint_dir), torch_dtype=torch.bfloat16)
556
  pipe.enable_safety_checker()
557
  pipe.to("cuda")
558