Fix trtllm-serve command for TensorRT-LLM ToT Main

#2
by yiguo-nv - opened
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -109,11 +109,13 @@ This model was obtained by quantizing the weights and activations of Wan2.2-T2V-
109
 
110
  ## Usage
111
 
112
- To serve this checkpoint with [TRTLLM](https://github.com/NVIDIA/TensorRT-LLM):
 
 
113
 
114
  ```sh
115
- # TRTLLM
116
- trtllm-serve nvidia/Wan2.2-T2V-A14B-Diffusers-NVFP4 --extra_visual_gen_options ./examples/visual_gen/serve/configs/wan.yml
117
  # SGLang
118
  PROMPT='A cat and a dog baking a cake together in a cozy kitchen. The cat carefully measures flour while the dog stirs batter in a glass bowl, sunlight through the window, smooth cinematic camera motion.'
119
 
 
109
 
110
  ## Usage
111
 
112
+ To serve this checkpoint with [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/visual_gen/serve/README.md):
113
+
114
+ Docker image used for the TensorRT-LLM example: `nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc0`.
115
 
116
  ```sh
117
+ # TRTLLM (run from the root of the TensorRT-LLM source checkout)
118
+ trtllm-serve nvidia/Wan2.2-T2V-A14B-Diffusers-NVFP4 --visual_gen_args ./examples/visual_gen/serve/configs/wan22.yml
119
  # SGLang
120
  PROMPT='A cat and a dog baking a cake together in a cozy kitchen. The cat carefully measures flour while the dog stirs batter in a glass bowl, sunlight through the window, smooth cinematic camera motion.'
121