Add SGLang serving instructions (#11)

- Super-squash branch 'main' using huggingface_hub (4f702f12a4166ddba0ff32fd4e128d390a625753)
- Add SGLang serving instructions (1ea4ae8c2d6ea184502a05834ec7bca234731957)
- Use SGLang main branch for Cosmos3 install (63e165c6d6e8cd4d9656c4b00f8e1a97686482af)
- change sglang serve section position (81166f28310d8ae96887523d1490aab2948bd804)
- add sglang to more sections (bc962f19ba1bddd410a0b03aa2d79dc7fd51137d)
- consolidate sglang serve section (2a579442c5df007ca48beff54d7a6fa56b9c97e3)
- Remove unrelated sound tokenizer files (41171bca665613c043f497d605969333c94f0013)
- Fix SGLang documentation links (1892515dd2c0db61066b7d9b4916c76ffb8c6091)


Co-authored-by: Ming-Yu Liu <mingyuliutw@users.noreply.huggingface.co>
Co-authored-by: mickqian <MickJ@users.noreply.huggingface.co>

README.md

@@ -10,54 +10,11 @@ tags:
   - cosmos3
   - vllm
   - vllm-omni
+  - sglang
+  - sglang-diffusion
   - diffusers
   - text, image, video, audio, and action generation
   - omnimodel
-countDownloads:
-  - checkpoint.json
-  - config.json
-  - generation_config.json
-  - model.safetensors.index.json
-  - model_index.json
-  - tokenizer.json
-  - tokenizer_config.json
-  - sound_tokenizer/config.json
-  - sound_tokenizer/diffusion_pytorch_model.safetensors
-  - text_tokenizer/tokenizer.json
-  - text_tokenizer/tokenizer_config.json
-  - transformer/config.json
-  - transformer/diffusion_pytorch_model-00001-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00002-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00003-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00004-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00005-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00006-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00007-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00008-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00009-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00010-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00011-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00012-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00013-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00014-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00015-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00016-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00017-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00018-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00019-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00020-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00021-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00022-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00023-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00024-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00025-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00026-of-00027.safetensors
-  - transformer/diffusion_pytorch_model-00027-of-00027.safetensors
-  - transformer/diffusion_pytorch_model.safetensors.index.json
-  - vae/config.json
-  - vae/diffusion_pytorch_model.safetensors
-  - vision_encoder/config.json
-  - vision_encoder/model.safetensors  
 ---
 
 # **Cosmos 3: Omnimodal World Models for Physical AI**
@@ -212,6 +169,7 @@ Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated sys
 - [PyTorch](https://github.com/nvidia/cosmos3)
 - [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
 - [Hugging Face Diffusers](https://huggingface.co/docs/diffusers/en/index)
+- [SGLang](https://github.com/sgl-project/sglang)
 
 **Supported Hardware Microarchitecture Compatibility:**
 
@@ -966,6 +924,71 @@ Example output:
 
 <video controls width="1280" height="720" src="https://huggingface.co/nvidia/Cosmos3-Super/resolve/main/assets/example_t2v_diffusers_output.mp4"></video>
 
+### SGLang
+
+[SGLang Diffusion](https://docs.sglang.io/docs/sglang-diffusion/index) can serve `nvidia/Cosmos3-Super` through OpenAI-compatible image and video generation endpoints. Install SGLang from the main branch with diffusion dependencies, then start the server:
+
+```bash
+git clone --branch main https://github.com/sgl-project/sglang.git
+cd sglang
+pip install -e "python[diffusion]"
+pip install "cosmos-guardrail==0.3.1"
+
+sglang serve \
+  --model-path nvidia/Cosmos3-Super \
+  --num-gpus 4
+```
+
+Cosmos 3 support in SGLang Diffusion currently requires the SGLang main branch. Switch to a stable SGLang release once Cosmos 3 support is included there.
+
+For the video-specialized checkpoint:
+
+```bash
+sglang serve \
+  --model-path nvidia/Cosmos3-Super-Image2Video \
+  --num-gpus 4
+```
+
+Supported SGLang endpoints:
+
+| Mode | Endpoint | Notes |
+| --- | --- | --- |
+| Text to image | `POST /v1/images/generations` | Returns base64 image data by default |
+| Text to video | `POST /v1/videos` | Creates an async job; poll `GET /v1/videos/{id}` and download `/content` |
+| Image to video | `POST /v1/videos` | Upload the conditioning image with `input_reference` |
+
+Example text-to-video request:
+
+```bash
+job_id=$(curl -sS -X POST http://localhost:30000/v1/videos \
+  --form-string "prompt=A small warehouse robot moves a blue box across a clean floor." \
+  --form-string "negative_prompt=blurry, distorted, low quality" \
+  --form-string "size=1280x720" \
+  --form-string "num_frames=81" \
+  --form-string "fps=24" \
+  --form-string "num_inference_steps=35" \
+  --form-string "guidance_scale=4.0" \
+  --form-string "flow_shift=10.0" \
+  --form-string "seed=42" \
+  --form-string 'extra_params={"guardrails":true,"use_resolution_template":false,"use_duration_template":false}' \
+  | python -c 'import json, sys; print(json.load(sys.stdin)["id"])')
+
+while true; do
+  status=$(curl -sS "http://localhost:30000/v1/videos/${job_id}" \
+    | python -c 'import json, sys; print(json.load(sys.stdin)["status"])')
+  [ "$status" = "completed" ] && break
+  [ "$status" = "failed" ] && exit 1
+  sleep 1
+done
+
+curl -sS -L "http://localhost:30000/v1/videos/${job_id}/content" \
+  -o cosmos3_super_t2v_output.mp4
+```
+
+Video-to-video, video-with-sound, and action generation are not supported by SGLang yet.
+
+For complete serving instructions and request examples, see the [Cosmos3 SGLang cookbook](https://docs.sglang.io/cookbook/diffusion/Cosmos/Cosmos3).
+
 ## Limitations
 
 Cosmos3 may produce imperfect outputs in challenging scenarios. Generation artifacts include temporal inconsistency, unstable camera or object motion, imprecise physical interactions, inaccurate audio-video synchronization, and action-state drift — especially in long-horizon or high-resolution outputs. Reasoning may also be incorrect: object states, causal relationships, spatial geometry, temporal ordering, agent intent, and future outcomes can be misinferred, and complex or long-context inputs may yield hallucinated entities, inconsistent interpretations, or implausible predictions. Because the model lacks an explicit physics simulator, 3D geometry, 4D space-time evolution, object permanence, contact dynamics, and physical laws are only approximated — producing artifacts such as disappearing or morphing objects, unrealistic collisions, and physically implausible motions. Quality further degrades in out-of-distribution environments, safety-critical edge cases, and domains underrepresented in training.
@@ -974,7 +997,7 @@ Cosmos3 outputs should not be treated as physically accurate simulation, reliabl
 
 ## Inference
 
-**Acceleration Engine:** [PyTorch](https://pytorch.org/), [vLLM](https://github.com/vllm-project/vllm), [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [Hugging Face Diffusers](https://github.com/huggingface/diffusers)
+**Acceleration Engine:** [PyTorch](https://pytorch.org/), [vLLM](https://github.com/vllm-project/vllm), [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [Hugging Face Diffusers](https://github.com/huggingface/diffusers), [SGLang](https://github.com/sgl-project/sglang), [SGLang Diffusion](https://docs.sglang.io/docs/sglang-diffusion/index)
 
 **Test Hardware:** GB200 and H100