Usage: switch reasnor examples to vLLM

README.md

@@ -622,9 +622,31 @@ The predicted per-frame relative poses are integrated into an absolute camera tr
 
 <img width="1280" src="assets/edge_action_id_av_1_output.png">
 
-### Reasoning
+### vLLM
 
-The reasoner generates text from a prompt and an optional image or video (`vision_path`); output is written to `reasoner_text.txt`.
+#### Container
+
+```bash
+docker pull vllm/vllm-openai:cosmos3
+```
+
+#### General Invocation
+
+You can use the `vllm` package to deploy the Cosmos3-Edge reasoner as an OpenAI-compatible API endpoint. The recommended vLLM serving configuration for `nvidia/Cosmos3-Edge` on a single GPU is:
+
+```bash
+vllm serve nvidia/Cosmos3-Edge \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --max-model-len 131072 \
+  --allowed-local-media-path / \
+  --mm-processor-kwargs '{"do_resize": true, "min_pixels": 4096, "max_pixels": 16777216}' \
+  --media-io-kwargs '{"video": {"fps": -1, "num_frames": 256}}'
+```
+
+#### Examples
+
+##### Reasoning
 
 Image input:
 
@@ -636,29 +658,47 @@ User prompt:
 The task is to put flower into the red bottle. Generate a plan consisting of subtasks for accomplish the task.
 ```
 
-```shell
-cat > reasoner_edge.json <<'JSON'
-{
-    "model_mode": "reasoner",
-    "prompt": "The task is to put flower into the red bottle. Generate a plan consisting of subtasks for accomplish the task.",
-    "vision_path": "https://huggingface.co/nvidia/Cosmos3-Edge/resolve/main/assets/example_reasoning_input.png"
-}
-JSON
+```python
+import base64
+import json
+from pathlib import Path
 
-python -m cosmos_framework.scripts.inference \
-    --parallelism-preset=latency \
-    -i reasoner_edge.json \
-    -o outputs/reasoner_edge \
-    --checkpoint-path Cosmos3-Edge \
-    --seed 0
-```
+import openai
 
-The generated text is written to `outputs/reasoner_edge/<sample_name>/reasoner_text.txt`.
+# 1. Read the image reasoning prompt
+example = json.load(open("assets/example_reasoning_prompt.json"))
+image_path = Path("assets/example_reasoning_input.png").resolve()
+image_url = (
+    "data:image/png;base64," + base64.b64encode(image_path.read_bytes()).decode()
+)
+
+# 2. Query the OpenAI-compatible vLLM server
+client = openai.OpenAI(
+    api_key="EMPTY",
+    base_url="http://localhost:8000/v1",
+)
+
+response = client.chat.completions.create(
+    model=client.models.list().data[0].id,
+    messages=[
+        {
+            "role": "user",
+            "content": [
+                {"type": "image_url", "image_url": {"url": image_url}},
+                {"type": "text", "text": example["prompt"]},
+            ],
+        },
+    ],
+    max_tokens=example["max_tokens"],
+)
+
+# 3. Print the default response (thinking is enabled)
+print(response.choices[0].message.content)
+```
 
 Example output:
 
 ```text
-<think>
 Got it, let's try to figure out how to put the flower into the red bottle. First, I need to identify the objects: the flower is the green one with the red center, and the red bottle is the red container. So the steps would involve moving the flower to the red bottle.
 
 First, the robot arm needs to locate the flower. Then, grasp the flower. After grasping, lift it, then move it towards the red bottle. Then, position the flower over the red bottle's opening, and finally release it into the bottle. Wait, but maybe I need to check if the red bottle has an opening that can accept the flower. Assuming the red bottle is a container with an opening, so the steps would be: move to flower, grasp, lift, move to red bottle, position over opening, release. Let me make sure each step is clear.
@@ -676,9 +716,11 @@ To accomplish the task of putting the flower into the red bottle, the plan shoul
 Each step ensures the flower is picked up, transported, and placed into the red bottle in a controlled manner.
 ```
 
-Thinking is enabled by default. Disabling it requires setting the reasoner chat template's `enable_thinking=False`, which the offline CLI does not currently expose — use the online serving path (`chat_template_kwargs={"enable_thinking": False}`) for a no-think response.
+Thinking is enabled by default. To disable it, add the following argument to the `client.chat.completions.create(...)` call:
 
-Guardrails are enabled by default (sourced from `nvidia/Cosmos-Guardrail1`); pass `--no-guardrails` to disable, or `--offload-guardrail-models` to keep them on CPU. For multi-GPU recipes, online Ray serving, and the full argument reference, see cosmos-framework `docs/inference.md`.
+```python
+extra_body={"chat_template_kwargs": {"enable_thinking": False}}
+```
 
 ## Limitations