Update quick start docs and public prediction video workflow

README.md

@@ -11,16 +11,16 @@ tags:
 
 # VLAC2-Qwen3VL-30B-A3B-Progress
 
-VLAC2-Qwen3VL-30B-A3B-Progress is a video-language progress estimation model for robotic manipulation. Given a task description and a video, it predicts time-progress keypoints that can be aligned into a dense progress curve.
+VLAC2-Qwen3VL-30B-A3B-Progress is a video-language progress estimation model for robotic manipulation. Given a task description and a video, it predicts time-progress keypoints that can be aligned into a progress curve.
 
-This release contains the inference-ready model weights, tokenizer and processor files, three bundled demo episodes, and a minimal local inference example.
+This release contains the inference-ready model weights, tokenizer and processor files, three bundled demo episodes, reference prediction outputs, and minimal local inference utilities.
 
 ## Model
 
 - Base model: `Qwen/Qwen3-VL-30B-A3B-Instruct`
 - Architecture: `Qwen3VLMoeForConditionalGeneration`
 - Recommended VLAC model type in local evaluation scripts: `qwen3_moe_vl`
-- Bundled quick start uses the `chunk_all` prompt and samples the input video at `2 Hz`
+- Bundled quick start defaults to the `chunk_all` prompt and `2 Hz` video sampling
 
 ## Highlights
 
@@ -45,19 +45,46 @@ model = AutoModelForImageTextToText.from_pretrained(
 
 ## Quick Start
 
-Run one of the bundled examples from the model release root:
+Run progress inference on an arbitrary video:
 
 ```bash
-python quick_start/run_example.py --example-id example_01
-python quick_start/run_example.py --example-id example_02
-python quick_start/run_example.py --example-id example_03
+python quick_start/run_example.py \
+  --video-path /path/to/video.mp4 \
+  --task-instruction "把洋葱放进快递箱里。"
 ```
 
-For local VLAC benchmark scripts, use:
+Render a prediction-only preview video from the JSONL output:
 
 ```bash
-MODEL_PATH=/path/to/VLAC2-Qwen3VL-30B-A3B-Progress
-MODEL_TYPE=qwen3_moe_vl
+python quick_start/render_prediction_video.py \
+  --input-jsonl quick_start/outputs/video.jsonl
 ```
 
-More details are provided in `quick_start/README.md`.
+## Reproduce Bundled Examples
+
+You can reproduce the bundled reference outputs directly from the release root.
+
+Single example:
+
+```bash
+python quick_start/run_example.py \
+  --model-path /path/to/VLAC2-Qwen3VL-30B-A3B-Progress \
+  --metadata-path examples/example_01/metadata.json \
+  --output-jsonl quick_start/outputs/example_01.jsonl
+
+python quick_start/render_prediction_video.py \
+  --input-jsonl quick_start/outputs/example_01.jsonl \
+  --output-video quick_start/outputs/example_01_pred_progress.mp4
+```
+
+Verify consistency with the bundled reference outputs:
+
+```bash
+python quick_start/verify_prediction_consistency.py \
+  --pred-jsonl quick_start/outputs/example_01.jsonl \
+  --reference-jsonl examples/reference_outputs/example_01.jsonl \
+  --pred-video quick_start/outputs/example_01_pred_progress.mp4 \
+  --reference-video examples/reference_outputs/example_01_pred_progress.mp4
+```
+
+More detailed usage is provided in `quick_start/README.md`.

examples/reference_outputs/example_01_pred_progress.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:95c5e7bb99377d2844d15d4772313e749e8cd5a176ab03ca19ebc54a5ab30a7a
-size 2088505
+oid sha256:3ea6fb101f9823664cd7531ce5297f9490c40e6de7e5c1bb584b9d6fa5223301
+size 2043716

examples/reference_outputs/example_02_pred_progress.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:013ffd92c2eeecebbbf748f2bc543b5ad62d9b52635b52e186c6d2d5bf48811e
-size 2603165
+oid sha256:691fe25a519518ac2c24ba86c71ab7714d41f5616482cc0f079b874f897f679f
+size 2546146

examples/reference_outputs/example_03_pred_progress.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:684ef749e15396234157b0d09b536f0020b48b34eefdc26f8feb9b7f81d218de
-size 1119731
+oid sha256:211dfef38fd8e374f20b4bcf83a607fd6e3461fafb53a8228386995bbab118fe
+size 1436259

quick_start/README.md

@@ -1,7 +1,7 @@
 # Quick Start
 
 This quick start runs VLAC progress inference on an arbitrary input video.
-It samples the video at `2 Hz` by default, sends the sampled frames and prompt to the model, and writes one JSONL row with the raw response and parsed predicted progress points.
+It samples the input video at `2 Hz` by default, sends the sampled frames and prompt to the model, writes one JSONL row with the raw response and parsed predicted progress points, and can optionally render a predicted-progress preview video with a separate helper script.
 
 ## Requirements
 
@@ -12,11 +12,12 @@ It samples the video at `2 Hz` by default, sends the sampled frames and prompt t
 - `decord`
 - `numpy`
 - `Pillow`
+- `opencv-python` for preview video rendering
 - at least `1` GPU with enough memory for this `30B` checkpoint
 
-## Run
+## Generic Inference
 
-From the model release root:
+Run on an arbitrary input video with a task instruction:
 
 ```bash
 python quick_start/run_example.py \
@@ -24,7 +25,7 @@ python quick_start/run_example.py \
   --task-instruction "把洋葱放进快递箱里。"
 ```
 
-If you have a more detailed plan, pass it as text or file:
+If you have a more detailed task plan, pass it as text or file:
 
 ```bash
 python quick_start/run_example.py \
@@ -50,19 +51,70 @@ python quick_start/run_example.py \
   --task-instruction "把洋葱放进快递箱里。"
 ```
 
+## Use Bundled Examples
+
+The bundled examples already provide `metadata.json` files with task information, so you can reproduce the reference outputs directly from the release root.
+
+Single example:
+
+```bash
+python quick_start/run_example.py \
+  --model-path /path/to/VLAC2-Qwen3VL-30B-A3B-Progress \
+  --metadata-path examples/example_01/metadata.json \
+  --output-jsonl quick_start/outputs/example_01.jsonl
+
+python quick_start/render_prediction_video.py \
+  --input-jsonl quick_start/outputs/example_01.jsonl \
+  --output-video quick_start/outputs/example_01_pred_progress.mp4
+
+python quick_start/verify_prediction_consistency.py \
+  --pred-jsonl quick_start/outputs/example_01.jsonl \
+  --reference-jsonl examples/reference_outputs/example_01.jsonl \
+  --pred-video quick_start/outputs/example_01_pred_progress.mp4 \
+  --reference-video examples/reference_outputs/example_01_pred_progress.mp4
+```
+
+All three bundled examples:
+
+```bash
+for ex in example_01 example_02 example_03; do
+  python quick_start/run_example.py \
+    --model-path /path/to/VLAC2-Qwen3VL-30B-A3B-Progress \
+    --metadata-path "examples/${ex}/metadata.json" \
+    --output-jsonl "quick_start/outputs/${ex}.jsonl"
+
+  python quick_start/render_prediction_video.py \
+    --input-jsonl "quick_start/outputs/${ex}.jsonl" \
+    --output-video "quick_start/outputs/${ex}_pred_progress.mp4"
+
+  python quick_start/verify_prediction_consistency.py \
+    --pred-jsonl "quick_start/outputs/${ex}.jsonl" \
+    --reference-jsonl "examples/reference_outputs/${ex}.jsonl" \
+    --pred-video "quick_start/outputs/${ex}_pred_progress.mp4" \
+    --reference-video "examples/reference_outputs/${ex}_pred_progress.mp4"
+done
+```
+
 ## Output
 
-By default the script writes:
+By default the inference script writes:
 
 - `quick_start/outputs/<video_stem>.jsonl`
 
 Each output file contains one JSONL row with:
 
 - input video path
+- optional metadata fields when `--metadata-path` is used
 - sampled frame indices and timestamps
 - decoded video statistics
 - prompt text
 - raw model response
 - parsed predicted key points: `时间 / 进度`
 
-No ground truth, alignment, metrics, or comparison plots are produced by this quick start script.
+The preview video helper writes:
+
+- `<jsonl_stem>_pred_progress.mp4`
+
+The preview video shows the source video on top and predicted progress-so-far on the bottom.
+
+No ground truth, alignment, metrics, or comparison plots are produced by the inference script or preview renderer.

quick_start/render_prediction_video.py

@@ -0,0 +1,243 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import math
+from pathlib import Path
+
+import cv2
+import numpy as np
+
+THIS_DIR = Path(__file__).resolve().parent
+MODEL_ROOT = THIS_DIR.parent
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Render a predicted-progress preview video from a quick_start JSONL output.",
+    )
+    parser.add_argument(
+        "--input-jsonl",
+        type=Path,
+        required=True,
+        help="Path to the quick_start JSONL output.",
+    )
+    parser.add_argument(
+        "--video-path",
+        type=Path,
+        default=None,
+        help="Optional input video override. Defaults to the video_path stored in the JSONL row.",
+    )
+    parser.add_argument(
+        "--output-video",
+        type=Path,
+        default=None,
+        help="Optional output path. Defaults to <input_jsonl_stem>_pred_progress.mp4",
+    )
+    parser.add_argument(
+        "--panel-height",
+        type=int,
+        default=260,
+        help="Height in pixels for the lower progress panel.",
+    )
+    return parser.parse_args()
+
+
+def load_row(path: Path) -> dict:
+    lines = path.read_text(encoding="utf-8").splitlines()
+    if not lines:
+        raise SystemExit(f"Empty JSONL file: {path}")
+    return json.loads(lines[0])
+
+
+def resolve_video_path(path_arg: Path | None, row: dict) -> Path:
+    if path_arg is not None:
+        path = path_arg.resolve()
+        if not path.exists():
+            raise SystemExit(f"Missing video: {path}")
+        return path
+
+    row_video_path = row.get("video_path")
+    if not row_video_path:
+        raise SystemExit("Missing video_path in JSONL row. Pass --video-path explicitly.")
+    candidate = Path(str(row_video_path))
+    if candidate.is_absolute():
+        path = candidate
+    else:
+        path = (MODEL_ROOT / candidate).resolve()
+    if not path.exists():
+        raise SystemExit(f"Missing video resolved from JSONL row: {path}")
+    return path
+
+
+def render_preview(
+    *,
+    row: dict,
+    video_path: Path,
+    output_path: Path,
+    panel_height: int,
+) -> None:
+    pred_times = [float(x) for x in row.get("pred_curve_point_times_sec", [])]
+    pred_vals = [float(x) for x in row.get("pred_curve_point_progress", [])]
+    if not pred_times or not pred_vals:
+        raise SystemExit("No predicted progress points found in JSONL row.")
+
+    cap = cv2.VideoCapture(str(video_path))
+    if not cap.isOpened():
+        raise SystemExit(f"Failed to open video: {video_path}")
+
+    fps = float(cap.get(cv2.CAP_PROP_FPS))
+    frame_count = int(round(cap.get(cv2.CAP_PROP_FRAME_COUNT)))
+    frame_w = int(round(cap.get(cv2.CAP_PROP_FRAME_WIDTH)))
+    frame_h = int(round(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)))
+    if fps <= 0 or frame_count <= 0 or frame_w <= 0 or frame_h <= 0:
+        cap.release()
+        raise SystemExit(
+            f"Invalid decoded video stats from {video_path}: fps={fps}, frames={frame_count}, "
+            f"width={frame_w}, height={frame_h}"
+        )
+
+    total_h = frame_h + panel_height
+    writer = cv2.VideoWriter(str(output_path), cv2.VideoWriter_fourcc(*"mp4v"), fps, (frame_w, total_h))
+    if not writer.isOpened():
+        cap.release()
+        raise SystemExit(f"Failed to open output video writer: {output_path}")
+
+    duration = max(0.0, (frame_count - 1) / fps)
+    dense_t = np.arange(frame_count, dtype=np.float32) / fps
+    dense_v = np.interp(
+        dense_t,
+        np.array(pred_times, dtype=np.float32),
+        np.array(pred_vals, dtype=np.float32),
+        left=float(pred_vals[0]),
+        right=float(pred_vals[-1]),
+    )
+
+    y_min = min(-40.0, math.floor(min(pred_vals) / 10.0) * 10.0 - 10.0)
+    y_max = max(100.0, math.ceil(max(pred_vals) / 10.0) * 10.0 + 10.0)
+    left, right, top, bottom = 78, 24, 28, 52
+    chart_x0, chart_x1 = left, frame_w - right
+    chart_y0, chart_y1 = frame_h + top, total_h - bottom
+    tick_vals = np.linspace(y_min, y_max, num=6)
+    title = str(row.get("example_id") or video_path.stem)
+
+    def map_x(t: float) -> int:
+        if duration <= 1e-9:
+            return chart_x0
+        return int(round(chart_x0 + (chart_x1 - chart_x0) * (t / duration)))
+
+    def map_y(v: float) -> int:
+        ratio = (v - y_min) / max(1e-9, (y_max - y_min))
+        return int(round(chart_y1 - ratio * (chart_y1 - chart_y0)))
+
+    try:
+        idx = 0
+        while True:
+            ok, frame = cap.read()
+            if not ok:
+                break
+            if idx >= frame_count:
+                break
+
+            canvas = np.full((total_h, frame_w, 3), 255, dtype=np.uint8)
+            canvas[:frame_h, :, :] = frame
+
+            cv2.putText(
+                canvas,
+                f"{title} predicted progress so far",
+                (18, frame_h + 22),
+                cv2.FONT_HERSHEY_SIMPLEX,
+                0.65,
+                (20, 20, 20),
+                2,
+                cv2.LINE_AA,
+            )
+            cv2.rectangle(canvas, (chart_x0, chart_y0), (chart_x1, chart_y1), (210, 210, 210), 1)
+
+            for tick in tick_vals:
+                y = map_y(float(tick))
+                cv2.line(canvas, (chart_x0, y), (chart_x1, y), (238, 238, 238), 1)
+                cv2.putText(
+                    canvas,
+                    f"{int(round(tick))}",
+                    (12, y + 5),
+                    cv2.FONT_HERSHEY_SIMPLEX,
+                    0.44,
+                    (90, 90, 90),
+                    1,
+                    cv2.LINE_AA,
+                )
+
+            for frac in np.linspace(0.0, 1.0, num=5):
+                t_mark = duration * frac
+                x = map_x(float(t_mark))
+                cv2.line(canvas, (x, chart_y0), (x, chart_y1), (245, 245, 245), 1)
+                cv2.putText(
+                    canvas,
+                    f"{t_mark:.1f}s",
+                    (x - 16, total_h - 16),
+                    cv2.FONT_HERSHEY_SIMPLEX,
+                    0.42,
+                    (100, 100, 100),
+                    1,
+                    cv2.LINE_AA,
+                )
+
+            current_t = float(dense_t[idx])
+            current_v = float(dense_v[idx])
+            pts = [[map_x(float(dense_t[j])), map_y(float(dense_v[j]))] for j in range(idx + 1)]
+            if len(pts) >= 2:
+                cv2.polylines(canvas, [np.array(pts, dtype=np.int32)], False, (0, 140, 255), 3, cv2.LINE_AA)
+            elif len(pts) == 1:
+                cv2.circle(canvas, tuple(pts[0]), 3, (0, 140, 255), -1, cv2.LINE_AA)
+
+            for t, v in zip(pred_times, pred_vals):
+                if t <= current_t + 1e-6:
+                    cv2.circle(canvas, (map_x(t), map_y(v)), 5, (20, 20, 20), -1, cv2.LINE_AA)
+
+            cx, cy = map_x(current_t), map_y(current_v)
+            cv2.line(canvas, (cx, chart_y0), (cx, chart_y1), (120, 120, 120), 1)
+            cv2.circle(canvas, (cx, cy), 6, (0, 90, 220), -1, cv2.LINE_AA)
+            cv2.putText(
+                canvas,
+                f"time={current_t:.2f}s  pred={current_v:.1f}%",
+                (frame_w - 340, frame_h + 22),
+                cv2.FONT_HERSHEY_SIMPLEX,
+                0.55,
+                (30, 30, 30),
+                2,
+                cv2.LINE_AA,
+            )
+
+            writer.write(canvas)
+            idx += 1
+    finally:
+        cap.release()
+        writer.release()
+
+
+def main() -> int:
+    args = parse_args()
+    input_jsonl = args.input_jsonl.resolve()
+    if not input_jsonl.exists():
+        raise SystemExit(f"Missing JSONL: {input_jsonl}")
+    row = load_row(input_jsonl)
+    video_path = resolve_video_path(args.video_path, row)
+    output_path = args.output_video or input_jsonl.with_name(f"{input_jsonl.stem}_pred_progress.mp4")
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    render_preview(
+        row=row,
+        video_path=video_path,
+        output_path=output_path,
+        panel_height=args.panel_height,
+    )
+    print(f"input_jsonl={input_jsonl}")
+    print(f"video_path={video_path}")
+    print(f"output_video={output_path.resolve()}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

quick_start/run_example.py

@@ -7,6 +7,7 @@ import os
 import re
 import tempfile
 from pathlib import Path
+from typing import Any
 
 import numpy as np
 from PIL import Image
@@ -37,8 +38,14 @@ def parse_args() -> argparse.Namespace:
     parser.add_argument(
         "--video-path",
         type=Path,
-        required=True,
-        help="Path to the input video.",
+        default=None,
+        help="Path to the input video. Optional when --metadata-path is provided and metadata contains video_path.",
+    )
+    parser.add_argument(
+        "--metadata-path",
+        type=Path,
+        default=None,
+        help="Optional metadata JSON path. If provided, task fields and relative video_path can be inferred from it.",
     )
     parser.add_argument(
         "--task-instruction",
@@ -92,8 +99,9 @@ def parse_args() -> argparse.Namespace:
 
 
 def read_text_value(text_value: str | None, file_value: Path | None, *, field_name: str) -> str | None:
+    cli_name = field_name.replace("_", "-")
     if text_value is not None and file_value is not None:
-        raise SystemExit(f"Only one of --{field_name} and --{field_name}-file may be set.")
+        raise SystemExit(f"Only one of --{cli_name} and --{cli_name}-file may be set.")
     if file_value is not None:
         if not file_value.exists():
             raise SystemExit(f"Missing {field_name} file: {file_value}")
@@ -103,6 +111,29 @@ def read_text_value(text_value: str | None, file_value: Path | None, *, field_na
     return None
 
 
+def load_metadata(metadata_path: Path | None) -> tuple[Path | None, dict[str, Any] | None]:
+    if metadata_path is None:
+        return None, None
+    resolved = metadata_path.resolve()
+    if not resolved.exists():
+        raise SystemExit(f"Missing metadata JSON: {resolved}")
+    payload = json.loads(resolved.read_text(encoding="utf-8"))
+    if not isinstance(payload, dict):
+        raise SystemExit(f"Metadata JSON must contain an object: {resolved}")
+    return resolved, payload
+
+
+def extract_plan_text(task_instruction: str, task_description: str) -> str:
+    lines = [line.strip() for line in str(task_description or "").splitlines() if line.strip()]
+    if not lines:
+        return ""
+    first_line = lines[0].rstrip("：:")
+    normalized_instruction = str(task_instruction or "").strip().rstrip("：:")
+    if normalized_instruction and first_line == normalized_instruction:
+        lines = lines[1:]
+    return "\n".join(lines).strip()
+
+
 def build_chunk_all_prompt(task_instruction: str, task_plan: str | None) -> str:
     task_instruction = task_instruction.strip()
     task_plan = (task_plan or "").strip()
@@ -116,23 +147,64 @@ def build_chunk_all_prompt(task_instruction: str, task_plan: str | None) -> str:
     )
 
 
-def resolve_prompt(args: argparse.Namespace) -> tuple[str, str, str | None, str | None]:
+def resolve_prompt(args: argparse.Namespace, metadata: dict[str, Any] | None) -> tuple[str, str, str | None, str | None]:
     prompt = read_text_value(args.prompt, args.prompt_file, field_name="prompt")
-    task_plan = read_text_value(args.task_plan, args.task_plan_file, field_name="task-plan")
-    task_instruction = args.task_instruction.strip() if args.task_instruction else None
+    task_plan = read_text_value(args.task_plan, args.task_plan_file, field_name="task_plan")
+
+    metadata_task_instruction = None
+    metadata_task_plan = None
+    if metadata is not None:
+        metadata_task_instruction = str(metadata.get("task_instruction") or "").strip() or None
+        metadata_task_description = str(metadata.get("task_description") or "").strip()
+        if metadata_task_instruction and metadata_task_description:
+            metadata_task_plan = extract_plan_text(metadata_task_instruction, metadata_task_description) or None
+
+    task_instruction = args.task_instruction.strip() if args.task_instruction else metadata_task_instruction
+    if task_plan is None:
+        task_plan = metadata_task_plan
 
     if prompt:
         return prompt, "raw_prompt", task_instruction, task_plan
 
     if not task_instruction:
         raise SystemExit(
-            "Either provide --prompt/--prompt-file, or provide --task-instruction "
-            "with optional --task-plan/--task-plan-file."
+            "Either provide --prompt/--prompt-file, or provide --task-instruction, "
+            "or provide --metadata-path with task_instruction."
         )
 
     return build_chunk_all_prompt(task_instruction, task_plan), "task_instruction_plus_plan", task_instruction, task_plan
 
 
+def resolve_video_path(
+    video_path_arg: Path | None,
+    metadata: dict[str, Any] | None,
+) -> Path:
+    if video_path_arg is not None:
+        video_path = video_path_arg.resolve()
+        if not video_path.exists():
+            raise SystemExit(f"Missing video: {video_path}")
+        return video_path
+
+    if metadata is None or not metadata.get("video_path"):
+        raise SystemExit("Either provide --video-path, or provide --metadata-path with video_path.")
+
+    metadata_video_path = Path(str(metadata["video_path"]))
+    if metadata_video_path.is_absolute():
+        video_path = metadata_video_path
+    else:
+        video_path = (MODEL_ROOT / metadata_video_path).resolve()
+    if not video_path.exists():
+        raise SystemExit(f"Missing video resolved from metadata: {video_path}")
+    return video_path
+
+
+def maybe_relativize_to_model_root(path: Path) -> str:
+    try:
+        return str(path.resolve().relative_to(MODEL_ROOT.resolve()))
+    except ValueError:
+        return str(path.resolve())
+
+
 def compute_sampled_indices(num_frames: int, fps: float, sample_hz: float) -> tuple[list[int], list[float]]:
     if num_frames <= 0:
         raise SystemExit(f"Invalid decoded frame count: {num_frames}")
@@ -286,14 +358,14 @@ def parse_point_blocks(text: str) -> tuple[list[float], list[float]]:
 
 def main() -> int:
     args = parse_args()
-    video_path = args.video_path.resolve()
+    metadata_path, metadata = load_metadata(args.metadata_path)
     model_path = args.model_path.resolve()
-    if not video_path.exists():
-        raise SystemExit(f"Missing video: {video_path}")
     if not model_path.exists():
         raise SystemExit(f"Missing model path: {model_path}")
 
-    prompt, prompt_source, task_instruction, task_plan = resolve_prompt(args)
+    prompt, prompt_source, task_instruction, task_plan = resolve_prompt(args, metadata)
+    video_path = resolve_video_path(args.video_path, metadata)
+    video_path_for_output = maybe_relativize_to_model_root(video_path)
     output_path = args.output_jsonl or (THIS_DIR / "outputs" / f"{video_path.stem}.jsonl")
     output_path.parent.mkdir(parents=True, exist_ok=True)
 
@@ -329,23 +401,36 @@ def main() -> int:
         )[0].choices[0].message.content
 
     pred_point_times_sec, pred_point_progress = parse_point_blocks(response)
-    output_row = {
-        "video_path": str(video_path),
-        "task_instruction": task_instruction,
-        "task_plan": task_plan,
-        "prompt_source": prompt_source,
-        "prompt_variant": "chunk_all",
-        "input_sample_hz": float(args.sample_hz),
-        "input_frame_indices": sampled_indices,
-        "input_timestamps_sec": sampled_timestamps_sec,
-        "input_frame_count": len(sampled_indices),
-        "decoded_video_stats": video_stats,
-        "prompt": prompt,
-        "response": response,
-        "pred_curve_parse_ok": bool(pred_point_progress),
-        "pred_curve_point_times_sec": pred_point_times_sec,
-        "pred_curve_point_progress": pred_point_progress,
-    }
+
+    output_row: dict[str, Any] = {}
+    if metadata is not None:
+        for key in ("example_id", "bucket", "global_episode_id", "task_instruction", "task_description"):
+            if key in metadata:
+                output_row[key] = metadata.get(key)
+    if "task_instruction" not in output_row and task_instruction is not None:
+        output_row["task_instruction"] = task_instruction
+    if task_plan:
+        output_row["task_plan"] = task_plan
+    if metadata_path is not None:
+        output_row["metadata_path"] = maybe_relativize_to_model_root(metadata_path)
+
+    output_row.update(
+        {
+            "video_path": video_path_for_output,
+            "prompt_source": prompt_source,
+            "prompt_variant": "chunk_all" if prompt_source != "raw_prompt" else "custom",
+            "input_sample_hz": float(args.sample_hz),
+            "input_frame_indices": sampled_indices,
+            "input_timestamps_sec": sampled_timestamps_sec,
+            "input_frame_count": len(sampled_indices),
+            "decoded_video_stats": video_stats,
+            "prompt": prompt,
+            "response": response,
+            "pred_curve_parse_ok": bool(pred_point_progress),
+            "pred_curve_point_times_sec": pred_point_times_sec,
+            "pred_curve_point_progress": pred_point_progress,
+        }
+    )
     output_path.write_text(json.dumps(output_row, ensure_ascii=False) + "\n", encoding="utf-8")
 
     print(f"video_path={video_path}")

quick_start/verify_prediction_consistency.py

@@ -0,0 +1,148 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+from pathlib import Path
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Verify that a newly generated prediction JSONL/video matches a reference output.",
+    )
+    parser.add_argument("--pred-jsonl", type=Path, required=True, help="Newly generated prediction JSONL.")
+    parser.add_argument("--reference-jsonl", type=Path, required=True, help="Reference prediction JSONL.")
+    parser.add_argument("--pred-video", type=Path, default=None, help="Optional newly generated preview video.")
+    parser.add_argument("--reference-video", type=Path, default=None, help="Optional reference preview video.")
+    parser.add_argument("--tolerance", type=float, default=1e-6, help="Numeric tolerance for float comparisons.")
+    return parser.parse_args()
+
+
+def load_row(path: Path) -> dict:
+    lines = path.read_text(encoding="utf-8").splitlines()
+    if not lines:
+        raise SystemExit(f"Empty JSONL file: {path}")
+    return json.loads(lines[0])
+
+
+def get_alias_value(row: dict, *keys: str):
+    for key in keys:
+        if key in row:
+            return row[key]
+    return None
+
+
+def compare_numeric_lists(name: str, lhs, rhs, tolerance: float, errors: list[str]) -> None:
+    if lhs is None or rhs is None:
+        errors.append(f"{name}: missing on one side")
+        return
+    if len(lhs) != len(rhs):
+        errors.append(f"{name}: length mismatch {len(lhs)} != {len(rhs)}")
+        return
+    for idx, (a, b) in enumerate(zip(lhs, rhs, strict=True)):
+        if abs(float(a) - float(b)) > tolerance:
+            errors.append(f"{name}: mismatch at index {idx}: {a} != {b}")
+            return
+
+
+def sha256_file(path: Path) -> str:
+    h = hashlib.sha256()
+    with path.open("rb") as f:
+        while True:
+            chunk = f.read(1024 * 1024)
+            if not chunk:
+                break
+            h.update(chunk)
+    return h.hexdigest()
+
+
+def main() -> int:
+    args = parse_args()
+    pred_row = load_row(args.pred_jsonl.resolve())
+    ref_row = load_row(args.reference_jsonl.resolve())
+    errors: list[str] = []
+
+    pred_response = str(pred_row.get("response", "")).strip()
+    ref_response = str(ref_row.get("response", "")).strip()
+    if pred_response != ref_response:
+        errors.append("response: text mismatch")
+
+    pred_parse_ok = bool(pred_row.get("pred_curve_parse_ok"))
+    ref_parse_ok = bool(ref_row.get("pred_curve_parse_ok"))
+    if pred_parse_ok != ref_parse_ok:
+        errors.append(f"pred_curve_parse_ok: mismatch {pred_parse_ok} != {ref_parse_ok}")
+
+    compare_numeric_lists(
+        "pred_curve_point_times_sec",
+        pred_row.get("pred_curve_point_times_sec"),
+        ref_row.get("pred_curve_point_times_sec"),
+        args.tolerance,
+        errors,
+    )
+    compare_numeric_lists(
+        "pred_curve_point_progress",
+        pred_row.get("pred_curve_point_progress"),
+        ref_row.get("pred_curve_point_progress"),
+        args.tolerance,
+        errors,
+    )
+
+    compare_numeric_lists(
+        "input_frame_indices",
+        get_alias_value(pred_row, "input_frame_indices", "input_frame_indices_2hz"),
+        get_alias_value(ref_row, "input_frame_indices", "input_frame_indices_2hz"),
+        args.tolerance,
+        errors,
+    )
+    compare_numeric_lists(
+        "input_timestamps_sec",
+        get_alias_value(pred_row, "input_timestamps_sec", "input_timestamps_sec_2hz"),
+        get_alias_value(ref_row, "input_timestamps_sec", "input_timestamps_sec_2hz"),
+        args.tolerance,
+        errors,
+    )
+
+    pred_sample_hz = float(get_alias_value(pred_row, "input_sample_hz") or 0.0)
+    ref_sample_hz = float(get_alias_value(ref_row, "input_sample_hz") or 0.0)
+    if abs(pred_sample_hz - ref_sample_hz) > args.tolerance:
+        errors.append(f"input_sample_hz: mismatch {pred_sample_hz} != {ref_sample_hz}")
+
+    pred_frame_count = int(get_alias_value(pred_row, "input_frame_count", "input_frame_count_2hz") or 0)
+    ref_frame_count = int(get_alias_value(ref_row, "input_frame_count", "input_frame_count_2hz") or 0)
+    if pred_frame_count != ref_frame_count:
+        errors.append(f"input_frame_count: mismatch {pred_frame_count} != {ref_frame_count}")
+
+    if args.pred_video is not None or args.reference_video is not None:
+        if args.pred_video is None or args.reference_video is None:
+            errors.append("video comparison requires both --pred-video and --reference-video")
+        else:
+            pred_video = args.pred_video.resolve()
+            ref_video = args.reference_video.resolve()
+            if not pred_video.exists():
+                errors.append(f"Missing pred video: {pred_video}")
+            if not ref_video.exists():
+                errors.append(f"Missing reference video: {ref_video}")
+            if pred_video.exists() and ref_video.exists():
+                pred_sha = sha256_file(pred_video)
+                ref_sha = sha256_file(ref_video)
+                if pred_sha != ref_sha:
+                    errors.append(f"preview_video: sha256 mismatch {pred_sha} != {ref_sha}")
+
+    if errors:
+        print("CONSISTENCY_CHECK=FAILED")
+        for item in errors:
+            print(item)
+        return 1
+
+    print("CONSISTENCY_CHECK=PASSED")
+    print(f"pred_jsonl={args.pred_jsonl.resolve()}")
+    print(f"reference_jsonl={args.reference_jsonl.resolve()}")
+    if args.pred_video is not None and args.reference_video is not None:
+        print(f"pred_video={args.pred_video.resolve()}")
+        print(f"reference_video={args.reference_video.resolve()}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())