RynnValue-8B demo: reference prefix-uniform pipeline, examples, gradio 5.50

.gitattributes

@@ -40,3 +40,5 @@ examples/franka_put_box_in_drawer.mp4 filter=lfs diff=lfs merge=lfs -text
 examples/so100_pick_up_the_cube_and_place_it_in_the_box.mp4 filter=lfs diff=lfs merge=lfs -text
 assets/franka_box_into_drawer.mp4 filter=lfs diff=lfs merge=lfs -text
 assets/so101_lego_into_box.mp4 filter=lfs diff=lfs merge=lfs -text
+examples/franka_box_into_drawer.mp4 filter=lfs diff=lfs merge=lfs -text
+examples/so101_lego_into_box.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,40 +1,62 @@
 ---
-title: RynnValue 8B
+title: RynnValue-8B
 emoji: 🦾
 colorFrom: yellow
 colorTo: gray
 sdk: gradio
-sdk_version: 6.24.0
+sdk_version: 5.50.0
 app_file: app.py
 short_description: How much longer will this robot take?
 python_version: "3.12"
 startup_duration_timeout: 1h
 pinned: false
+license: apache-2.0
+models:
+  - Alibaba-DAMO-Academy/RynnValue-8B
+tags:
+  - robotics
+  - value-model
+  - video
 ---
 
 # RynnValue-8B
 
 Demo of [`Alibaba-DAMO-Academy/RynnValue-8B`](https://huggingface.co/Alibaba-DAMO-Academy/RynnValue-8B),
 a ~9.6B robotic value model built on Qwen3-VL. Given a robot manipulation video and the task
-instruction, it predicts the **remaining time to completion** at every point in the clip and writes
-a short analysis block (video description, instruction match, success).
+instruction, it predicts the **remaining time to task completion (seconds)** at every point along
+the clip — rendered as a curve synchronised with the video — plus a short **Analysis** block
+(video description, instruction `Match`, task `Success`).
 
 The app follows the official reference implementation
 ([`rynn_infer/inference.py`](https://github.com/alibaba-damo-academy/RynnValue/blob/main/rynn_infer/inference.py)):
 prefix-uniform sampling, where evaluation step *i* resamples `frames[0:i]` to `num_frames` frames
-and reads the model's **last** prediction slot, so each score only conditions on frames seen so far.
-`plot_utils.py` is vendored from the same repo (Apache-2.0) with the trend plot memoised per
-prediction step for speed.
+and reads the model's **last** prediction slot, so each score only conditions on frames seen so
+far; the custom `pred_slot_isolated_eager` attention implementation; and a greedy Analysis
+generation pass over the full-video prefix. `plot_utils.py` is vendored from the same repo
+(Apache-2.0) with the trend plot memoised per prediction step (it is otherwise re-rendered once
+per output video frame).
 
 Deviations from the reference, forced by the ZeroGPU time budget: fewer evaluation steps
-(16 vs one-per-frame), fewer frames per step (16 vs 64), smaller frames (448 px vs 640 px), and the
-rendered video is temporally subsampled to ≤420 frames (playback fps scaled to match, so durations
+(16 vs one-per-frame), fewer frames per step (32 vs 64), smaller frames (384 px vs 640 px), and the
+rendered video is temporally subsampled to ≤320 frames (playback fps scaled to match, so durations
 and the ground-truth reference curve are unchanged). All are adjustable in *Advanced settings*.
 
-## Example assets
+`transformers` is pinned to 4.57.x (what the checkpoint's remote code targets), which requires
+`huggingface-hub<1.0` — hence the Gradio 5.x SDK version.
 
-- `assets/franka_box_into_drawer.mp4` — the sample clip shipped with the official RynnValue repo
-  (Apache-2.0), transcoded to 640×360.
-- `assets/so101_lego_into_box.mp4` — episode 1 of
+## Example assets & attribution
+
+- `examples/franka_box_into_drawer.mp4` — the demo clip bundled with
+  [alibaba-damo-academy/RynnValue](https://github.com/alibaba-damo-academy/RynnValue)
+  (Apache-2.0), re-encoded to 640 px.
+- `examples/soar_put_green_stick_in_brown_bowl.mp4`, `examples/berkeley_rpt_stack_cup.mp4`,
+  `examples/jaco_play_pick_up_green_cup.mp4` — RoboMeter benchmark clips bundled in the same
+  repository (MIT), originating from
+  [Open X-Embodiment](https://robotics-transformer-x.github.io/) (SOAR / Berkeley RPT /
+  Jaco Play), CC BY 4.0. The task strings are the ones used in the RoboMeter README.
+- `examples/so101_lego_into_box.mp4` — episode 1 of
   [`lerobot/svla_so101_pickplace`](https://huggingface.co/datasets/lerobot/svla_so101_pickplace)
   (Apache-2.0), side camera.
+
+The last example row pairs the Franka video with an unrelated instruction to show the
+video/instruction matching behaviour (`Match: No`).

app.py

@@ -1,10 +1,10 @@
 """RynnValue-8B — robotic value model demo.
 
-Predicts, for every step of a robot manipulation video, how much time is left
-until the task is finished, and writes a short analysis (description / does the
-video match the instruction / did it succeed).
+Given a robot manipulation video and the task instruction, predicts how much
+time is left until the task is finished at every point along the clip, plus a
+short analysis block (description / instruction match / success).
 
-Faithful to the official reference implementation
+Follows the official reference implementation
 (https://github.com/alibaba-damo-academy/RynnValue, `rynn_infer/inference.py`):
 prefix-uniform sampling — for evaluation step *i* the prefix `frames[0:i]` is
 resampled to `num_frames` frames and the model's **last** prediction slot is
@@ -15,7 +15,7 @@ import os
 
 os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
 
-import spaces  # noqa: E402  (must come before torch)
+import spaces  # noqa: E402  (must precede torch / transformers)
 
 import re  # noqa: E402
 import tempfile  # noqa: E402
@@ -32,25 +32,26 @@ from plot_utils import save_video_with_trend  # noqa: E402
 
 MODEL_ID = "Alibaba-DAMO-Academy/RynnValue-8B"
 
-# Defaults — every one of these is also the default of its UI component so that
-# clicking an example and pressing "Analyze" behave identically.
+# Defaults — each is also the default of its UI component, so clicking an
+# example and pressing "Analyze" behave identically.
 DEFAULT_ROBOT = "a Franka single-arm robot"
 DEFAULT_CAMERA = "the main camera"
 DEFAULT_NUM_STEPS = 16
-DEFAULT_NUM_FRAMES = 16
-DEFAULT_MAX_SIDE = 448
+DEFAULT_NUM_FRAMES = 32
+DEFAULT_MAX_SIDE = 384
 DEFAULT_MAX_NEW_TOKENS = 128
-BATCH_SIZE = 2
 
-# Rendering budget: the input video is temporally subsampled to at most this
-# many frames (playback fps is scaled to match, so wall-clock duration and the
-# ground-truth "remaining time" reference curve stay correct).
-MAX_RENDER_FRAMES = 420
+# Rendering budget: the input clip is temporally subsampled to at most this many
+# frames and the playback fps is scaled to match, so wall-clock duration and the
+# ground-truth "remaining time" reference curve are unchanged.
+MAX_RENDER_FRAMES = 320
 DISPLAY_MAX_SIDE = 640
 
 ROBOT_CHOICES = [
     "a Franka single-arm robot",
     "an SO-101 single-arm robot",
+    "a WidowX single-arm robot",
+    "a Jaco single-arm robot",
     "a Koch dual-arm robot",
     "an xArm single-arm robot",
     "an Trossen dual-arm robot",
@@ -67,78 +68,78 @@ CAMERA_CHOICES = [
 # --------------------------------------------------------------------------- #
 # Model
 # --------------------------------------------------------------------------- #
-config = AutoConfig.from_pretrained(MODEL_ID, trust_remote_code=True)
+print(f"Loading {MODEL_ID} ...", flush=True)
+_config = AutoConfig.from_pretrained(MODEL_ID, trust_remote_code=True)
 # The exported config predates the attn-impl field, so force the custom
-# prediction-slot isolation attention exactly like the reference script does.
-config._attn_implementation = "pred_slot_isolated_eager"
+# prediction-slot isolation attention (mirrors rynn_infer/inference.py).
+_config._attn_implementation = "pred_slot_isolated_eager"
 
 model = AutoModel.from_pretrained(
     MODEL_ID,
-    config=config,
+    config=_config,
     trust_remote_code=True,
     torch_dtype=torch.bfloat16,
 )
-model = model.to("cuda").eval()
+# `torch_dtype=` leaves the value heads in fp32 (they are constructed with an
+# explicit dtype), so the trailing `dtype=` cast is required — the reference
+# script does the same `model.to(device=..., dtype=...)`.
+model = model.to(device="cuda", dtype=torch.bfloat16).eval()
 
 processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)
 tokenizer = processor.tokenizer
 EOS_TOKEN_ID = tokenizer.convert_tokens_to_ids("<|im_end|>")
+print("Model ready.", flush=True)
 
 # --------------------------------------------------------------------------- #
-# Video / sampling helpers (mirrors rynn_infer/inference.py)
+# Video / sampling helpers (ported from rynn_infer/inference.py)
 # --------------------------------------------------------------------------- #
 
 
-def _resize_to_max_side(img, max_side):
-    if max_side <= 0:
-        return img
+def _resize_long_side(img: Image.Image, max_side: int) -> Image.Image:
     w, h = img.size
-    if max(w, h) <= max_side:
+    if max_side <= 0 or max(w, h) <= max_side:
         return img
-    scale = max_side / max(w, h)
+    scale = max_side / float(max(w, h))
     return img.resize((max(1, int(round(w * scale))), max(1, int(round(h * scale)))), Image.BICUBIC)
 
 
-def load_video_frames(video_path):
-    """Decode a video into (frames, fps).
+def load_video(video_path: str):
+    """Decode a video into ``(frames, fps)``.
 
-    Frames are temporally subsampled to at most ``MAX_RENDER_FRAMES`` and the
-    returned fps is scaled accordingly, so the clip keeps its real duration.
+    Frames are downscaled to ``DISPLAY_MAX_SIDE`` on the fly and strided so that
+    at most ``MAX_RENDER_FRAMES`` are kept; ``fps`` is scaled accordingly so the
+    rendered clip keeps real-time playback speed.
     """
     if not video_path or not os.path.isfile(video_path):
-        raise gr.Error("Please provide a video file.")
+        raise gr.Error("Please upload a video first.")
 
     reader = imageio.get_reader(video_path)
-    meta = reader.get_meta_data()
-    fps = float(meta.get("fps") or 30.0)
-    duration = meta.get("duration")
-    est_total = int(duration * fps) if duration else 0
-
-    stride = max(1, int(np.ceil(est_total / MAX_RENDER_FRAMES))) if est_total else 1
-
-    frames = []
     try:
+        meta = reader.get_meta_data()
+        src_fps = float(meta.get("fps") or 30.0)
+        duration = float(meta.get("duration") or 0.0)
+        est_total = int(duration * src_fps) if duration > 0 else 0
+        stride = 1
+        if est_total > MAX_RENDER_FRAMES:
+            stride = int(np.ceil(est_total / float(MAX_RENDER_FRAMES)))
+
+        frames = []
         for i, frame in enumerate(reader):
             if i % stride:
                 continue
-            img = Image.fromarray(frame).convert("RGB")
-            frames.append(_resize_to_max_side(img, DISPLAY_MAX_SIDE))
+            frames.append(_resize_long_side(Image.fromarray(frame).convert("RGB"), DISPLAY_MAX_SIDE))
+            if len(frames) >= MAX_RENDER_FRAMES + 8:
+                break
     finally:
         reader.close()
 
     if not frames:
-        raise gr.Error("No frames could be decoded from that video.")
-
-    # Guard against a bad duration estimate.
-    while len(frames) > MAX_RENDER_FRAMES:
-        frames = frames[::2]
-        stride *= 2
+        raise gr.Error("Could not decode any frame from this video.")
+    return frames, src_fps / float(stride)
 
-    return frames, fps / stride
 
-
-def sample_frame_indices(total, num_frames):
-    """Uniformly pick ``num_frames`` indices from ``total`` frames."""
+def sample_frame_indices(total: int, num_frames: int):
+    """Uniformly pick ``num_frames`` indices out of ``total`` frames."""
     if num_frames <= 0 or num_frames >= total:
         return list(range(total))
     if num_frames == 1:
@@ -152,7 +153,9 @@ _MATCH_RE = re.compile(r"-\s*Match:\s*(Yes|No)", re.IGNORECASE)
 _SUCCESS_RE = re.compile(r"-\s*Success:\s*(Yes|No)", re.IGNORECASE)
 
 
-def parse_analysis(text):
+def parse_analysis(text: str) -> dict:
+    """Extract description / match / success from the generated Analysis block."""
+
     def _first(pattern):
         m = pattern.search(text)
         return m.group(1).strip() if m else None
@@ -164,22 +167,32 @@ def parse_analysis(text):
     }
 
 
-def _estimate_duration(*args, **kwargs):
-    """ZeroGPU duration estimate — scales with the prefill work requested."""
-    try:
-        num_steps = int(args[4]) if len(args) > 4 else DEFAULT_NUM_STEPS
-        num_frames = int(args[5]) if len(args) > 5 else DEFAULT_NUM_FRAMES
-    except (TypeError, ValueError):
-        num_steps, num_frames = DEFAULT_NUM_STEPS, DEFAULT_NUM_FRAMES
-    return int(min(300, 45 + 0.09 * num_steps * num_frames))
+def _estimate_duration(
+    video_path=None,
+    instruction="",
+    robot_description=DEFAULT_ROBOT,
+    camera_description=DEFAULT_CAMERA,
+    num_steps=DEFAULT_NUM_STEPS,
+    num_frames=DEFAULT_NUM_FRAMES,
+    max_image_side=DEFAULT_MAX_SIDE,
+    max_new_tokens=DEFAULT_MAX_NEW_TOKENS,
+    *args,
+    **kwargs,
+):
+    """ZeroGPU duration estimate; scales with the prefill work requested."""
+    steps = int(num_steps or DEFAULT_NUM_STEPS)
+    frames = int(num_frames or DEFAULT_NUM_FRAMES)
+    side = int(max_image_side or DEFAULT_MAX_SIDE)
+    per_step = 0.75 * (frames / 32.0) * (side / 384.0) ** 2
+    return int(min(240, 25 + steps * per_step + 0.05 * int(max_new_tokens or 128)))
 
 
 # --------------------------------------------------------------------------- #
 # Inference
 # --------------------------------------------------------------------------- #
 @spaces.GPU(duration=_estimate_duration)
-def analyze(
-    video: str,
+def analyze_video(
+    video_path: str,
     instruction: str,
     robot_description: str = DEFAULT_ROBOT,
     camera_description: str = DEFAULT_CAMERA,
@@ -189,46 +202,56 @@ def analyze(
     max_new_tokens: int = DEFAULT_MAX_NEW_TOKENS,
     progress=gr.Progress(),
 ):
-    """Score a robot manipulation video with RynnValue-8B.
+    """Predict how much time is left before a robot finishes a task.
 
-    Returns an annotated video (frames + predicted remaining-time curve) and a
-    markdown report with the model's description / instruction-match / success
-    judgement.
+    Runs RynnValue-8B over a robot manipulation video: for each evaluated step
+    the prefix of the video seen so far is uniformly resampled and the model
+    predicts the remaining time to task completion in seconds. It also generates
+    an Analysis block (what happens in the video, whether the video matches the
+    instruction, whether the task succeeded).
 
     Args:
-        video: path to a robot manipulation video (mp4).
-        instruction: the task the robot was asked to do, e.g. "Put the box in the drawer and close it".
-        robot_description: embodiment phrase for the meta block, e.g. "a Franka single-arm robot".
-        camera_description: viewpoint phrase for the meta block, e.g. "the main camera".
-        num_steps: number of evaluation points along the video (one forward pass each).
-        num_frames: frames fed to the model per evaluation point.
-        max_image_side: frames are downscaled to this longer side before the vision tower.
-        max_new_tokens: token budget for the generated analysis block.
+        video_path: Path to the robot trajectory video (mp4).
+        instruction: Natural-language task the robot is supposed to accomplish.
+        robot_description: Embodiment description, e.g. "a Franka single-arm robot".
+        camera_description: Camera viewpoint description, e.g. "the main camera".
+        num_steps: Number of prefix steps evaluated along the video.
+        num_frames: Frames uniformly resampled from each prefix.
+        max_image_side: Longer side each frame is resized to before the model sees it.
+        max_new_tokens: Token budget for the generated Analysis block.
+
+    Returns:
+        A video with the synchronized remaining-time curve, and a markdown report.
     """
     if not instruction or not instruction.strip():
-        raise gr.Error("Please give the task instruction the robot was asked to follow.")
+        raise gr.Error("Please give the task instruction the robot is supposed to follow.")
     instruction = instruction.strip()
-    robot_description = (robot_description or DEFAULT_ROBOT).strip()
-    camera_description = (camera_description or DEFAULT_CAMERA).strip()
+    robot_description = (robot_description or "").strip() or None
+    camera_description = (camera_description or "").strip() or None
+    if robot_description is None and camera_description is None:
+        raise gr.Error(
+            "This checkpoint was trained with meta information — fill in the "
+            "embodiment and/or viewpoint description."
+        )
+
     num_steps = int(num_steps)
     num_frames = int(num_frames)
     max_image_side = int(max_image_side)
     max_new_tokens = int(max_new_tokens)
 
-    t_all = time.perf_counter()
-    progress(0.0, desc="Decoding video…")
-    frames, fps = load_video_frames(video)
+    t0 = time.perf_counter()
+    progress(0.05, desc="Decoding video…")
+    frames, fps = load_video(video_path)
     total = len(frames)
-    model_images = [_resize_to_max_side(f, max_image_side) for f in frames]
-
+    model_frames = [_resize_long_side(f, max_image_side) for f in frames]
     eval_indices = sample_frame_indices(total, num_steps)
+    t_decode = time.perf_counter() - t0
 
     def build_prefix_sample(end_idx):
         frame_idx = np.linspace(0, end_idx, num_frames, dtype=int)
-        prefix_images = [model_images[j] for j in frame_idx]
         return processor.process_episode(
             instruction=instruction,
-            images=prefix_images,
+            images=[model_frames[j] for j in frame_idx],
             robot_description=robot_description,
             camera_description=camera_description,
         )
@@ -237,9 +260,7 @@ def analyze(
         batch_kwargs = dict(
             input_ids=torch.cat([s["input_ids"] for s in samples], dim=0).to("cuda").long(),
             attention_mask=torch.cat([s["attention_mask"] for s in samples], dim=0).to("cuda").long(),
-            pixel_values=torch.cat(
-                [s["pixel_values"].flatten(0, 1) for s in samples], dim=0
-            ).to("cuda", dtype=torch.bfloat16),
+            pixel_values=torch.cat([s["pixel_values"].flatten(0, 1) for s in samples], dim=0).to("cuda"),
             image_grid_thw=torch.cat(
                 [s["image_grid_thw"].flatten(0, 1) for s in samples], dim=0
             ).to("cuda").long(),
@@ -257,30 +278,32 @@ def analyze(
             pred = pred[:, 0]
         return pred.float().reshape(-1).tolist()
 
-    t_gpu = time.perf_counter()
-    pred_value = []
-    final_sample = None
-    batch = []
+    t1 = time.perf_counter()
+    batch_size = 4 if (num_frames <= 32 and max_image_side <= 448) else 2
+    pred_value, batch, final_sample = [], [], None
     for step, end_idx in enumerate(eval_indices):
         sample = build_prefix_sample(end_idx)
         if step == len(eval_indices) - 1:
             final_sample = sample
         batch.append(sample)
-        if len(batch) >= BATCH_SIZE or step == len(eval_indices) - 1:
+        if len(batch) >= batch_size or step == len(eval_indices) - 1:
             pred_value.extend(run_batch(batch))
             batch = []
             progress(
-                0.05 + 0.7 * len(pred_value) / len(eval_indices),
-                desc=f"Value pass {len(pred_value)}/{len(eval_indices)}",
+                0.05 + 0.65 * len(pred_value) / max(1, len(eval_indices)),
+                desc=f"Value prediction {len(pred_value)}/{len(eval_indices)}",
             )
+    t_value = time.perf_counter() - t1
 
-    progress(0.78, desc="Writing analysis…")
+    # ---- Analysis pass on the full-video prefix ---------------------------- #
+    progress(0.72, desc="Generating analysis…")
+    t2 = time.perf_counter()
     input_ids = final_sample["input_ids"].to("cuda").long()
     with torch.inference_mode():
         gen_out = model.generate(
             input_ids=input_ids,
             attention_mask=final_sample["attention_mask"].to("cuda").long(),
-            pixel_values=final_sample["pixel_values"].flatten(0, 1).to("cuda", dtype=torch.bfloat16),
+            pixel_values=final_sample["pixel_values"].flatten(0, 1).to("cuda"),
             image_grid_thw=final_sample["image_grid_thw"].flatten(0, 1).to("cuda").long(),
             max_new_tokens=max_new_tokens,
             do_sample=False,
@@ -291,73 +314,80 @@ def analyze(
         )
     analysis_text = tokenizer.decode(gen_out[0, input_ids.shape[1]:], skip_special_tokens=True)
     analysis = parse_analysis(analysis_text)
-    gpu_seconds = time.perf_counter() - t_gpu
+    t_gen = time.perf_counter() - t2
 
+    # ---- Render ------------------------------------------------------------ #
     progress(0.85, desc="Rendering trend video…")
-    out_file = os.path.join(tempfile.mkdtemp(), "rynnvalue_trend.mp4")
+    t3 = time.perf_counter()
+    out_path = tempfile.NamedTemporaryFile(suffix=".mp4", delete=False).name
     save_video_with_trend(
         images=frames,
         value=pred_value,
-        output_path=out_file,
-        fps=fps,
+        output_path=out_path,
+        fps=max(1.0, float(fps)),
         title="Remaining Time (s)",
         task_title=instruction,
         sampled_indices=eval_indices,
     )
+    t_render = time.perf_counter() - t3
+    total_time = time.perf_counter() - t0
 
-    total_seconds = time.perf_counter() - t_all
-    first_v, last_v = pred_value[0], pred_value[-1]
-    true_total = (eval_indices[-1] - eval_indices[0]) / float(fps)
+    clip_len = (total - 1) / float(fps)
+    first_pred, last_pred = float(pred_value[0]), float(pred_value[-1])
 
-    def _tick(v):
+    def _badge(v):
         if v is None:
             return "—"
-        return {"yes": "✅ Yes", "no": "❌ No"}.get(v.lower(), v)
+        return "✅ Yes" if v.lower() == "yes" else "❌ No"
 
     report = f"""### Analysis
-
 - **Video description:** {analysis['description'] or '—'}
-- **Matches the instruction:** {_tick(analysis['match'])}
-- **Task succeeded:** {_tick(analysis['success'])}
-
-### Predicted time-to-completion
+- **Matches the instruction:** {_badge(analysis['match'])}
+- **Task succeeded:** {_badge(analysis['success'])}
 
-| | |
-|---|---|
-| At the first frame | **{first_v:.2f} s** remaining |
-| At the last frame | **{last_v:.2f} s** remaining |
-| Actual clip length | {true_total:.2f} s |
+### Predicted remaining time
+| | predicted | actual (clip) |
+|---|---|---|
+| at the first frame | **{first_pred:.2f} s** | {clip_len:.2f} s |
+| at the last frame | **{last_pred:.2f} s** | 0.00 s |
 
-<sub>{len(eval_indices)} evaluation steps · {num_frames} frames/step · {total} frames rendered @ {fps:.1f} fps ·
-{gpu_seconds:.1f} s model time · {total_seconds:.1f} s total</sub>
+<sub>{len(eval_indices)} prefix steps · {num_frames} frames/step · {max_image_side} px ·
+decode {t_decode:.1f}s · value {t_value:.1f}s · analysis {t_gen:.1f}s · render {t_render:.1f}s ·
+total {total_time:.1f}s</sub>
 
-<details><summary>Raw model output</summary>
+<details><summary>Raw generation</summary>
 
 ```
 {analysis_text.strip()}
 ```
 </details>"""
 
-    return out_file, report
+    print(
+        f"[timing] decode={t_decode:.2f}s value={t_value:.2f}s gen={t_gen:.2f}s "
+        f"render={t_render:.2f}s total={total_time:.2f}s steps={len(eval_indices)} "
+        f"frames={num_frames} side={max_image_side}",
+        flush=True,
+    )
+    return out_path, report
 
 
 # --------------------------------------------------------------------------- #
 # UI
 # --------------------------------------------------------------------------- #
 CSS = """
-#col-container { max-width: 1180px; margin: 0 auto; }
+#col-container { max-width: 1200px; margin: 0 auto; }
 .dark .gradio-container { color: var(--body-text-color); }
 """
 
 with gr.Blocks(theme=gr.themes.Citrus(), css=CSS, title="RynnValue-8B") as demo:
     with gr.Column(elem_id="col-container"):
         gr.Markdown(
-            """# RynnValue-8B — how much longer will this robot take?
+            """# RynnValue-8B — how much longer will the robot take?
 
-A robotic value model from **Alibaba DAMO Academy**: given a manipulation video and the task
-instruction, it predicts the **remaining time to completion** at every point in the clip, and
-writes a short analysis (what it sees, whether the video matches the instruction, whether the
-task succeeded).
+A general-purpose value model for robot manipulation from Alibaba DAMO Academy. Give it a
+trajectory video plus the task instruction: it predicts the **remaining time to completion
+(seconds)** at every step along the clip, and says whether the video actually matches the
+instruction and whether the task succeeded.
 
 [Model](https://huggingface.co/Alibaba-DAMO-Academy/RynnValue-8B) ·
 [Code](https://github.com/alibaba-damo-academy/RynnValue) ·
@@ -367,98 +397,131 @@ task succeeded).
 
         with gr.Row():
             with gr.Column(scale=1):
-                video = gr.Video(label="Robot manipulation video", sources=["upload"])
-                instruction = gr.Textbox(
+                video_in = gr.Video(label="Robot trajectory video", sources=["upload"])
+                instruction_in = gr.Textbox(
                     label="Task instruction",
                     placeholder="Put the box in the drawer and close it",
                 )
                 with gr.Row():
-                    robot_description = gr.Dropdown(
+                    robot_in = gr.Dropdown(
                         ROBOT_CHOICES,
                         value=DEFAULT_ROBOT,
                         label="Embodiment",
                         allow_custom_value=True,
+                        scale=1,
                     )
-                    camera_description = gr.Dropdown(
+                    camera_in = gr.Dropdown(
                         CAMERA_CHOICES,
                         value=DEFAULT_CAMERA,
                         label="Viewpoint",
                         allow_custom_value=True,
+                        scale=1,
                     )
-                run = gr.Button("Analyze", variant="primary")
+                run_btn = gr.Button("Analyze trajectory", variant="primary")
             with gr.Column(scale=1):
-                out_video = gr.Video(label="Predicted value trend", autoplay=True)
-                report = gr.Markdown()
+                video_out = gr.Video(label="Remaining-time curve", autoplay=True)
+                report_out = gr.Markdown()
 
         with gr.Accordion("Advanced settings", open=False):
             with gr.Row():
-                num_steps = gr.Slider(
+                num_steps_in = gr.Slider(
                     4, 48, value=DEFAULT_NUM_STEPS, step=1,
-                    label="Evaluation steps",
-                    info="Points on the curve. One forward pass each.",
+                    label="Prefix steps",
+                    info="How many points along the video are evaluated",
                 )
-                num_frames = gr.Slider(
-                    4, 32, value=DEFAULT_NUM_FRAMES, step=1,
+                num_frames_in = gr.Slider(
+                    8, 64, value=DEFAULT_NUM_FRAMES, step=8,
                     label="Frames per step",
-                    info="Frames resampled from the prefix seen so far.",
+                    info="Frames uniformly resampled from each prefix",
                 )
             with gr.Row():
-                max_image_side = gr.Slider(
-                    256, 640, value=DEFAULT_MAX_SIDE, step=32,
+                image_side_in = gr.Slider(
+                    224, 640, value=DEFAULT_MAX_SIDE, step=32,
                     label="Max image side (px)",
+                    info="Frames are downscaled to this before the model sees them",
                 )
-                max_new_tokens = gr.Slider(
+                tokens_in = gr.Slider(
                     32, 256, value=DEFAULT_MAX_NEW_TOKENS, step=16,
-                    label="Max analysis tokens",
+                    label="Analysis max new tokens",
                 )
             gr.Markdown(
-                "The reference implementation uses 64 frames per step and one step per frame; "
-                "the defaults here are trimmed so a run fits comfortably in a ZeroGPU slot."
+                "<sub>The reference implementation evaluates one prefix step per frame with 64 "
+                "frames at 640 px; the defaults here are trimmed so a run fits comfortably in a "
+                "ZeroGPU slot.</sub>"
             )
 
         gr.Examples(
             examples=[
                 [
-                    "assets/franka_box_into_drawer.mp4",
+                    "examples/franka_box_into_drawer.mp4",
                     "Put the box in the drawer and close it",
                     "a Franka single-arm robot",
                     "the main camera",
                 ],
                 [
-                    "assets/so101_lego_into_box.mp4",
+                    "examples/soar_put_green_stick_in_brown_bowl.mp4",
+                    "Put green stick in brown bowl",
+                    "a WidowX single-arm robot",
+                    "the main camera",
+                ],
+                [
+                    "examples/berkeley_rpt_stack_cup.mp4",
+                    "Pick up the yellow cup and stack it on the other cup",
+                    "a Franka single-arm robot",
+                    "the wrist-mounted camera",
+                ],
+                [
+                    "examples/jaco_play_pick_up_green_cup.mp4",
+                    "Pick up the green cup",
+                    "a Jaco single-arm robot",
+                    "the main camera",
+                ],
+                [
+                    "examples/so101_lego_into_box.mp4",
                     "Put the pink lego brick into the transparent box",
                     "an SO-101 single-arm robot",
                     "the side camera",
                 ],
                 [
-                    "assets/franka_box_into_drawer.mp4",
-                    "Fold the blue towel and put it on the shelf",
+                    "examples/franka_box_into_drawer.mp4",
+                    "Fold the towel and put it in the basket",
                     "a Franka single-arm robot",
                     "the main camera",
                 ],
             ],
-            inputs=[video, instruction, robot_description, camera_description],
-            outputs=[out_video, report],
-            fn=analyze,
+            inputs=[video_in, instruction_in, robot_in, camera_in],
+            outputs=[video_out, report_out],
+            fn=analyze_video,
             cache_examples=True,
             cache_mode="lazy",
-            label="Examples (the third one is deliberately mismatched)",
+            examples_per_page=6,
+            label="Examples (the last row deliberately mismatches video and instruction)",
+        )
+
+        gr.Markdown(
+            """<sub>Example clips — `franka_box_into_drawer` is the demo clip bundled with the
+[RynnValue repo](https://github.com/alibaba-damo-academy/RynnValue) (Apache-2.0);
+`soar_*`, `berkeley_rpt_*` and `jaco_play_*` are the RoboMeter benchmark clips bundled in the same
+repository (MIT), originating from [Open X-Embodiment](https://robotics-transformer-x.github.io/)
+(CC BY 4.0); `so101_lego_into_box` is episode 1 of
+[lerobot/svla_so101_pickplace](https://huggingface.co/datasets/lerobot/svla_so101_pickplace)
+(Apache-2.0).</sub>"""
         )
 
-    run.click(
-        analyze,
+    run_btn.click(
+        fn=analyze_video,
         inputs=[
-            video,
-            instruction,
-            robot_description,
-            camera_description,
-            num_steps,
-            num_frames,
-            max_image_side,
-            max_new_tokens,
+            video_in,
+            instruction_in,
+            robot_in,
+            camera_in,
+            num_steps_in,
+            num_frames_in,
+            image_side_in,
+            tokens_in,
         ],
-        outputs=[out_video, report],
-        api_name="analyze",
+        outputs=[video_out, report_out],
+        api_name="analyze_video",
     )
 
 if __name__ == "__main__":

examples/put_the_box_in_the_drawer_and_close_it.mp4

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:48590c921c75ba487706f86b24341209ce3eba354d75587b0be36ddad0d89951
-size 1135507

examples/so100_pick_up_the_cube_and_place_it_in_the_box.mp4

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:094306960c45bb1d81aa60bc618323ef0b0697e034ea28bcefe2401d1d37b271
-size 654381

requirements.txt

@@ -1,3 +1,5 @@
+# transformers 4.57.x is what the checkpoint's remote code targets, and it pins
+# huggingface-hub<1.0 — hence the gradio 5.x SDK version in README.md.
 transformers==4.57.6
 accelerate
 torchvision