Add Beam serverless-GPU captioner (Qwen2.5-VL) with batching

BLIP-base is a 0.25B COCO captioner with no OCR, so on tutorial screenshots
it produces generic text ("a computer screen with a website on it") that adds
nothing to a step-by-step guide. Serve Qwen2.5-VL-7B-Instruct on a Beam
serverless GPU instead; it reads on-screen text and understands UI elements.

- beam_app.py: batched endpoint, scale-to-zero, weights cached on a Volume.
torch is pinned to 2.7.1 deliberately: Beam hosts run a CUDA 12.9 driver and
an unpinned torch resolves to a cu13 wheel, which fails as a silent
torch.cuda.is_available() == False and a bare HTTP 500 at request time.
- vision.py: Beam backend plus caption_batch(), chunked to keep request
bodies bounded. Falls back to the HF Inference API and then local BLIP, so
an unreachable or scaled-to-zero endpoint degrades instead of breaking.
- guide.py: caption the frame pool in one batched call rather than N calls.
- .beamignore: exclude work/ (32MB of frames) and .env from image uploads.

Captioning is opt-in: without DOCUMAKER_BEAM_CAPTION_URL, behaviour is
unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.beamignore

@@ -0,0 +1,38 @@
+# Generated by Beam SDK
+.beamignore
+.git
+.idea
+.python-version
+.vscode
+.venv
+venv
+__pycache__
+.DS_Store
+.config
+drive/MyDrive
+.coverage
+.pytest_cache
+.ipynb
+.ruff_cache
+.dockerignore
+.ipynb_checkpoints
+.env.local
+.envrc
+**/__pycache__/
+**/.pytest_cache/
+**/node_modules/
+**/.venv/
+*.pyc
+.next/
+.circleci
+
+# --- DocuMaker additions ---
+# Runtime artifacts: uploaded videos, extracted frames, audio, generated DOCX.
+# Without this the whole work/ tree (tens of MB) is uploaded on every deploy.
+work/
+# Secrets must never be baked into the container image. The Beam SDK ignores
+# .env.local but not .env, which is where DocuMaker keeps its tokens.
+.env
+.gradio/
+.cache/
+dep5.log

.env.example

@@ -22,6 +22,16 @@ DOCUMAKER_ENABLE_VISION=1          # set 0 to skip captioning entirely
 # DOCUMAKER_VLM_PROVIDER=
 # DOCUMAKER_LOCAL_CAPTION_MODEL=Salesforce/blip-image-captioning-base
 
+# --- Beam serverless-GPU captioner (see beam_app.py) ---
+# When set, this is the preferred captioner: a real VLM (Qwen2.5-VL) on a GPU,
+# instead of the 0.25B BLIP fallback that cannot read on-screen text. The HF
+# Inference API and local BLIP stay as fallbacks, so an unreachable endpoint
+# degrades rather than breaks the run.
+# Get both values from:  beam deploy beam_app.py:caption
+# DOCUMAKER_BEAM_CAPTION_URL=
+# DOCUMAKER_BEAM_TOKEN=
+# DOCUMAKER_BEAM_TIMEOUT=300         # a cold container pages in weights first
+
 # --- Whisper (local faster-whisper) ---
 DOCUMAKER_WHISPER_MODEL=small      # tiny | base | small | medium | large-v3
 DOCUMAKER_WHISPER_DEVICE=auto      # auto | cuda | cpu

README.md

@@ -148,6 +148,72 @@ demo); set `DOCUMAKER_WHISPER_MODEL=base` in the Space *Settings → Variables*
 snappier transcription. You can push to **both** GitHub and the Space (add both as
 git remotes).
 
+## Serverless GPU captions (Beam)
+
+The default local captioner, BLIP-base, is a 0.25B model trained on COCO photos.
+It has no OCR, so on tutorial screenshots it produces generic text like *"a
+computer screen with a website on it"* — near-useless for a step-by-step guide.
+
+[`beam_app.py`](beam_app.py) serves **Qwen2.5-VL-7B-Instruct** on a serverless
+GPU instead. It reads on-screen text and understands UI affordances, and it
+scales to zero when idle.
+
+```bash
+uv tool install beam-client && beam configure default --token <YOUR_TOKEN>
+beam deploy beam_app.py:caption
+```
+
+Put the printed URL and your Beam token in `.env`:
+
+```
+DOCUMAKER_BEAM_CAPTION_URL=https://<your-endpoint>.app.beam.cloud
+DOCUMAKER_BEAM_TOKEN=<your-beam-token>
+```
+
+Captioning then prefers Beam, falling back to the HF Inference API and then
+local BLIP — an unreachable or scaled-to-zero endpoint degrades the output
+rather than breaking the run. The whole frame pool is sent in **one** batched
+request (see `vision.caption_batch`), so a cold start is paid once per document,
+not once per frame.
+
+### Choosing a GPU
+
+`DOCUMAKER_BEAM_GPU` selects the card. Qwen2.5-VL-7B in bf16 is ~16.5GB of
+weights, so 24GB is the practical floor.
+
+**No 16GB card is usable on Beam today**, which is why the default is 24GB:
+`A4000` deploys but reports *"No compute capacity currently supports A4000"* and
+never gets hardware, and `T4`/`V100` are rejected outright with *"This GPU type
+is not supported. Please use an A10G or RTX 4090 instead."*
+
+| GPU | VRAM | Notes |
+|---|---|---|
+| `A10G` | 24GB | **Default.** Ampere, native bf16, explicitly supported by Beam |
+| `RTX4090` | 24GB | The other explicitly supported option; Ada, faster |
+| `L4` | 24GB | Ada, efficient — check capacity before relying on it |
+| `A100_40` | 40GB | Overkill for a 7B VLM |
+| `A4000` | 16GB | ⚠️ Deploys, but currently has no capacity |
+| `T4` / `V100` | 16GB | ⚠️ Rejected by Beam. `V100` would also fail on CUDA support |
+
+To fit a smaller card, set `DOCUMAKER_BEAM_MODEL=Qwen/Qwen2.5-VL-3B-Instruct`
+(~7GB in bf16). `DOCUMAKER_BEAM_MAX_PIXELS` bounds VRAM either way — Qwen2.5-VL
+scales visual tokens with input resolution, so an uncapped screenshot can
+balloon usage.
+
+### Why torch is pinned
+
+`beam_app.py` pins `torch==2.7.1` deliberately. Beam's hosts run a **CUDA 12.9
+driver**; an unpinned `torch` resolves to a cu13 wheel that the driver is too old
+to run. The failure is silent and misleading — `torch.cuda.is_available()` just
+returns `False`, the model load fails on `device_map="cuda:0"`, and the endpoint
+returns a bare `HTTP 500`. If you bump torch, verify the wheel's CUDA version
+still matches the host driver.
+
+If the AWQ dependency chain ever breaks, the no-quantization fallback is
+`DOCUMAKER_BEAM_MODEL=Qwen/Qwen2.5-VL-3B-Instruct` (~7GB in bf16, still far
+better than BLIP). `DOCUMAKER_BEAM_MAX_PIXELS` bounds VRAM — Qwen2.5-VL scales
+visual tokens with input resolution, so an uncapped screenshot can balloon usage.
+
 ## Configuration
 
 All settings are environment variables (see [.env.example](.env.example)). Highlights:
@@ -156,7 +222,11 @@ All settings are environment variables (see [.env.example](.env.example)). Highl
 |---|---|---|
 | `DOCUMAKER_LLM_MODEL` | `Qwen/Qwen2.5-7B-Instruct` | Text LLM (any HF instruct model) |
 | `DOCUMAKER_VLM_MODEL` | `Qwen/Qwen2-VL-7B-Instruct` | API vision model tried before local BLIP |
-| `DOCUMAKER_LOCAL_CAPTION_MODEL` | `Salesforce/blip-image-captioning-base` | Local captioner |
+| `DOCUMAKER_LOCAL_CAPTION_MODEL` | `Salesforce/blip-image-captioning-base` | Local captioner (last-resort fallback) |
+| `DOCUMAKER_BEAM_CAPTION_URL` | *(unset)* | Beam GPU captioner; preferred when set |
+| `DOCUMAKER_BEAM_TOKEN` | *(unset)* | Beam auth token |
+| `DOCUMAKER_BEAM_GPU` | `A10G` | GPU for the Beam endpoint (24GB) |
+| `DOCUMAKER_BEAM_MODEL` | `Qwen/Qwen2.5-VL-7B-Instruct` | VLM served on Beam |
 | `DOCUMAKER_ENABLE_VISION` | `1` | Set `0` to skip captioning |
 | `DOCUMAKER_WHISPER_MODEL` | `small` | `tiny`…`large-v3` |
 | `DOCUMAKER_WHISPER_DEVICE` | `auto` | `auto` / `cuda` / `cpu` |

beam_app.py

@@ -0,0 +1,204 @@
+"""Beam serverless-GPU captioner for DocuMaker.
+
+Replaces the local BLIP fallback in ``src/vision.py`` with a real vision-language
+model. BLIP is a 0.25B COCO captioner with no OCR — on tutorial screenshots it
+emits generic text like "a computer screen with a website on it". Qwen2.5-VL
+reads on-screen text and understands UI affordances, which is what a step-by-step
+guide actually needs.
+
+The endpoint is *batched*: DocuMaker captions one frame per step (see
+``src/guide.py``), so sending the whole set in one request turns N cold-start
+round-trips into one.
+
+Deploy:
+    beam deploy beam_app.py:caption
+
+Model weights are cached on a Beam Volume, so only the first container pays the
+download cost.
+"""
+from __future__ import annotations
+
+import base64
+import io
+import os
+
+from beam import Image, QueueDepthAutoscaler, Volume, endpoint
+
+# --- Tunables ---------------------------------------------------------------
+# Full bf16 weights (~16.5GB) — comfortable on the 24GB A10G. The AWQ build was
+# only needed to fit 16GB, and AutoAWQ is deprecated (last tested on torch 2.6 /
+# transformers 4.51), so dropping it removes a fragile dependency. For a smaller
+# card, Qwen/Qwen2.5-VL-3B-Instruct is ~7GB and still far better than BLIP.
+MODEL_ID = os.getenv("DOCUMAKER_BEAM_MODEL", "Qwen/Qwen2.5-VL-7B-Instruct")
+# A10G (24GB). No 16GB card is usable here: A4000 reports no capacity, and Beam
+# rejects T4/V100 outright ("use an A10G or RTX 4090 instead"). The 7B bf16
+# weights need ~16.5GB, leaving headroom for the vision encoder and KV cache.
+GPU = os.getenv("DOCUMAKER_BEAM_GPU", "A10G")
+CACHE_DIR = "./hf-cache"
+
+# Qwen2.5-VL scales its visual token count with input resolution, so an
+# unbounded screenshot can balloon VRAM. Cap it: 1280 * 28 * 28 keeps a typical
+# 1080p screenshot well inside budget while preserving legible UI text.
+MAX_PIXELS = int(os.getenv("DOCUMAKER_BEAM_MAX_PIXELS", str(1280 * 28 * 28)))
+MIN_PIXELS = int(os.getenv("DOCUMAKER_BEAM_MIN_PIXELS", str(256 * 28 * 28)))
+
+DEFAULT_PROMPT = (
+    "In one concise sentence, describe what this screenshot from a tutorial shows, "
+    "focusing on the on-screen UI element or the action being performed. "
+    "Do not begin with phrases like 'The image shows'."
+)
+
+image = Image(
+    python_version="python3.11",
+    python_packages=[
+        # PIN torch, do not float it. Beam's hosts run a CUDA 12.9 driver, and an
+        # unpinned `torch` resolves to a cu13 wheel whose CUDA runtime the driver
+        # is too old for — torch.cuda.is_available() silently returns False and
+        # the container dies on device_map="cuda:0" with a bare 500.
+        # torch 2.7.1 ships cu126 on PyPI, which the 12.9 driver runs fine.
+        "torch==2.7.1",
+        "torchvision==0.22.1",
+        "transformers==4.53.2",
+        "accelerate",
+        "qwen-vl-utils",
+        "pillow",
+    ],
+).with_envs([f"HF_HOME={CACHE_DIR}"])
+
+
+def load_model():
+    """Runs once per container (``on_start``), not once per request."""
+    import torch
+    from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
+
+    # Fail loudly here rather than with an opaque 500 from the request handler:
+    # a driver/wheel CUDA mismatch shows up exactly as "no CUDA available".
+    if not torch.cuda.is_available():
+        raise RuntimeError(
+            f"CUDA unavailable (torch {torch.__version__}). The pinned torch build "
+            "must match Beam's host driver — see the pin note on `image` above."
+        )
+    print(f"[documaker-captioner] torch {torch.__version__} on "
+          f"{torch.cuda.get_device_name(0)}")
+
+    processor = AutoProcessor.from_pretrained(
+        MODEL_ID, min_pixels=MIN_PIXELS, max_pixels=MAX_PIXELS, cache_dir=CACHE_DIR
+    )
+
+    # transformers v5 renamed ``torch_dtype`` to ``dtype``; v4 only knows the old
+    # spelling. Try the new one first so this works on either.
+    # bfloat16: A10G is Ampere, so bf16 is native and avoids the fp16 overflow
+    # Qwen2.5-VL is prone to. Falls back to fp16 on pre-Ampere cards.
+    dtype = torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16
+    common = {"device_map": "cuda:0", "cache_dir": CACHE_DIR}
+    try:
+        model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
+            MODEL_ID, dtype=dtype, **common
+        )
+    except TypeError:
+        model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
+            MODEL_ID, torch_dtype=dtype, **common
+        )
+
+    model.eval()
+    print(f"[documaker-captioner] loaded {MODEL_ID} on {model.device}")
+    return processor, model
+
+
+def _decode_image(raw: str):
+    """Accept a bare base64 string or a full ``data:image/...;base64,`` URI."""
+    from PIL import Image as PILImage
+
+    if not raw:
+        raise ValueError("empty image payload")
+    if raw.startswith("data:"):
+        raw = raw.split(",", 1)[1]
+    return PILImage.open(io.BytesIO(base64.b64decode(raw))).convert("RGB")
+
+
+@endpoint(
+    name="documaker-captioner",
+    image=image,
+    gpu=GPU,
+    cpu=2,
+    memory="16Gi",
+    on_start=load_model,
+    volumes=[Volume(name="documaker-hf-cache", mount_path=CACHE_DIR)],
+    # Weights take ~40s to page in on a cold container. Staying warm for 5
+    # minutes means a user processing several videos in a sitting pays that
+    # once, while an idle endpoint still scales to zero.
+    keep_warm_seconds=300,
+    timeout=600,
+    autoscaler=QueueDepthAutoscaler(max_containers=2, tasks_per_container=1),
+)
+def caption(context, **inputs):
+    """Caption a batch of frames.
+
+    Input::
+
+        {"items": [{"image": "<b64|data-uri>", "context": "optional step text"}],
+         "prompt": "optional override",
+         "max_new_tokens": 96}
+
+    Output::
+
+        {"captions": ["...", ...], "model": "...", "count": N}
+
+    Captions are returned positionally, so ``captions[i]`` belongs to
+    ``items[i]``. A frame that fails to decode or generate yields ``""`` rather
+    than failing the whole batch — DocuMaker treats an empty caption as "no
+    caption" and the guide still builds.
+    """
+    import torch
+
+    processor, model = context.on_start_value
+
+    items = inputs.get("items") or []
+    if not items:
+        return {"captions": [], "model": MODEL_ID, "count": 0}
+
+    base_prompt = inputs.get("prompt") or DEFAULT_PROMPT
+    max_new_tokens = int(inputs.get("max_new_tokens") or 96)
+
+    captions: list[str] = []
+    for item in items:
+        try:
+            img = _decode_image(item.get("image", ""))
+            prompt = base_prompt
+            step_context = (item.get("context") or "").strip()
+            if step_context:
+                prompt += f" For context, this step is about: {step_context[:200]}"
+
+            messages = [
+                {
+                    "role": "user",
+                    "content": [
+                        {"type": "image", "image": img},
+                        {"type": "text", "text": prompt},
+                    ],
+                }
+            ]
+            text = processor.apply_chat_template(
+                messages, tokenize=False, add_generation_prompt=True
+            )
+            model_inputs = processor(
+                text=[text], images=[img], padding=True, return_tensors="pt"
+            ).to(model.device)
+
+            with torch.no_grad():
+                generated = model.generate(
+                    **model_inputs,
+                    max_new_tokens=max_new_tokens,
+                    do_sample=False,
+                )
+            # Strip the prompt tokens before decoding.
+            trimmed = generated[0][model_inputs.input_ids.shape[1]:]
+            caption_text = processor.decode(
+                trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=True
+            ).strip()
+            captions.append(caption_text)
+        except Exception as exc:  # one bad frame must not sink the batch
+            print(f"[documaker-captioner] frame failed: {exc}")
+            captions.append("")
+
+    return {"captions": captions, "model": MODEL_ID, "count": len(captions)}

scripts/test_beam_captions.py

@@ -0,0 +1,64 @@
+"""Smoke-test the Beam GPU captioner against real frames.
+
+Usage (from the repo root, with DOCUMAKER_BEAM_CAPTION_URL / DOCUMAKER_BEAM_TOKEN
+set in .env or the environment):
+
+    python scripts/test_beam_captions.py [frame.png ...]
+
+With no arguments it picks a few frames out of ``work/``. Prints the caption each
+backend produces so you can see the Qwen2.5-VL vs BLIP difference directly.
+"""
+from __future__ import annotations
+
+import sys
+import time
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from src import config, vision  # noqa: E402
+
+
+def find_frames(limit: int = 3) -> list[Path]:
+    work = Path(__file__).resolve().parent.parent / "work"
+    frames = sorted(work.glob("*/frames/*.png"))
+    return frames[:limit]
+
+
+def main() -> int:
+    paths = [Path(a) for a in sys.argv[1:]] or find_frames()
+    paths = [p for p in paths if p.exists()]
+    if not paths:
+        print("No frames found. Pass image paths explicitly.")
+        return 1
+
+    print(f"Beam URL   : {config.BEAM_CAPTION_URL or '(unset)'}")
+    print(f"Beam token : {'set' if config.BEAM_CAPTION_TOKEN else '(unset)'}")
+    print(f"Frames     : {len(paths)}\n")
+
+    if not config.BEAM_CAPTION_URL:
+        print("DOCUMAKER_BEAM_CAPTION_URL is unset — caption_batch would fall back "
+              "to the HF API / local BLIP. Set it to test the GPU path.")
+        return 1
+
+    start = time.time()
+    captions = vision.caption_batch([(p, "") for p in paths])
+    elapsed = time.time() - start
+
+    for path, caption in zip(paths, captions):
+        print(f"--- {path.name}")
+        print(f"    {caption or '(empty)'}\n")
+
+    ok = sum(1 for c in captions if c)
+    print(f"{ok}/{len(paths)} captioned in {elapsed:.1f}s "
+          f"({elapsed / max(len(paths), 1):.1f}s per frame)")
+    # _BEAM_DISABLED flips on the first hard failure (bad URL/token/endpoint).
+    if vision._BEAM_DISABLED:
+        print("\nNOTE: the Beam backend errored and was disabled for this run — "
+              "captions above (if any) came from the HF API or local BLIP.")
+        return 1
+    return 0 if ok else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/config.py

@@ -116,6 +116,20 @@ LOCAL_CAPTION_MODEL = os.getenv(
     "DOCUMAKER_LOCAL_CAPTION_MODEL", "Salesforce/blip-image-captioning-base"
 )
 
+# --- Beam serverless-GPU captioner (see beam_app.py) -------------------------
+# When BEAM_CAPTION_URL is set it becomes the preferred captioner, ahead of the
+# HF Inference API and local BLIP. Both remain as fallbacks, so an unreachable
+# or scaled-to-zero endpoint degrades instead of breaking the run.
+BEAM_CAPTION_URL = os.getenv("DOCUMAKER_BEAM_CAPTION_URL", "").strip()
+BEAM_CAPTION_TOKEN = os.getenv("DOCUMAKER_BEAM_TOKEN", "").strip()
+# A cold container must download/page in the weights before it answers; the
+# first call of a session can legitimately take a couple of minutes.
+BEAM_CAPTION_TIMEOUT = float(os.getenv("DOCUMAKER_BEAM_TIMEOUT", "300"))
+# Frames per request. A long video can yield dozens of frames; sending them all
+# at once makes a multi-MB body and a generation run long enough to hit the
+# endpoint timeout. Chunking keeps each request bounded.
+BEAM_CAPTION_BATCH_SIZE = int(os.getenv("DOCUMAKER_BEAM_BATCH_SIZE", "12"))
+
 # --- Whisper (local faster-whisper) -----------------------------------------
 WHISPER_MODEL = os.getenv("DOCUMAKER_WHISPER_MODEL", "small")
 WHISPER_DEVICE = os.getenv("DOCUMAKER_WHISPER_DEVICE", "auto").strip().lower()

src/guide.py

@@ -157,14 +157,17 @@ def assemble_guide(
     # Caption the pool up front (once per frame, context-free to keep the
     # relevance signal unbiased) so captions feed both selection and figures.
     if do_caption and frames_sorted:
-        for i, rec in enumerate(frames_sorted):
-            if rec.caption is None:
-                if progress:
-                    progress(
-                        0.05 + 0.45 * (i / len(frames_sorted)),
-                        f"Captioning frame {i + 1}/{len(frames_sorted)}…",
-                    )
-                rec.caption = vision.caption_image(rec.path, token=token) or ""
+        pending = [rec for rec in frames_sorted if rec.caption is None]
+        if pending:
+            if progress:
+                progress(0.05, f"Captioning {len(pending)} frames…")
+            # One batched request when the Beam GPU endpoint is configured;
+            # caption_batch falls back to per-frame captioning otherwise.
+            captions = vision.caption_batch(
+                [(rec.path, "") for rec in pending], token=token
+            )
+            for rec, cap in zip(pending, captions):
+                rec.caption = cap or ""
 
     used: set[str] = set()
     steps: list[GuideStep] = []

src/vision.py

@@ -20,6 +20,10 @@ _LOCAL_FAILED = False
 # Many free HF accounts have no provider that serves a vision-chat model. Once
 # the API VLM fails, stop retrying it for the session and use local BLIP.
 _API_VLM_DISABLED = False
+# Same idea for the Beam GPU endpoint: one hard failure (bad URL, bad token,
+# endpoint deleted) disables it for the session rather than paying the timeout
+# on every frame.
+_BEAM_DISABLED = False
 
 _CAPTION_PROMPT = (
     "In one concise sentence, describe what this screenshot from a tutorial shows, "
@@ -69,6 +73,84 @@ def _caption_via_api(image_path: str | Path, prompt: str, token: str | None) ->
     return (resp.choices[0].message.content or "").strip()
 
 
+def _beam_post(items: list[tuple[str | Path, str]], prompt: str) -> list[str] | None:
+    """One request to the Beam endpoint. ``None`` on any failure."""
+    global _BEAM_DISABLED
+
+    import requests
+
+    payload = {
+        "items": [
+            {"image": _data_uri(path), "context": ctx or ""} for path, ctx in items
+        ],
+        "prompt": prompt,
+    }
+    headers = {"Content-Type": "application/json"}
+    if config.BEAM_CAPTION_TOKEN:
+        headers["Authorization"] = f"Bearer {config.BEAM_CAPTION_TOKEN}"
+
+    try:
+        resp = requests.post(
+            config.BEAM_CAPTION_URL,
+            json=payload,
+            headers=headers,
+            timeout=config.BEAM_CAPTION_TIMEOUT,
+        )
+        resp.raise_for_status()
+        captions = resp.json().get("captions")
+        if not isinstance(captions, list) or len(captions) != len(items):
+            return None
+        return [str(c or "").strip() for c in captions]
+    except Exception:
+        # Auth/URL problems repeat on every call, so stop trying this session.
+        _BEAM_DISABLED = True
+        return None
+
+
+def _beam_caption_batch(
+    items: list[tuple[str | Path, str]], prompt: str
+) -> list[str] | None:
+    """Caption frames via the Beam GPU endpoint, chunked.
+
+    ``items`` is a list of ``(image_path, context)``. Returns captions aligned to
+    ``items``, or ``None`` if the endpoint is unconfigured or any chunk fails —
+    the caller then falls back to the HF API and local BLIP for the whole set,
+    which keeps the outcome predictable rather than half-Beam/half-BLIP.
+    """
+    if _BEAM_DISABLED or not config.BEAM_CAPTION_URL or not items:
+        return None
+
+    size = max(1, config.BEAM_CAPTION_BATCH_SIZE)
+    captions: list[str] = []
+    for start in range(0, len(items), size):
+        chunk = _beam_post(items[start:start + size], prompt)
+        if chunk is None:
+            return None
+        captions.extend(chunk)
+    return captions
+
+
+def caption_batch(
+    items: list[tuple[str | Path, str]], *, token: str | None = None
+) -> list[str]:
+    """Caption a list of ``(image_path, context)`` pairs.
+
+    Prefers one batched call to the Beam GPU endpoint. Without it, falls back to
+    per-frame captioning via :func:`caption_image` so behaviour is unchanged when
+    Beam is not configured.
+    """
+    if not config.ENABLE_VISION or not items:
+        return ["" for _ in items]
+
+    captions = _beam_caption_batch(items, _CAPTION_PROMPT)
+    if captions is not None:
+        return captions
+
+    return [
+        caption_image(path, token=token, context=ctx) or "" for path, ctx in items
+    ]
+
+
 def _load_local_captioner() -> None:
     """Load the BLIP captioner directly (the image-to-text pipeline task was
     removed in transformers 5). Uses the GPU if a CUDA build of torch is present.
@@ -129,10 +211,10 @@ def caption_image(
 ) -> str | None:
     """Return a one-line caption for a frame, or None if captioning is off/failed.
 
-    With a ``token`` it tries an API vision-chat model first (if any provider
-    serves one), then falls back to local BLIP. After the API VLM fails once it
-    is skipped for the rest of the session to avoid repeated dead calls. Local
-    BLIP needs no token.
+    Order of preference: the Beam GPU endpoint (if ``DOCUMAKER_BEAM_CAPTION_URL``
+    is set), then an API vision-chat model (if any provider serves one), then
+    local BLIP. After a backend fails once it is skipped for the rest of the
+    session to avoid repeated dead calls. Local BLIP needs no token.
     """
     global _API_VLM_DISABLED
     if not config.ENABLE_VISION:
@@ -141,6 +223,10 @@ def caption_image(
     if context:
         prompt += f" For context, this step is about: {context[:200]}"
 
+    beam = _beam_caption_batch([(image_path, context)], _CAPTION_PROMPT)
+    if beam and beam[0]:
+        return beam[0]
+
     if token and not _API_VLM_DISABLED:
         try:
             caption = _caption_via_api(image_path, prompt, token)