Fix skipped videos without a runtime limit; calibrate video pricing

#2
by pepijn223 HF Staff - opened
LeRobot org

Fixes the first production run of Human demo videos (pepijn223/rebot_diverse_picking_100_annotated_video): every video was skipped as time_stopped.

Bug

Provider.remaining_time() returns a placeholder 300 s when no custom runtime limit is set (the default). The video step needs 420 s, so it refused every video on a job with 4 h left. The edits had already run, so the job paid $0.48 for 12 edits and produced no videos.

Fix

  • FalMedia now takes the real deadline: the runtime limit, or the job timeout minus a 15 min setup margin when none is set (worker.video_deadline).
  • The time check runs before the edit, so an edit whose video could not finish is never paid for.
  • Regression tests cover both.

Calibrated estimate (from that run)

  • Videos = Ī£ per episode (1 + seconds / 5.3), up to 1.5Ɨ in the upper bound, capped per episode. The run had 33 subtasks in 159.7 s over 3 episodes; the formula gives 33.
  • Media price at fal list prices: $0.03/MP Ɨ 1.05 MP (1024Ɨ1024 edit) + $0.05/s Ɨ 5 s ā‰ˆ $0.28/video. It was $0.34, the padded reservation.
  • VLM tokens per video: ~1,750 in / 550 out, previously 6,100 / 650.
  • The budget still reserves $0.34 while a request is in flight, but now reconciles to the list price on success. media_usd in run.json therefore reports list-price spend.

Verified

  • 92 tests pass; bun run build is clean.
  • Real worker run with no runtime limit (annotate_episode + real Provider + FalMedia) on a ReBot episode: the video is completed (640Ɨ480, hand grasps the green cloth) for $0.2815 media + $0.005 VLM.

šŸ¤– Generated with Claude Code

LeRobot org

Two more fixes, both found while regenerating the first run's videos:

  • FalMedia fills price fields missing from an older pinned profile from the current catalog. Without this, a KeyError after a paid edit discarded it.
  • Failed segments now store a short error next to error_type in index.json, so failures can be diagnosed from the output.

94 tests pass.

pepijn223 changed pull request status to open
pepijn223 changed pull request status to merged

Sign up or log in to comment