README.md

metadata

title: VAR - RFDETR
emoji: 🥅
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

VAR - RFDETR — Offside Visualizer

RF-DETR variant: detection uses Roboflow's RF-DETR-Seg (large) with a toggle between bounding boxes and segments. Upload a match clip, scrub to the moment the ball is played, reconstruct selected players in 3D with SAM 3D Body, and place them on a virtual pitch with a draggable offside plane.

Pipeline

1. Upload a video clip.
2. Scrub to the offside frame (slider + prev/next, frames seeked on demand).
3. Detect players on that frame — the only GPU step, cached per (video, frame, threshold).
4. Select the players to analyze and mark the defenders (incl. GK).
5. Click two goal-parallel lines (4 points) on the detected frame to fix the offside axis.
6. Build the 3D scene; drag the offside plane and read the OFFSIDE / NO-OFFSIDE verdict.

The GPU runs once per frame (pipeline/gpu.py). Scrubbing, line geometry, placement, plotting, and the draggable plane are all CPU on the cached result.

Code layout

app.py            Gradio UI + event wiring (CPU)
pipeline/
  video.py        frame seek/probe (CPU)
  gpu.py          model load + reconstruct_frame  ← the ONLY GPU code
  geometry.py     vanishing point, ground fit, field frame, scene (CPU)
  overlay.py      detection boxes + line-click drawing (CPU)

Isolating the GPU in pipeline/gpu.py means moving inference to a serverless backend (Modal / ZeroGPU) later only touches reconstruct_frame.

Deploy

This is a Docker SDK Space for dedicated GPU hardware (A100 recommended):

1. Set hardware to an A100 tier.
2. Add a secret HF_TOKEN — a read token for an account with approved access to the gated facebook/sam-3d-body-dinov3. (Override the repo with the SAM3D_REPO_ID env var if you use a different checkpoint.)
3. First boot builds the image and downloads ~7 GB of weights — give it time. After that, the model stays warm until you pause the Space.

Cost control: dedicated GPU bills while the Space is running, with no auto-shutoff. Pause the Space from its settings when you are not using it.

Why not ZeroGPU?

ZeroGPU allocates the GPU per call, caps call duration, enforces a daily quota, and cold-loads the ~7 GB model stack on each allocation — a poor fit for an interactive video-scrubbing session, and it requires the Gradio SDK (not Docker).

Notes / limits

• Scale comes from the reconstructed body height, so positions are approximate metres — good for relative offside ordering, not sub-10 cm officiating calls. The verdict surfaces a "too close to call" band rather than implying false precision.
• The offside point currently uses the forward-most body vertex including arms; excluding arms (via MHR body-part labels) is planned — see TODO.md.
• "Find the offside moment" is manual scrubbing; automatic pass-instant detection (ball tracking) is future work.