README.md

metadata

title: Kicky AI
emoji: ⚽
colorFrom: green
colorTo: yellow
sdk: gradio
sdk_version: 6.18.0
python_version: '3.12'
app_file: server.py
pinned: false
license: apache-2.0
short_description: check if you have the skill to make it the worldcup
tags:
  - track:backyard
  - sponsor:openbmb
  - sponsor:openai
  - sponsor:nvidia
  - sponsor:modal
  - achievement:offgrid
  - achievement:welltuned
  - achievement:offbrand
  - achievement:llama
  - achievement:sharing
  - achievement:fieldnotes

⚽ Kicky AI — Football Shot Analysis

They measure your shot. You fix it.

🎥 Demo video: https://www.youtube.com/watch?v=knL8shghyBU · 📰 Blog: https://dcrey7.substack.com/p/world-fut-coach · 🐦 X thread: https://x.com/dcrey77/status/2066331088331358643

Upload a shot clip (or pick a held-out test example). From a single clip, one small model + pure geometry/physics produce:

Output	How
Goal / no-goal + timestamp	ball ∩ goal + shot physics (kick → flight → settle-in-net vs rebound / over-the-bar)
Who shot it	ball ∩ player possession over time (last touch before the ball reaches goal)
Shooting foot (L/R)	MediaPipe (BlazePose) on the shooter at the kick frame
Shot speed (km/h)	the ball's apparent diameter (real ball = 22 cm) as a px→m scale
AI coaching	a vision-LLM grades the shot: Verdict → Fix → Fix → Drill

The video shows player segmentation masks (with P1/P2/P3 labels that flip to P# BALL on possession), a green goal mask, a smooth ball trail, and a red goal-spot marker — plus a separate Result table, Shot-speed card and Pose-Detection card.

The approach — foundation-model labelling → real-time distillation

The interesting part isn't one model, it's a pipeline that needs zero manual labels:

1. Auto-label every clip with SAM3 ("ball" / "person" / "goal post"); where SAM3 misses the small/blurred/tilted ball, NVIDIA LocateAnything-3B fills the gap.
2. Derive events geometrically — ball∩player → possession; ball∩goal + trajectory physics → goal (the rest-in-net vs rebound vs over-the-bar rules handle 2D depth ambiguity).
3. Distil those auto-labels into one fast RF-DETR-Seg model (ball / player / goal) — replacing the slow SAM3 + LA stack at inference.

The distilled detector runs live on ZeroGPU; everything else is CPU geometry.

The AI coach — two vision models, one toggle

After the geometry, a vision-LLM looks at the strike frame and grades the shot, grounded on the measured stats (goal / foot / speed + pose angles). It streams in separately so the results never wait on it:

• ⚡ Offline — MiniCPM-V-4.6 (OpenBMB, ~1.3B) runs fully on this Space's own ZeroGPU, no external call.
• ☁️ Online — NVIDIA Nemotron-Nano-12B-v2-VL (GGUF Q4_K_M + mmproj) served through a llama.cpp server on Modal (A10G GPU).

Built with — sponsors, models & badges

• ⚡ Modal (sponsor) — runtime for the online Nemotron coach (llama.cpp server); see modal_coach.py.
• 🟣 NVIDIA (sponsor) — Nemotron-Nano-12B-v2-VL powers the online coach; LocateAnything-3B rescues the tiny/tilted ball during auto-labelling.
• 🟢 OpenBMB (sponsor) — MiniCPM-V-4.6 is the offline, on-device coach.
• 🤖 OpenAI Codex (sponsor) — built the browser annotation tools (Codex-attributed commits in this Space's git history).
• 🔗 Shared agent trace (Sharing is Caring) — the sanitized Codex build trace is on the Hub: kicky-ai-codex-trace.
• 🦙 llama.cpp (Llama Champion) — runtime for the Nemotron GGUF (Q4_K_M) coach.
• 🎨 Custom UI (Off-Brand) — a bespoke football-stadium frontend on gradio.Server (hand-written HTML/JS + SVG pitch, not default Gradio Blocks).
• 🛠️ Fine-tuned model (Well-Tuned) — RF-DETR-Seg, distilled and published at kicky-ai-rfdetr-seg.
• 📴 Runs offline (Off the Grid) — choose Offline and the whole pipeline + coach run on-device with no cloud API.
• 📝 Write-up (Field Notes) — full blog on Substack + docs/blog.md.

Results (held-out 12-clip test set)

Detector	Goal	Leg	Pose-capture
SAM3 + LA (teacher, zero-shot)	83 %	82 %	92 %
RF-DETR-Seg-Small (student, distilled)	75 %	75 %	100 %

Honest note: goal detection on monocular 2D tops out in the low-to-mid 80s % — a ball in front of the goal and a ball in the net are the same pixels (depth ambiguity).

Model → build-small-hackathon/kicky-ai-rfdetr-seg · Dataset → build-small-hackathon/kicky-ai-spf

Every model is small (RF-DETR-Seg-Small · MiniCPM-V-4.6 1.3B · Nemotron-Nano-12B), well under the 32B cap. Built for the Build Small Hackathon by a Sunday-league player who wanted to know if his strikes have World Cup form.