| --- |
| title: Kicky AI |
| emoji: ⚽ |
| colorFrom: green |
| colorTo: yellow |
| sdk: gradio |
| sdk_version: 6.18.0 |
| python_version: '3.12' |
| app_file: server.py |
| pinned: false |
| license: apache-2.0 |
| short_description: check if you have the skill to make it the worldcup |
| tags: |
| - track:backyard |
| - sponsor:openbmb |
| - sponsor:openai |
| - sponsor:nvidia |
| - sponsor:modal |
| - achievement:offgrid |
| - achievement:welltuned |
| - achievement:offbrand |
| - achievement:llama |
| - achievement:sharing |
| - achievement:fieldnotes |
| --- |
| |
| # ⚽ Kicky AI — Football Shot Analysis |
|
|
| **They measure your shot. You fix it.** |
|
|
| 🎥 **Demo video:** https://www.youtube.com/watch?v=knL8shghyBU · |
| 📰 **Blog:** https://dcrey7.substack.com/p/world-fut-coach · |
| 🐦 **X thread:** https://x.com/dcrey77/status/2066331088331358643 |
|
|
| Upload a shot clip (or pick a held-out test example). From a single clip, **one small model + |
| pure geometry/physics** produce: |
|
|
| | Output | How | |
| |---|---| |
| | **Goal / no-goal + timestamp** | ball ∩ goal + shot physics (kick → flight → settle-in-net vs rebound / over-the-bar) | |
| | **Who shot it** | ball ∩ player possession over time (last touch before the ball reaches goal) | |
| | **Shooting foot (L/R)** | MediaPipe (BlazePose) on the shooter at the kick frame | |
| | **Shot speed (km/h)** | the ball's apparent diameter (real ball = 22 cm) as a px→m scale | |
| | **AI coaching** | a vision-LLM grades the shot: Verdict → Fix → Fix → Drill | |
|
|
| The video shows **player segmentation masks** (with `P1`/`P2`/`P3` labels that flip to |
| `P# BALL` on possession), a **green goal mask**, a smooth **ball trail**, and a red |
| **goal-spot marker** — plus a separate **Result** table, **Shot-speed** card and |
| **Pose-Detection** card. |
|
|
| ## The approach — foundation-model labelling → real-time distillation |
|
|
| The interesting part isn't one model, it's a **pipeline that needs zero manual labels**: |
|
|
| 1. **Auto-label** every clip with **SAM3** ("ball" / "person" / "goal post"); where SAM3 |
| misses the small/blurred/tilted ball, **NVIDIA LocateAnything-3B** fills the gap. |
| 2. **Derive events geometrically** — ball∩player → possession; ball∩goal + trajectory |
| physics → goal (the rest-in-net vs rebound vs over-the-bar rules handle 2D depth |
| ambiguity). |
| 3. **Distil** those auto-labels into one fast **RF-DETR-Seg** model (ball / player / goal) |
| — replacing the slow SAM3 + LA stack at inference. |
|
|
| The distilled detector runs live on **ZeroGPU**; everything else is CPU geometry. |
|
|
| ## The AI coach — two vision models, one toggle |
|
|
| After the geometry, a **vision-LLM looks at the strike frame** and grades the shot, grounded on |
| the measured stats (goal / foot / speed + pose angles). It streams in separately so the results |
| never wait on it: |
|
|
| - ⚡ **Offline** — **MiniCPM-V-4.6** (OpenBMB, ~1.3B) runs **fully on this Space's own ZeroGPU**, |
| no external call. |
| - ☁️ **Online** — **NVIDIA Nemotron-Nano-12B-v2-VL** (GGUF Q4_K_M + mmproj) served through a |
| **llama.cpp** server on **Modal** (A10G GPU). |
|
|
| ## Built with — sponsors, models & badges |
|
|
| - ⚡ **Modal** *(sponsor)* — runtime for the **online Nemotron coach** (llama.cpp server); see |
| `modal_coach.py`. |
| - 🟣 **NVIDIA** *(sponsor)* — **Nemotron-Nano-12B-v2-VL** powers the online coach; **LocateAnything-3B** |
| rescues the tiny/tilted ball during auto-labelling. |
| - 🟢 **OpenBMB** *(sponsor)* — **MiniCPM-V-4.6** is the **offline, on-device coach**. |
| - 🤖 **OpenAI Codex** *(sponsor)* — built the browser **annotation tools** (Codex-attributed |
| commits in this Space's git history). |
| - 🔗 **Shared agent trace** *(Sharing is Caring)* — the sanitized Codex build trace is on the Hub: |
| [`kicky-ai-codex-trace`](https://huggingface.co/datasets/build-small-hackathon/kicky-ai-codex-trace). |
| - 🦙 **llama.cpp** *(Llama Champion)* — runtime for the Nemotron GGUF (Q4_K_M) coach. |
| - 🎨 **Custom UI** *(Off-Brand)* — a bespoke football-stadium frontend on **`gradio.Server`** |
| (hand-written HTML/JS + SVG pitch, not default Gradio Blocks). |
| - 🛠️ **Fine-tuned model** *(Well-Tuned)* — RF-DETR-Seg, distilled and published at |
| [`kicky-ai-rfdetr-seg`](https://huggingface.co/build-small-hackathon/kicky-ai-rfdetr-seg). |
| - 📴 **Runs offline** *(Off the Grid)* — choose **Offline** and the whole pipeline + coach run |
| **on-device with no cloud API**. |
| - 📝 **Write-up** *(Field Notes)* — full blog on |
| [Substack](https://dcrey7.substack.com/p/world-fut-coach) + `docs/blog.md`. |
|
|
| ## Results (held-out 12-clip test set) |
|
|
| | Detector | Goal | Leg | Pose-capture | |
| |---|---|---|---| |
| | SAM3 + LA (teacher, zero-shot) | 83 % | 82 % | 92 % | |
| | **RF-DETR-Seg-Small** (student, distilled) | 75 % | 75 % | **100 %** | |
|
|
| > Honest note: goal detection on monocular 2D tops out in the low-to-mid 80s % — a ball |
| > *in front of* the goal and a ball *in the net* are the same pixels (depth ambiguity). |
|
|
| Model → [`build-small-hackathon/kicky-ai-rfdetr-seg`](https://huggingface.co/build-small-hackathon/kicky-ai-rfdetr-seg) · |
| Dataset → [`build-small-hackathon/kicky-ai-spf`](https://huggingface.co/datasets/build-small-hackathon/kicky-ai-spf) |
|
|
| Every model is small (RF-DETR-Seg-Small · MiniCPM-V-4.6 1.3B · Nemotron-Nano-12B), well under |
| the 32B cap. Built for the **Build Small Hackathon** by a Sunday-league player who wanted to |
| know if his strikes have World Cup form. |
|
|
