Upload folder using huggingface_hub
Browse files- README.md +80 -0
- model.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
pipeline_tag: image-classification
|
| 5 |
+
tags:
|
| 6 |
+
- robotics
|
| 7 |
+
- egocentric-video
|
| 8 |
+
- capture-qa
|
| 9 |
+
- hand-detection
|
| 10 |
+
- edge
|
| 11 |
+
- tinyml
|
| 12 |
+
datasets:
|
| 13 |
+
- lerobot/droid_1.0.1
|
| 14 |
+
- imageomics/KABR
|
| 15 |
+
model-index:
|
| 16 |
+
- name: referee-hands-s
|
| 17 |
+
results:
|
| 18 |
+
- task:
|
| 19 |
+
type: image-classification
|
| 20 |
+
name: Hand/gripper visibility (clip-level)
|
| 21 |
+
dataset:
|
| 22 |
+
type: referee-lab/hands-visible
|
| 23 |
+
name: referee-lab/hands-visible (Prime Intellect Environments Hub)
|
| 24 |
+
split: full
|
| 25 |
+
metrics:
|
| 26 |
+
- type: accuracy
|
| 27 |
+
name: Clip accuracy
|
| 28 |
+
value: 0.9778
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
# referee-hands-s
|
| 32 |
+
|
| 33 |
+
A **1.5M-parameter** MobileNetV3-small that answers one question about robot-training
|
| 34 |
+
video: *is a hand or gripper visible?* Distilled from gemma3:27b pseudo-labels
|
| 35 |
+
(4,138 frames from DROID / ALOHA-sim / KABR — all CC BY / CC0 / MIT), 96px inputs,
|
| 36 |
+
built to fit a microcontroller inference budget.
|
| 37 |
+
|
| 38 |
+
## Scoreboard (public benchmark: [referee-lab/hands-visible](https://github.com/publu/referee-lab))
|
| 39 |
+
|
| 40 |
+
| Model | Params | Accuracy |
|
| 41 |
+
|---|---|---|
|
| 42 |
+
| gemma3:27b (the teacher) | 27B | 97.8% |
|
| 43 |
+
| **referee-hands-s (this model)** | **1.5M** | **97.8%** |
|
| 44 |
+
| qwen2.5vl:32b | 32B | 74.4% |
|
| 45 |
+
| *do-nothing floor (constant "yes")* | — | *68.9%* |
|
| 46 |
+
| qwen2.5vl:7b | 7B | 45.6% |
|
| 47 |
+
| EgoSieve-S (hand-visibility head) | 22M | 40.0% |
|
| 48 |
+
| MediaPipe HandLandmarker | — | 40.0% |
|
| 49 |
+
|
| 50 |
+
Scored on the public 45-clip answer sheet, clip verdict = any frame above 0.5.
|
| 51 |
+
**Caveat, stated plainly:** training frames come from different clips but the same
|
| 52 |
+
source videos as the benchmark — an in-domain result. A cross-video split is the
|
| 53 |
+
planned follow-up. The benchmark's own labels are machine-graded provisional with a
|
| 54 |
+
published grading record.
|
| 55 |
+
|
| 56 |
+
## Usage
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
import torch
|
| 60 |
+
from torchvision import models, transforms
|
| 61 |
+
from safetensors.torch import load_file
|
| 62 |
+
|
| 63 |
+
net = models.mobilenet_v3_small()
|
| 64 |
+
net.classifier[3] = torch.nn.Linear(net.classifier[3].in_features, 1)
|
| 65 |
+
net.load_state_dict(load_file("model.safetensors"))
|
| 66 |
+
net.eval()
|
| 67 |
+
|
| 68 |
+
tf = transforms.Compose([transforms.Resize((96, 96)), transforms.ToTensor()])
|
| 69 |
+
p = torch.sigmoid(net(tf(img).unsqueeze(0)).squeeze()) # P(hand/gripper visible)
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
Clip-level: sample ~12 frames, answer yes if `max(p) > 0.5`.
|
| 73 |
+
|
| 74 |
+
## Provenance
|
| 75 |
+
|
| 76 |
+
Teacher: gemma3:27b (96.7-97.8% on the same benchmark). Training/eval media:
|
| 77 |
+
DROID (CC BY 4.0), lerobot aloha sim (MIT), KABR (CC0). The benchmark, its
|
| 78 |
+
validation gates, and all baseline runs: https://github.com/publu/referee-lab —
|
| 79 |
+
five capture-QA evals, publicly runnable on the Prime Intellect Environments Hub
|
| 80 |
+
(`prime eval run referee-lab/hands-visible`).
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b793f7a6d252d5e81928c7f3ee7c9d168f7feea1b6267c556ea8b7e0951ee90d
|
| 3 |
+
size 6147500
|