Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -4,11 +4,9 @@ library_name: pytorch
|
|
| 4 |
pipeline_tag: image-classification
|
| 5 |
tags:
|
| 6 |
- robotics
|
| 7 |
-
- egocentric-video
|
| 8 |
- capture-qa
|
| 9 |
- hand-detection
|
| 10 |
- edge
|
| 11 |
-
- tinyml
|
| 12 |
datasets:
|
| 13 |
- lerobot/droid_1.0.1
|
| 14 |
- imageomics/KABR
|
|
@@ -20,7 +18,7 @@ model-index:
|
|
| 20 |
name: Hand/gripper visibility (clip-level)
|
| 21 |
dataset:
|
| 22 |
type: referee-lab/hands-visible
|
| 23 |
-
name: referee-lab/hands-visible
|
| 24 |
split: full
|
| 25 |
metrics:
|
| 26 |
- type: accuracy
|
|
@@ -30,28 +28,29 @@ model-index:
|
|
| 30 |
|
| 31 |
# referee-hands-s
|
| 32 |
|
| 33 |
-
A
|
| 34 |
-
|
| 35 |
-
(4,138 frames from DROID / ALOHA-sim / KABR — all CC BY / CC0 / MIT), 96px inputs,
|
| 36 |
-
built to fit a microcontroller inference budget.
|
| 37 |
|
| 38 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
| Model | Params | Accuracy |
|
| 41 |
|---|---|---|
|
| 42 |
-
| gemma3:27b
|
| 43 |
-
| **referee-hands-s
|
| 44 |
| qwen2.5vl:32b | 32B | 74.4% |
|
| 45 |
-
| *
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
| MediaPipe HandLandmarker | — | 40.0% |
|
| 49 |
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
|
| 56 |
## Usage
|
| 57 |
|
|
@@ -64,17 +63,11 @@ net = models.mobilenet_v3_small()
|
|
| 64 |
net.classifier[3] = torch.nn.Linear(net.classifier[3].in_features, 1)
|
| 65 |
net.load_state_dict(load_file("model.safetensors"))
|
| 66 |
net.eval()
|
| 67 |
-
|
| 68 |
tf = transforms.Compose([transforms.Resize((96, 96)), transforms.ToTensor()])
|
| 69 |
p = torch.sigmoid(net(tf(img).unsqueeze(0)).squeeze()) # P(hand/gripper visible)
|
| 70 |
```
|
| 71 |
|
| 72 |
-
Clip-level: sample ~12 frames,
|
| 73 |
-
|
| 74 |
-
## Provenance
|
| 75 |
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
validation gates, and all baseline runs: https://github.com/publu/referee-lab —
|
| 79 |
-
five capture-QA evals, publicly runnable on the Prime Intellect Environments Hub
|
| 80 |
-
(`prime eval run referee-lab/hands-visible`).
|
|
|
|
| 4 |
pipeline_tag: image-classification
|
| 5 |
tags:
|
| 6 |
- robotics
|
|
|
|
| 7 |
- capture-qa
|
| 8 |
- hand-detection
|
| 9 |
- edge
|
|
|
|
| 10 |
datasets:
|
| 11 |
- lerobot/droid_1.0.1
|
| 12 |
- imageomics/KABR
|
|
|
|
| 18 |
name: Hand/gripper visibility (clip-level)
|
| 19 |
dataset:
|
| 20 |
type: referee-lab/hands-visible
|
| 21 |
+
name: referee-lab/hands-visible
|
| 22 |
split: full
|
| 23 |
metrics:
|
| 24 |
- type: accuracy
|
|
|
|
| 28 |
|
| 29 |
# referee-hands-s
|
| 30 |
|
| 31 |
+
A 1.5M-parameter MobileNetV3-small classifier for hand/gripper visibility in
|
| 32 |
+
robot-training footage. 96px inputs, sized for microcontroller-class deployment.
|
|
|
|
|
|
|
| 33 |
|
| 34 |
+
## Benchmark
|
| 35 |
+
|
| 36 |
+
97.8% clip accuracy on [referee-lab/hands-visible](https://github.com/publu/referee-lab)
|
| 37 |
+
(45-clip public benchmark, robot + wildlife domains) — matching gemma3:27b at
|
| 38 |
+
1/18,000th the parameters.
|
| 39 |
|
| 40 |
| Model | Params | Accuracy |
|
| 41 |
|---|---|---|
|
| 42 |
+
| gemma3:27b | 27B | 97.8% |
|
| 43 |
+
| **referee-hands-s** | **1.5M** | **97.8%** |
|
| 44 |
| qwen2.5vl:32b | 32B | 74.4% |
|
| 45 |
+
| *constant-answer floor* | — | *68.9%* |
|
| 46 |
+
|
| 47 |
+
## Intended use and scope
|
|
|
|
| 48 |
|
| 49 |
+
Version 1 targets fixed-camera and wrist-camera **robot workspace footage** —
|
| 50 |
+
the domain it was trained on (DROID, ALOHA-sim, KABR negatives). Human
|
| 51 |
+
egocentric and general webcam scenes are outside the v1 training distribution;
|
| 52 |
+
a mixed-domain v2 is in development. Use for capture-pipeline gating in the
|
| 53 |
+
supported domain.
|
| 54 |
|
| 55 |
## Usage
|
| 56 |
|
|
|
|
| 63 |
net.classifier[3] = torch.nn.Linear(net.classifier[3].in_features, 1)
|
| 64 |
net.load_state_dict(load_file("model.safetensors"))
|
| 65 |
net.eval()
|
|
|
|
| 66 |
tf = transforms.Compose([transforms.Resize((96, 96)), transforms.ToTensor()])
|
| 67 |
p = torch.sigmoid(net(tf(img).unsqueeze(0)).squeeze()) # P(hand/gripper visible)
|
| 68 |
```
|
| 69 |
|
| 70 |
+
Clip-level verdict: sample ~12 frames, yes if `max(p) > 0.5`.
|
|
|
|
|
|
|
| 71 |
|
| 72 |
+
Training: distilled from gemma3:27b pseudo-labels on CC BY / CC0 / MIT media.
|
| 73 |
+
Benchmark, gates, and tooling: https://github.com/publu/referee-lab
|
|
|
|
|
|
|
|
|