File size: 2,760 Bytes
7a973dd ea89fb9 7a973dd b8cb229 7a973dd ea89fb9 7a973dd b8cb229 ea89fb9 b8cb229 7a973dd ea89fb9 cafc462 7a973dd ea89fb9 cafc462 ea89fb9 7a973dd ea89fb9 7a973dd ea89fb9 7a973dd ea89fb9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
license: apache-2.0
library_name: pytorch
pipeline_tag: image-classification
tags:
- robotics
- capture-qa
- hand-detection
- edge
datasets:
- lerobot/droid_1.0.1
- imageomics/KABR
model-index:
- name: referee-hands-s
results:
- task:
type: image-classification
name: Hand/gripper visibility (clip-level)
dataset:
type: referee-lab/hands-visible
name: referee-lab/hands-visible
split: full
metrics:
- type: accuracy
name: Provisional development-sheet clip accuracy
value: 0.9778
---
# referee-hands-s
A 1.5M-parameter MobileNetV3-small classifier for hand/gripper visibility in
robot-training footage. 96px inputs, sized for microcontroller-class deployment.
The published 97.8% result is from a 45-clip, in-domain development sheet that
was repeatedly consulted during development and has not yet passed the declared
independent two-grader gate. It is reproducible development evidence, not a
frozen external benchmark.
## Benchmark
97.8% clip accuracy on
[referee-lab/hands-visible](https://github.com/publu/referee-lab) (45-clip
provisional development sheet, robot + wildlife domains) — matching gemma3:27b
on that sheet at 1/18,000th the parameters.
| Model | Params | Accuracy |
|---|---|---|
| gemma3:27b | 27B | 97.8% |
| **referee-hands-s** | **1.5M** | **97.8%** |
| referee-hands-s, cross-video split | 1.5M | 95.6% |
| qwen2.5vl:32b | 32B | 74.4% |
| *constant-answer floor* | — | *68.9%* |
The cross-video row keeps every training frame at least 60 seconds away from
any test clip on its source stream. Under that separation the model scores
100% on real footage (41/41); its two misses are synthetic drawn-hand control
clips outside the training distribution.
## Intended use and scope
Version 1 targets fixed-camera and wrist-camera **robot workspace footage** —
the domain it was trained on (DROID, ALOHA-sim, KABR negatives). Human
egocentric and general webcam scenes are outside the v1 training distribution;
a mixed-domain v2 is in development. Use for capture-pipeline gating in the
supported domain.
## Usage
```python
import torch
from torchvision import models, transforms
from safetensors.torch import load_file
net = models.mobilenet_v3_small()
net.classifier[3] = torch.nn.Linear(net.classifier[3].in_features, 1)
net.load_state_dict(load_file("model.safetensors"))
net.eval()
tf = transforms.Compose([transforms.Resize((96, 96)), transforms.ToTensor()])
p = torch.sigmoid(net(tf(img).unsqueeze(0)).squeeze()) # P(hand/gripper visible)
```
Clip-level verdict: sample ~12 frames, yes if `max(p) > 0.5`.
Training: distilled from gemma3:27b pseudo-labels on CC BY / CC0 / MIT media.
Benchmark, gates, and tooling: https://github.com/publu/referee-lab
|