itspublu commited on
Commit
7a973dd
·
verified ·
1 Parent(s): 8bdd6bc

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +80 -0
  2. model.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: pytorch
4
+ pipeline_tag: image-classification
5
+ tags:
6
+ - robotics
7
+ - egocentric-video
8
+ - capture-qa
9
+ - hand-detection
10
+ - edge
11
+ - tinyml
12
+ datasets:
13
+ - lerobot/droid_1.0.1
14
+ - imageomics/KABR
15
+ model-index:
16
+ - name: referee-hands-s
17
+ results:
18
+ - task:
19
+ type: image-classification
20
+ name: Hand/gripper visibility (clip-level)
21
+ dataset:
22
+ type: referee-lab/hands-visible
23
+ name: referee-lab/hands-visible (Prime Intellect Environments Hub)
24
+ split: full
25
+ metrics:
26
+ - type: accuracy
27
+ name: Clip accuracy
28
+ value: 0.9778
29
+ ---
30
+
31
+ # referee-hands-s
32
+
33
+ A **1.5M-parameter** MobileNetV3-small that answers one question about robot-training
34
+ video: *is a hand or gripper visible?* Distilled from gemma3:27b pseudo-labels
35
+ (4,138 frames from DROID / ALOHA-sim / KABR — all CC BY / CC0 / MIT), 96px inputs,
36
+ built to fit a microcontroller inference budget.
37
+
38
+ ## Scoreboard (public benchmark: [referee-lab/hands-visible](https://github.com/publu/referee-lab))
39
+
40
+ | Model | Params | Accuracy |
41
+ |---|---|---|
42
+ | gemma3:27b (the teacher) | 27B | 97.8% |
43
+ | **referee-hands-s (this model)** | **1.5M** | **97.8%** |
44
+ | qwen2.5vl:32b | 32B | 74.4% |
45
+ | *do-nothing floor (constant "yes")* | — | *68.9%* |
46
+ | qwen2.5vl:7b | 7B | 45.6% |
47
+ | EgoSieve-S (hand-visibility head) | 22M | 40.0% |
48
+ | MediaPipe HandLandmarker | — | 40.0% |
49
+
50
+ Scored on the public 45-clip answer sheet, clip verdict = any frame above 0.5.
51
+ **Caveat, stated plainly:** training frames come from different clips but the same
52
+ source videos as the benchmark — an in-domain result. A cross-video split is the
53
+ planned follow-up. The benchmark's own labels are machine-graded provisional with a
54
+ published grading record.
55
+
56
+ ## Usage
57
+
58
+ ```python
59
+ import torch
60
+ from torchvision import models, transforms
61
+ from safetensors.torch import load_file
62
+
63
+ net = models.mobilenet_v3_small()
64
+ net.classifier[3] = torch.nn.Linear(net.classifier[3].in_features, 1)
65
+ net.load_state_dict(load_file("model.safetensors"))
66
+ net.eval()
67
+
68
+ tf = transforms.Compose([transforms.Resize((96, 96)), transforms.ToTensor()])
69
+ p = torch.sigmoid(net(tf(img).unsqueeze(0)).squeeze()) # P(hand/gripper visible)
70
+ ```
71
+
72
+ Clip-level: sample ~12 frames, answer yes if `max(p) > 0.5`.
73
+
74
+ ## Provenance
75
+
76
+ Teacher: gemma3:27b (96.7-97.8% on the same benchmark). Training/eval media:
77
+ DROID (CC BY 4.0), lerobot aloha sim (MIT), KABR (CC0). The benchmark, its
78
+ validation gates, and all baseline runs: https://github.com/publu/referee-lab —
79
+ five capture-QA evals, publicly runnable on the Prime Intellect Environments Hub
80
+ (`prime eval run referee-lab/hands-visible`).
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b793f7a6d252d5e81928c7f3ee7c9d168f7feea1b6267c556ea8b7e0951ee90d
3
+ size 6147500