| --- |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: video-classification |
| base_model: facebook/dinov2-small |
| datasets: |
| - itspublu/EgoSieve-Eval |
| tags: |
| - robotics |
| - egocentric-video |
| - video-quality |
| - dataset-curation |
| - physical-ai |
| model-index: |
| - name: EgoSieve-S |
| results: |
| - task: |
| type: video-classification |
| name: Egocentric video readiness |
| dataset: |
| type: itspublu/EgoSieve-Eval |
| name: EgoSieve-Eval |
| split: test |
| metrics: |
| - type: f1 |
| name: Readiness macro F1 |
| value: 0.67394871 |
| - type: roc_auc |
| name: Issue macro AUROC |
| value: 0.74073329 |
| - type: average_precision |
| name: Issue macro average precision |
| value: 0.72219606 |
| - type: f1 |
| name: Boundary micro F1 |
| value: 0.10303030 |
| --- |
| |
| # EgoSieve-S |
|
|
| EgoSieve-S ranks manipulation-ready spans in first-person video. It produces |
| three readiness logits (`KEEP`, `REVIEW`, `REJECT`), seven observable issue |
| scores, diagnostic start/end boundary proposals, and a normalized retrieval |
| embedding. It is a dataset-curation model, not a robot policy. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForVideoClassification, AutoProcessor |
| |
| processor = AutoProcessor.from_pretrained( |
| "itspublu/EgoSieve-S", revision="v0.1.0", trust_remote_code=True |
| ) |
| model = AutoModelForVideoClassification.from_pretrained( |
| "itspublu/EgoSieve-S", revision="v0.1.0", trust_remote_code=True |
| ).eval() |
| outputs = model(**processor(frames, return_tensors="pt")) |
| ``` |
|
|
| The timestamp-aware scanner and JSONL compiler are provided by the `egosieve` |
| package. The checkpoint expects 12 center-sampled |
| RGB frames per window; use its bundled processor. |
|
|
| ## Training and evaluation |
|
|
| Data represented in the held-out evaluation: HoloAssist (CDLA-Permissive-2.0), HoloAssist controlled corruptions (CDLA-Permissive-2.0). Splits are grouped by |
| original capture unit. Readiness, calibration, and boundary results use |
| 142 human-grounded readiness rows: |
| 0 direct human and |
| 142 human-derived. Boundary results use |
| 97 human-grounded boundary rows. Issue results |
| use 145 labeled rows: 0 |
| human, 109 human-derived, and |
| 36 programmatic |
| controlled corruptions. Unlabeled task targets are masked. The release bundle |
| includes raw held-out predictions with task-level provenance, split |
| assignments, exact run configuration, and metric provenance. |
|
|
| Human-derived rows are not direct, independent EgoSieve rubric judgments. Treat |
| them as proxy evidence and consult the source dataset cards and their |
| dataset-specific proxy details before comparing or interpreting these metrics. |
|
|
| For v0.1, readiness and boundaries come from a fixed-grid occupancy rule over |
| reviewed HoloAssist fine-action intervals. `low_hand_activity` is an occupancy |
| proxy, and `acting_hand_not_visible` follows HoloAssist's acting-hand modifier; |
| it does not assert that every hand is absent. The other five issue metrics |
| measure injected-corruption versus unmodified-reference discrimination. Those |
| references were not independently audited as natural issue negatives. |
|
|
| - Readiness macro F1: 0.6739 |
| - Issue macro AUROC: 0.7407 |
| - Issue macro average precision: 0.7222 |
| - Boundary micro F1 at 0.30s: 0.1030 |
| - Readiness ECE: 0.1018 |
|
|
| ## Intended use and limitations |
|
|
| Use the model to rank raw egocentric windows, route uncertain spans for review, |
| and create embeddings for near-duplicate search. Readiness remains dependent on |
| the published rubric and capture domain. RGB cannot establish force, physical |
| success, consent, safety, metric depth, or legal publishability. Boundary |
| scores are proposals and are diagnostic-only in the v0.1 compiler. |
|
|
| First-person recordings can contain faces, screens, homes, and bystanders. |
| Apply a separate privacy and consent review before sharing any media. |
|
|
