README.md

metadata

title: Trajectory Reviewer
emoji: 🎯
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.1.0
app_file: app.py
pinned: false
short_description: Review and filter training data trajectories

🎯 Trajectory Reviewer

Quick dataset curation tool for assessing trajectory quality. Designed for rapid keep/remove decisions on training data.

Features

• Quick Decisions: One-click Keep/Remove/Review buttons
• Issue Tracking: Flag task mismatches, already completed tasks, and custom issues
• Smart Ledger: CSV-based tracking with live updates
• Shared Dataset: Collect evaluations from all users into centralized HF Dataset
• HF Space Sync: Auto-commits evaluations to Space repository
• Playback Speed: Adjustable 1x/2x/4x speed for efficient review
• Auto-advance: Automatically loads next trajectory after save

Usage

1. Load Dataset: Enter HuggingFace dataset repo and number of samples
2. Review: Watch video (use 2-3x playback speed for efficiency)
3. Decide: Click Keep/Remove/Review button
4. Add Details (optional): Flag issues and add notes for downstream analysis
5. Continue: App auto-advances to next trajectory

Workflow

Load 20 samples → Review video → Quick decision → Auto-advance

Speed: ~20-30 seconds per trajectory

Output

All evaluations saved to evaluations.csv:

dataset_repo,config_name,trajectory_id,task,decision,issue_type,notes,timestamp

Use the Download button in the table to export your evaluations.

Tips

• Use quick buttons (Keep/Remove/Review) for most trajectories
• Add detailed notes only when needed for issue aggregation
• Use 2x/4x playback speed buttons for faster review

Data Persistence

Shared HF Dataset

Collect all users' evaluations into one centralized dataset:

1. Create dataset: huggingface-cli repo create traj-evaluations --type dataset
2. Add to Space secrets:
   • HF_TOKEN: Your HF write token
   • EVAL_DATASET_REPO: username/traj-evaluations
3. Deploy Space

All evaluations automatically append to the shared dataset.

CSV Auto-Commit

On HF Space: Evaluations CSV automatically commits to the Space's git repository (zero config!)

Check the Files tab in your Space to download accumulated evaluations.

Related Tools

• Trajectory Endpoint Labeler: For precise frame-level endpoint labeling
• Use this reviewer first for quality filtering

---

Built with Gradio, OpenCV, and HuggingFace Datasets