|
|
--- |
|
|
title: Trajectory Reviewer |
|
|
emoji: 🎯 |
|
|
colorFrom: blue |
|
|
colorTo: purple |
|
|
sdk: gradio |
|
|
sdk_version: 6.1.0 |
|
|
app_file: app.py |
|
|
pinned: false |
|
|
short_description: Review and filter training data trajectories |
|
|
--- |
|
|
|
|
|
# 🎯 Trajectory Reviewer |
|
|
|
|
|
Quick dataset curation tool for assessing trajectory quality. Designed for rapid keep/remove decisions on training data. |
|
|
|
|
|
## Features |
|
|
|
|
|
- **Quick Decisions**: One-click Keep/Remove/Review buttons |
|
|
- **Issue Tracking**: Flag task mismatches, already completed tasks, and custom issues |
|
|
- **Smart Ledger**: CSV-based tracking with live updates |
|
|
- **Shared Dataset**: Collect evaluations from all users into centralized HF Dataset |
|
|
- **HF Space Sync**: Auto-commits evaluations to Space repository |
|
|
- **Playback Speed**: Adjustable 1x/2x/4x speed for efficient review |
|
|
- **Auto-advance**: Automatically loads next trajectory after save |
|
|
|
|
|
## Usage |
|
|
|
|
|
1. **Load Dataset**: Enter HuggingFace dataset repo and number of samples |
|
|
2. **Review**: Watch video (use 2-3x playback speed for efficiency) |
|
|
3. **Decide**: Click Keep/Remove/Review button |
|
|
4. **Add Details** (optional): Flag issues and add notes for downstream analysis |
|
|
5. **Continue**: App auto-advances to next trajectory |
|
|
|
|
|
## Workflow |
|
|
|
|
|
``` |
|
|
Load 20 samples → Review video → Quick decision → Auto-advance |
|
|
``` |
|
|
|
|
|
**Speed**: ~20-30 seconds per trajectory |
|
|
|
|
|
## Output |
|
|
|
|
|
All evaluations saved to `evaluations.csv`: |
|
|
|
|
|
```csv |
|
|
dataset_repo,config_name,trajectory_id,task,decision,issue_type,notes,timestamp |
|
|
``` |
|
|
|
|
|
Use the **Download** button in the table to export your evaluations. |
|
|
|
|
|
## Tips |
|
|
|
|
|
- Use quick buttons (Keep/Remove/Review) for most trajectories |
|
|
- Add detailed notes only when needed for issue aggregation |
|
|
- Use 2x/4x playback speed buttons for faster review |
|
|
|
|
|
## Data Persistence |
|
|
|
|
|
### Shared HF Dataset |
|
|
|
|
|
**Collect all users' evaluations into one centralized dataset**: |
|
|
|
|
|
1. Create dataset: `huggingface-cli repo create traj-evaluations --type dataset` |
|
|
2. Add to Space secrets: |
|
|
- `HF_TOKEN`: Your HF write token |
|
|
- `EVAL_DATASET_REPO`: `username/traj-evaluations` |
|
|
3. Deploy Space |
|
|
|
|
|
All evaluations automatically append to the shared dataset. |
|
|
|
|
|
### CSV Auto-Commit |
|
|
|
|
|
**On HF Space**: Evaluations CSV automatically commits to the Space's git repository (zero config!) |
|
|
|
|
|
Check the Files tab in your Space to download accumulated evaluations. |
|
|
|
|
|
## Related Tools |
|
|
|
|
|
- **Trajectory Endpoint Labeler**: For precise frame-level endpoint labeling |
|
|
- Use this reviewer first for quality filtering |
|
|
|
|
|
--- |
|
|
|
|
|
Built with Gradio, OpenCV, and HuggingFace Datasets |
|
|
|
