Update README with one-command setup instructions

README.md

@@ -5,185 +5,155 @@
 
 VINE is a video understanding model that processes videos along with categorical, unary, and binary keywords to return probability distributions over those keywords for detected objects and their relationships.
 
-## Quick Start
+## 🚀 One-Command Setup
+
+```bash
+wget https://huggingface.co/video-fm/vine/resolve/main/setup_vine_complete.sh
+bash setup_vine_complete.sh
+```
+
+**That's it!** This single script installs everything you need:
+- ✅ Python environment with all dependencies
+- ✅ SAM2 and GroundingDINO packages
+- ✅ All model checkpoints (~800 MB)
+- ✅ VINE model from HuggingFace (~1.8 GB)
+
+**Total time**: 10-15 minutes | **Total size**: ~2.6 GB
+
+See [QUICKSTART.md](QUICKSTART.md) for detailed instructions.
+
+## Quick Example
 
 ```python
 from transformers import AutoModel
-from vine_hf import VineConfig, VineModel, VinePipeline
+from vine_hf import VinePipeline
+from pathlib import Path
 
-# Load VINE model from HuggingFace
+# Load VINE from HuggingFace
 model = AutoModel.from_pretrained('video-fm/vine', trust_remote_code=True)
 
-# Create pipeline with your checkpoint paths
-vine_pipeline = VinePipeline(
+# Create pipeline (checkpoints downloaded by setup script)
+checkpoint_dir = Path("checkpoints")
+pipeline = VinePipeline(
     model=model,
     tokenizer=None,
-    sam_config_path="/path/to/sam2_config.yaml",
-    sam_checkpoint_path="/path/to/sam2_checkpoint.pt",
-    gd_config_path="/path/to/grounding_dino_config.py",
-    gd_checkpoint_path="/path/to/grounding_dino_checkpoint.pth",
+    sam_config_path=str(checkpoint_dir / "sam2_hiera_t.yaml"),
+    sam_checkpoint_path=str(checkpoint_dir / "sam2_hiera_tiny.pt"),
+    gd_config_path=str(checkpoint_dir / "GroundingDINO_SwinT_OGC.py"),
+    gd_checkpoint_path=str(checkpoint_dir / "groundingdino_swint_ogc.pth"),
     device="cuda",
     trust_remote_code=True
 )
 
-# Process a video
-results = vine_pipeline(
-    'path/to/video.mp4',
-    categorical_keywords=['human', 'dog', 'frisbee'],
+# Process video
+results = pipeline(
+    'video.mp4',
+    categorical_keywords=['person', 'dog', 'ball'],
     unary_keywords=['running', 'jumping'],
-    binary_keywords=['chasing', 'behind'],
-    return_top_k=3
+    binary_keywords=['chasing', 'next to'],
+    return_top_k=5
 )
+
+print(results['summary'])
 ```
 
-## Installation
+## Features
 
-### Option 1: Automated Setup (Recommended)
+- **Categorical Classification**: Classify objects in videos (e.g., "human", "dog", "frisbee")
+- **Unary Predicates**: Detect actions on single objects (e.g., "running", "jumping", "sitting")
+- **Binary Relations**: Detect relationships between object pairs (e.g., "behind", "chasing")
+- **Multi-Modal**: Combines vision (CLIP) with text-based segmentation (GroundingDINO + SAM2)
+- **Visualizations**: Optional annotated video outputs
 
-```bash
-# Download the setup script
-wget https://raw.githubusercontent.com/kevinxuez/vine_hf/main/setup_vine_demo.sh
+## Architecture
 
-# Run the setup
-bash setup_vine_demo.sh
+VINE uses a modular architecture:
 
-# Activate environment
-conda activate vine_demo
 ```
+HuggingFace Hub (video-fm/vine)
+├── VINE model weights (~1.8 GB)
+│   ├── Categorical CLIP (object classification)
+│   ├── Unary CLIP (single-object actions)
+│   └── Binary CLIP (object relationships)
+└── Architecture files
+
+User Environment (via setup script)
+├── Dependencies: laser, sam2, groundingdino
+└── Checkpoints: SAM2 (~149 MB), GroundingDINO (~662 MB)
+```
+
+This separation allows:
+- ✅ Lightweight model distribution
+- ✅ User control over checkpoint versions
+- ✅ Flexible deployment options
+- ✅ Standard HuggingFace practices
+
+## What the Setup Script Does
+
+```bash
+# 1. Creates conda environment (vine_demo)
+# 2. Installs PyTorch with CUDA
+# 3. Clones repositories:
+#    - video-sam2 (SAM2 package)
+#    - GroundingDINO (object detection)
+#    - LASER (video utilities)
+#    - vine_hf (VINE interface)
+# 4. Installs packages in editable mode
+# 5. Downloads model checkpoints:
+#    - sam2_hiera_tiny.pt (~149 MB)
+#    - groundingdino_swint_ogc.pth (~662 MB)
+#    - Config files
+# 6. Tests the installation
+```
+
+## Manual Installation
+
+If you prefer manual installation or need to customize:
 
-### Option 2: Manual Installation
+### 1. Create Environment
 
 ```bash
-# 1. Create conda environment
 conda create -n vine_demo python=3.10 -y
 conda activate vine_demo
-
-# 2. Install PyTorch with CUDA support
 pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu126
+```
+
+### 2. Install Dependencies
 
-# 3. Install core dependencies
-pip install transformers huggingface-hub safetensors
+```bash
+pip install transformers huggingface-hub safetensors opencv-python pillow
+```
 
-# 4. Clone and install required repositories
+### 3. Clone and Install Packages
+
+```bash
 git clone https://github.com/video-fm/video-sam2.git
 git clone https://github.com/video-fm/GroundingDINO.git
 git clone https://github.com/kevinxuez/LASER.git
 git clone https://github.com/kevinxuez/vine_hf.git
 
-# Install in editable mode
 pip install -e ./video-sam2
 pip install -e ./GroundingDINO
 pip install -e ./LASER
 pip install -e ./vine_hf
 
-# Build GroundingDINO extensions
-cd GroundingDINO && python setup.py build_ext --force --inplace && cd ..
+cd GroundingDINO && python setup.py build_ext --inplace && cd ..
 ```
 
-## Required Checkpoints
-
-VINE requires SAM2 and GroundingDINO checkpoints for segmentation. Download these separately:
+### 4. Download Checkpoints
 
-### SAM2 Checkpoint
 ```bash
+mkdir checkpoints && cd checkpoints
+
+# SAM2
 wget https://dl.fbaipublicfiles.com/segment_anything_2/072824/sam2_hiera_tiny.pt
-wget https://raw.githubusercontent.com/facebookresearch/sam2/main/sam2/configs/sam2.1/sam2.1_hiera_t.yaml
-```
+wget https://raw.githubusercontent.com/facebookresearch/sam2/main/sam2/configs/sam2.1/sam2.1_hiera_t.yaml -O sam2_hiera_t.yaml
 
-### GroundingDINO Checkpoint
-```bash
+# GroundingDINO
 wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
 wget https://raw.githubusercontent.com/IDEA-Research/GroundingDINO/main/groundingdino/config/GroundingDINO_SwinT_OGC.py
 ```
 
-## Architecture
-
-```
-video-fm/vine (HuggingFace Hub)
-├── VINE Model Weights (~1.8GB)
-│   ├── Categorical CLIP model (fine-tuned)
-│   ├── Unary CLIP model (fine-tuned)
-│   └── Binary CLIP model (fine-tuned)
-└── Architecture Files
-    ├── vine_config.py
-    ├── vine_model.py
-    ├── vine_pipeline.py
-    └── utilities
-
-User Provides:
-├── Dependencies (via pip/conda)
-│   ├── laser (video processing utilities)
-│   ├── sam2 (segmentation)
-│   └── groundingdino (object detection)
-└── Checkpoints (downloaded separately)
-    ├── SAM2 model files
-    └── GroundingDINO model files
-```
-
-## Why This Architecture?
-
-This separation of concerns provides several benefits:
-
-1. **Lightweight Distribution**: Only VINE-specific weights (~1.8GB) are on HuggingFace
-2. **Version Control**: Users can choose their preferred SAM2/GroundingDINO versions
-3. **Licensing**: Keeps different model licenses separate
-4. **Flexibility**: Easy to swap segmentation backends
-5. **Standard Practice**: Similar to models like LLaVA, BLIP-2, etc.
-
-## Full Usage Example
-
-```python
-import os
-from pathlib import Path
-from transformers import AutoModel
-from vine_hf import VinePipeline
-
-# Set up paths
-checkpoint_dir = Path("/path/to/checkpoints")
-sam_config = checkpoint_dir / "sam2_hiera_t.yaml"
-sam_checkpoint = checkpoint_dir / "sam2_hiera_tiny.pt"
-gd_config = checkpoint_dir / "GroundingDINO_SwinT_OGC.py"
-gd_checkpoint = checkpoint_dir / "groundingdino_swint_ogc.pth"
-
-# Load VINE from HuggingFace
-model = AutoModel.from_pretrained('video-fm/vine', trust_remote_code=True)
-
-# Create pipeline
-vine_pipeline = VinePipeline(
-    model=model,
-    tokenizer=None,
-    sam_config_path=str(sam_config),
-    sam_checkpoint_path=str(sam_checkpoint),
-    gd_config_path=str(gd_config),
-    gd_checkpoint_path=str(gd_checkpoint),
-    device="cuda:0",
-    trust_remote_code=True
-)
-
-# Process video
-results = vine_pipeline(
-    "path/to/video.mp4",
-    categorical_keywords=['person', 'dog', 'ball'],
-    unary_keywords=['running', 'jumping', 'sitting'],
-    binary_keywords=['chasing', 'next to', 'holding'],
-    object_pairs=[(0, 1), (0, 2)],  # person-dog, person-ball
-    return_top_k=5,
-    include_visualizations=True
-)
-
-# Access results
-print(f"Detected {results['summary']['num_objects_detected']} objects")
-print(f"Top categories: {results['summary']['top_categories']}")
-print(f"Top actions: {results['summary']['top_actions']}")
-print(f"Top relations: {results['summary']['top_relations']}")
-
-# Access detailed predictions
-for obj_id, predictions in results['categorical_predictions'].items():
-    print(f"\nObject {obj_id}:")
-    for prob, category in predictions:
-        print(f"  {category}: {prob:.3f}")
-```
-
 ## Output Format
 
 ```python
@@ -197,27 +167,55 @@ for obj_id, predictions in results['categorical_predictions'].items():
     "binary_predictions": {
         (frame_id, (obj1_id, obj2_id)): [(probability, relation), ...]
     },
-    "confidence_scores": {
-        "categorical": float,
-        "unary": float,
-        "binary": float
-    },
     "summary": {
         "num_objects_detected": int,
         "top_categories": [(category, probability), ...],
         "top_actions": [(action, probability), ...],
         "top_relations": [(relation, probability), ...]
-    },
-    "visualizations": {  # if include_visualizations=True
-        "vine": {
-            "all": {"frames": [...], "video_path": "..."},
-            ...
-        }
     }
 }
 ```
 
-## Configuration Options
+## Advanced Usage
+
+### Custom Segmentation
+
+```python
+# Use your own masks and bounding boxes
+results = model.predict(
+    video_frames=frames,
+    masks=your_masks,
+    bboxes=your_bboxes,
+    categorical_keywords=['person', 'dog'],
+    unary_keywords=['running'],
+    binary_keywords=['chasing']
+)
+```
+
+### SAM2 Only (No GroundingDINO)
+
+```python
+config = VineConfig(
+    segmentation_method="sam2",  # Uses SAM2 automatic mask generation
+    ...
+)
+```
+
+### Enable Visualizations
+
+```python
+results = pipeline(
+    'video.mp4',
+    categorical_keywords=['person', 'dog'],
+    include_visualizations=True,  # Creates annotated video
+    return_top_k=5
+)
+
+# Access annotated video
+video_path = results['visualizations']['vine']['all']['video_path']
+```
+
+## Configuration
 
 ```python
 from vine_hf import VineConfig
@@ -225,94 +223,127 @@ from vine_hf import VineConfig
 config = VineConfig(
     model_name="openai/clip-vit-base-patch32",  # CLIP backbone
     segmentation_method="grounding_dino_sam2",   # or "sam2"
-    box_threshold=0.35,                          # GroundingDINO threshold
-    text_threshold=0.25,                         # GroundingDINO threshold
+    box_threshold=0.35,                          # Detection threshold
+    text_threshold=0.25,                         # Text matching threshold
     target_fps=5,                                # Video sampling rate
     visualize=True,                              # Enable visualizations
     visualization_dir="outputs/",                # Output directory
-    debug_visualizations=False,                  # Debug mode
     device="cuda:0"                              # Device
 )
 ```
 
-## Deployment Examples
+## System Requirements
+
+- **OS**: Linux (Ubuntu 20.04+)
+- **Python**: 3.10+
+- **CUDA**: 11.8+ (for GPU)
+- **GPU**: 8GB+ VRAM (T4, V100, A100)
+- **RAM**: 16GB+
+- **Disk**: ~5GB free
+
+## Troubleshooting
+
+### CUDA Not Available
 
-### Local Script
 ```python
-# test_vine.py
-from transformers import AutoModel
-from vine_hf import VinePipeline
+import torch
+print(torch.cuda.is_available())  # Should be True
+```
 
-model = AutoModel.from_pretrained('video-fm/vine', trust_remote_code=True)
-pipeline = VinePipeline(model=model, ...)
-results = pipeline("video.mp4", ...)
+### Import Errors
+
+```bash
+conda activate vine_demo
+pip list | grep -E "laser|sam2|groundingdino"
 ```
 
-### HuggingFace Spaces
-```python
-# app.py for Gradio Space
-import gradio as gr
-from transformers import AutoModel
-from vine_hf import VinePipeline
+### Checkpoint Not Found
 
-model = AutoModel.from_pretrained('video-fm/vine', trust_remote_code=True)
-# ... set up pipeline and Gradio interface
+```bash
+ls -lh checkpoints/
+# Should show: sam2_hiera_tiny.pt, groundingdino_swint_ogc.pth
 ```
 
-### API Server
-```python
-# FastAPI server
-from fastapi import FastAPI
-from transformers import AutoModel
-from vine_hf import VinePipeline
+See [QUICKSTART.md](QUICKSTART.md) for detailed troubleshooting.
 
-app = FastAPI()
-model = AutoModel.from_pretrained('video-fm/vine', trust_remote_code=True)
-pipeline = VinePipeline(model=model, ...)
+## Example Applications
+
+### Sports Analysis
 
-@app.post("/process")
-async def process_video(video_path: str):
-    return pipeline(video_path, ...)
+```python
+results = pipeline(
+    'soccer_game.mp4',
+    categorical_keywords=['player', 'ball', 'referee'],
+    unary_keywords=['running', 'kicking', 'jumping'],
+    binary_keywords=['passing', 'tackling', 'defending']
+)
 ```
 
-## Troubleshooting
+### Surveillance
 
-### Import Errors
-```bash
-# Make sure all dependencies are installed
-pip list | grep -E "laser|sam2|groundingdino"
+```python
+results = pipeline(
+    'security_feed.mp4',
+    categorical_keywords=['person', 'vehicle', 'bag'],
+    unary_keywords=['walking', 'running', 'standing'],
+    binary_keywords=['approaching', 'following', 'carrying']
+)
+```
 
-# Reinstall if needed
-pip install -e ./LASER
-pip install -e ./video-sam2
-pip install -e ./GroundingDINO
+### Animal Behavior
+
+```python
+results = pipeline(
+    'wildlife.mp4',
+    categorical_keywords=['lion', 'zebra', 'elephant'],
+    unary_keywords=['eating', 'walking', 'resting'],
+    binary_keywords=['hunting', 'fleeing', 'protecting']
+)
 ```
 
-### CUDA Errors
+## Deployment
+
+### Gradio Demo
+
 ```python
-# Check CUDA availability
-import torch
-print(torch.cuda.is_available())
-print(torch.version.cuda)
+import gradio as gr
 
-# Use CPU if needed
-pipeline = VinePipeline(model=model, device="cpu", ...)
+def analyze_video(video, categories, actions, relations):
+    results = pipeline(
+        video,
+        categorical_keywords=categories.split(','),
+        unary_keywords=actions.split(','),
+        binary_keywords=relations.split(',')
+    )
+    return results['summary']
+
+gr.Interface(analyze_video, ...).launch()
 ```
 
-### Checkpoint Not Found
-```bash
-# Verify checkpoint paths
-ls -lh /path/to/sam2_hiera_tiny.pt
-ls -lh /path/to/groundingdino_swint_ogc.pth
+### FastAPI Server
+
+```python
+from fastapi import FastAPI
+
+app = FastAPI()
+model = AutoModel.from_pretrained('video-fm/vine', trust_remote_code=True)
+pipeline = VinePipeline(model=model, ...)
+
+@app.post("/analyze")
+async def analyze(video_path: str, keywords: dict):
+    return pipeline(video_path, **keywords)
 ```
 
-## System Requirements
+## Files in This Repository
 
-- **Python**: 3.10+
-- **CUDA**: 11.8+ (for GPU)
-- **GPU**: 8GB+ VRAM recommended (T4, V100, A100, etc.)
-- **RAM**: 16GB+ recommended
-- **Storage**: ~3GB for checkpoints
+- `setup_vine_complete.sh` - One-command setup script
+- `QUICKSTART.md` - Quick start guide
+- `README.md` - This file (complete documentation)
+- `vine_config.py` - VineConfig class
+- `vine_model.py` - VineModel class
+- `vine_pipeline.py` - VinePipeline class
+- `flattening.py` - Segment processing utilities
+- `vis_utils.py` - Visualization utilities
 
 ## Citation
 
@@ -327,19 +358,21 @@ ls -lh /path/to/groundingdino_swint_ogc.pth
 
 ## License
 
-This model and code are released under the MIT License. Note that SAM2 and GroundingDINO have their own respective licenses.
+This model is released under the MIT License. Note that SAM2 and GroundingDINO have their own respective licenses.
 
 ## Links
 
 - **Model**: https://huggingface.co/video-fm/vine
-- **Code**: https://github.com/kevinxuez/LASER
-- **vine_hf Package**: https://github.com/kevinxuez/vine_hf
-- **SAM2**: https://github.com/facebookresearch/sam2
-- **GroundingDINO**: https://github.com/IDEA-Research/GroundingDINO
+- **Quick Start**: [QUICKSTART.md](QUICKSTART.md)
+- **Setup Script**: [setup_vine_complete.sh](setup_vine_complete.sh)
+- **LASER GitHub**: https://github.com/kevinxuez/LASER
+- **Issues**: https://github.com/kevinxuez/LASER/issues
 
 ## Support
 
-For issues or questions:
-- **Model/Architecture**: [HuggingFace Discussions](https://huggingface.co/video-fm/vine/discussions)
-- **LASER Framework**: [GitHub Issues](https://github.com/kevinxuez/LASER/issues)
-- **vine_hf Package**: [GitHub Issues](https://github.com/kevinxuez/vine_hf/issues)
+- **Questions**: [HuggingFace Discussions](https://huggingface.co/video-fm/vine/discussions)
+- **Bugs**: [GitHub Issues](https://github.com/kevinxuez/LASER/issues)
+
+---
+
+**Made with ❤️ by the LASER team**