README.md

SAM 2 Few-Shot/Zero-Shot Segmentation Research

This repository contains research on combining Segment Anything Model 2 (SAM 2) with minimal supervision for domain-specific segmentation tasks.

Research Overview

The goal is to study how SAM 2 can be adapted to new object categories in specific domains (satellite imagery, fashion, robotics) using:

• Few-shot learning: 1-10 labeled examples per class
• Zero-shot learning: No labeled examples, using text prompts and visual similarity

Key Research Areas

1. Domain Adaptation

• Satellite Imagery: Buildings, roads, vegetation, water bodies
• Fashion: Clothing items, accessories, patterns
• Robotics: Industrial objects, tools, safety equipment

2. Learning Paradigms

• Prompt Engineering: Optimizing text prompts for SAM 2
• Visual Similarity: Using CLIP embeddings for zero-shot transfer
• Meta-learning: Learning to adapt quickly to new domains

3. Evaluation Metrics

• IoU (Intersection over Union)
• Dice Coefficient
• Boundary Accuracy
• Domain-specific metrics

Project Structure

├── data/                   # Dataset storage
├── models/                 # Model implementations
├── experiments/           # Experiment configurations
├── utils/                 # Utility functions
├── notebooks/             # Jupyter notebooks for analysis
├── results/               # Experiment results and visualizations
└── requirements.txt       # Dependencies

Quick Start

1. Install dependencies:

   pip install -r requirements.txt
   

2. Download SAM 2:

   python scripts/download_sam2.py
   

3. Run few-shot experiment:

   python experiments/few_shot_satellite.py
   

4. Run zero-shot experiment:

   python experiments/zero_shot_fashion.py
   

Research Papers

This work builds upon:

• SAM 2: Segment Anything Model 2
• CLIP: Learning Transferable Visual Representations
• Few-shot Learning for Semantic Segmentation

Contributing

Please read our contributing guidelines and code of conduct before submitting pull requests.

License

MIT License - see LICENSE file for details.