ImageDeduper / README.md
basilbenny1002's picture
Upload 11 files
a05a818 verified
metadata
title: Image Selector Backend
emoji: πŸ–ΌοΈ
colorFrom: blue
colorTo: purple
sdk: docker
app_port: 7860
pinned: false
license: mit

Image Selector Backend API

A FastAPI backend for intelligent image deduplication and selection. Upload images, group similar ones using ResNet50 embeddings, and keep only the best image from each group based on aesthetics scoring.

πŸš€ Features

  • Smart Deduplication: Groups similar images using deep learning embeddings
  • Aesthetics Scoring: Ranks images within each group using a CLIP-based aesthetics model
  • REST API: Simple endpoints for upload, process, progress tracking, and download
  • Per-User Sessions: Each user gets isolated processing and temporary storage
  • Auto Cleanup: Temporary files are removed after download

πŸ“‘ API Endpoints

Health Check

GET /

Returns: {"status": "ok"}

Upload Images

POST /upload

Form fields:

  • user_id (text): Your unique session ID
  • file (file): Image file to upload

Start Processing

POST /process

Form fields:

  • user_id (text): Your session ID
  • similarity (float, optional): Similarity threshold (default: 0.87)
  • use_aesthetics (bool, optional): Enable aesthetics scoring (default: true)

Check Progress

GET /progress/{user_id}

Returns processing status, percentage, and ETA.

Download Results

GET /download/{user_id}

Returns a ZIP file containing the best images from each group.

πŸ”§ How It Works

  1. Upload: Send images with a unique user_id
  2. Process: Trigger processing - the backend will:
    • Generate embeddings for all images
    • Group similar images (similarity threshold configurable)
    • Score each image using an aesthetics model
    • Keep the best image from each group
  3. Download: Retrieve a ZIP of selected images
  4. Auto-cleanup: All temporary files are deleted after download

πŸ’» Local Development

git clone <this-repo>
cd <repo-folder>
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 7860

🎨 Frontend

Use with the web UI:

πŸ“¦ Tech Stack

  • FastAPI: Modern async web framework
  • PyTorch: Deep learning for embeddings
  • ResNet50: Feature extraction model
  • CLIP: Aesthetics prediction model
  • SQLite: Temporary embeddings storage

⚠️ Notes for HuggingFace Spaces

  • Uses /tmp for ephemeral storage (free tier)
  • For persistent storage, upgrade to a paid tier and data will use /data
  • Model weights download on first run (~500MB for ResNet50 + aesthetics model)
  • GPU recommended for faster processing (upgrade to T4 or better)

πŸ“„ License

MIT License - see LICENSE file

πŸ”— Project Origin

Based on Image-Selecter