README.md

---
title: PhotoBench-Protected
emoji: 🛡️
colorFrom: green
colorTo: green
sdk: gradio
app_file: app.py
pinned: true
license: apache-2.0
short_description: PhotoBench-Protected agent-only leaderboard
tags:
- leaderboard
- image-retrieval
- benchmark
- agent
---

# PhotoBench-Protected Leaderboard

Agent-only leaderboard for PhotoBench-Protected, where only partial captions, embeddings, and metadata are provided.

## Quick Start

1. Download the protected dataset from [PhotoBench-Protected Dataset](https://huggingface.co/datasets/SorrowTea/PhotoBench-Protected)
2. Build your agent-based retrieval system using the provided features
3. Submit predictions in JSON format

## Important Notice

**PhotoBench-Protected** is our initial open-source release with limited information sources. This benchmark focuses exclusively on **agent planning** ability.

- For **unrestricted retrieval with raw images**, please use the [full PhotoBench Leaderboard](https://huggingface.co/spaces/SorrowTea/PhotoBench/).
- The test sets for PhotoBench-Protected and PhotoBench (full) are **different**.
- Please confirm you are submitting to the correct leaderboard before uploading.

## Evaluation Metrics

- **Recall@k** for k ∈ {1, 5, 10, 20, 50, 100}
- **NDCG@k** for k ∈ {1, 5, 10, 20, 50, 100}