File size: 2,183 Bytes
745e788
65b059f
 
ae7277c
 
745e788
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
---
title: AgentBase Search Platform
emoji: πŸ”
colorFrom: indigo
colorTo: purple
sdk: streamlit
sdk_version: 1.44.0
app_file: streamlit_app.py
pinned: false
---

# AgentBase Search Platform

**AgentBase** is an interactive search demo for AI agent retrieval, built on nearly 10,000 real-world agents sourced from the [GPT Store](https://chatgpt.com/gpts), [Google Cloud Marketplace](https://cloud.google.com/marketplace), and [AgentAI Platform](https://agent.ai/).

🌐 [Project Page](https://bingo-w.github.io/AgentSearchBench) β€’ πŸ’» [Codebase](https://github.com/Bingo-W/AgentSearchBench) β€’ πŸ“„ [Paper](https://arxiv.org/abs/2604.22436)

---

## Overview

This Space is a live demo for **AgentSearchBench**: a large-scale benchmark for AI agent search. Given a natural language query, the platform retrieves the most relevant agents from AgentBase using a choice of retrieval models and indexing configurations.

**Retrieval models:**
- **BGE-Large** β€” dense retrieval using `BAAI/bge-large-en-v1.5`
- **ToolRet** β€” dense retrieval fine-tuned for tool search
- **BM25** β€” sparse retrieval baseline
- **Keyword** β€” boolean keyword matching

**Indexing configurations:**
- **v1** β€” all metadata columns with priority ordering (name and description weighted first)
- **naive** β€” agent name and description only

---

## Related Resources

| Resource | Description |
|----------|-------------|
| [AgentSearchBench-Tasks](https://huggingface.co/datasets/AgentSearch/AgentSearchBench-Tasks) | Benchmark tasks for agent retrieval and reranking |
| [AgentSearchBench-Agents](https://huggingface.co/datasets/AgentSearch/AgentSearchBench-Agents) | AgentBase: 9,759 real-world AI agents with metadata |
| [AgentSearchBench-Responses](https://huggingface.co/datasets/AgentSearch/AgentSearchBench-Responses) | 60K+ raw agent execution responses |

---

## Citation

```bibtex
@article{wu2026agentsearchbench,
      title={AgentSearchBench: A Benchmark for AI Agent Search in the Wild}, 
      author={Bin Wu and Arastun Mammadli and Xiaoyu Zhang and Emine Yilmaz},
      year={2026},
      eprint={2604.22436},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
}
```