---
title: ModelLens
emoji: 🔭
colorFrom: indigo
colorTo: pink
sdk: gradio
sdk_version: 4.44.0
python_version: '3.11'
app_file: app.py
pinned: false
license: mit
short_description: Finding the Best Model for Your Task from Myriads of Models
---

# ModelLens — Finding the Best Model for Your Task from Myriads of Models

Describe your dataset → pick a task and metric → get a ranked list of HuggingFace
models likely to perform well on it. Backed by the `MLPMetric` (ablation_no_id)
checkpoint trained on the `unified_augmented` corpus, with a candidate pool of
~47k HuggingFace models.

## How it works

1. Your dataset description is embedded with OpenAI `text-embedding-3-small`
   (1536-dim, the same encoder used during training).
2. The MLPMetric scores every candidate model conditioned on the embedding +
   chosen task + chosen metric.
3. We return the top-k, optionally filtered by parameter count, "official
   pretrained only", or "HuggingFace-hosted only".

## Bring your own OpenAI key

This Space does **not** ship with a baked-in OpenAI key. Paste your own
`sk-...` key into the "OpenAI API key" field — it is sent directly to OpenAI
for that single request and is **not stored, logged, or reused** by this Space.
A query costs roughly **$0.000001** on your account (about a millionth of a
dollar).

If you don't have a key yet: https://platform.openai.com/api-keys

## Files in this Space

```
app.py              Gradio entry point
recommend.py        Recommender (loads checkpoint + model pool, embeds dataset desc)
inference_lib.py    Self-contained MLPMetric implementation (no module/ tree needed)
build_model_pool.py Offline helper to (re)build assets/model_pool.npz
requirements.txt    Pinned deps
assets/
  model_pool.npz    Pre-computed candidate pool (47k models, size+family ids, popularity, HF urls)
checkpoint/
  MLPMetric.pt      ~37 MB trained weights
  args.json         Training-time hyperparameters (model dims, num_*)
data/
  task2id.json      Task vocab
  metric2id.json    Metric vocab
```

The Space looks for the checkpoint at `checkpoint/MLPMetric.pt` and the data
JSONs at `data/`. Override with env vars `MODEL_CKPT`, `MODEL_ARGS`, `DATA_DIR`,
`POOL_PATH` if you lay things out differently.

## Running locally

```bash
cd web
pip install -r requirements.txt
# either set OPENAI_API_KEY in env, or paste it into the UI at runtime
python app.py
# open http://localhost:7860
```

## Rebuilding the model pool

When you bump the candidate set (e.g. add new HF models to `model2id.json` /
`model_profile.json`):

```bash
python web/build_model_pool.py \
    --data-dir data/unified_augmented \
    --args     checkpoint/mlp/unified_augmented/ablation_no_model_id_no_dataset_id/args.json \
    --out      web/assets/model_pool.npz \
    --min-popularity 0
```