Update README.md
Browse files
README.md
CHANGED
|
@@ -1,185 +1,132 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
--
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
0.5pt. A 203M open model above a closed commercial system and a 1.24B model.
|
| 88 |
-
</p>
|
| 89 |
-
</section>
|
| 90 |
-
|
| 91 |
-
<section>
|
| 92 |
-
<h2>The MODA family</h2>
|
| 93 |
-
<table>
|
| 94 |
-
<tr><th>model</th><th>what it is</th><th>availability</th></tr>
|
| 95 |
-
<tr>
|
| 96 |
-
<td><a href="https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m"><b>MODA</b></a> (203M)</td>
|
| 97 |
-
<td>zero-new-parameter serving recipe over frozen FashionSigLIP; 4/6 statistically significant full-corpus wins over its own base model</td>
|
| 98 |
-
<td>open source + open weights</td>
|
| 99 |
-
</tr>
|
| 100 |
-
<tr>
|
| 101 |
-
<td><a href="https://huggingface.co/HopitAI/moda-pro-lite"><b>MODA Pro Lite</b></a> (213M)</td>
|
| 102 |
-
<td>trained encoder (verified fashion-vocab build); beats MODA on catalog search as a plain bi-encoder β no recipe required</td>
|
| 103 |
-
<td>open weights</td>
|
| 104 |
-
</tr>
|
| 105 |
-
<tr>
|
| 106 |
-
<td><b>MODA Pro</b></td>
|
| 107 |
-
<td>our hosted retrieval system β rank 1 or 2 on 9 of 10 benchmark cells at single-model query latency and cost</td>
|
| 108 |
-
<td>closed Β· <a href="https://hopit.ai">hosted</a></td>
|
| 109 |
-
</tr>
|
| 110 |
-
<tr>
|
| 111 |
-
<td><a href="https://huggingface.co/HopitAI/moda-fashion-distilled"><b>MODA-SigLIP-Distilled</b></a> (203M)</td>
|
| 112 |
-
<td>image-to-image specialist β #1 open model on LookBench</td>
|
| 113 |
-
<td>open weights (+ <a href="https://huggingface.co/HopitAI/moda-fashion-matryoshka">matryoshka</a>, <a href="https://huggingface.co/HopitAI/moda-fashion-distilled-512d">512d</a>, <a href="https://huggingface.co/HopitAI/moda-fashion-vision-fp16">fp16-vision</a>, <a href="https://huggingface.co/HopitAI/moda-fashion-crossdomain">cross-domain</a> variants)</td>
|
| 114 |
-
</tr>
|
| 115 |
-
</table>
|
| 116 |
-
</section>
|
| 117 |
-
|
| 118 |
-
<section>
|
| 119 |
-
<h2>Why trust these numbers</h2>
|
| 120 |
-
<ul>
|
| 121 |
-
<li><b>Full corpus only</b> β no subsampled galleries; screening runs are never mixed with full-corpus rows.</li>
|
| 122 |
-
<li><b>One harness</b> β every model, ours and competitors', runs identical preprocessing and protocol.</li>
|
| 123 |
-
<li><b>Losses shown</b> β every model card links the cells it loses; paired-bootstrap CIs where significance is claimed.</li>
|
| 124 |
-
<li><b>Reproducible</b> β code, harness, and per-cell receipts in the <a href="https://github.com/hopit-ai/Moda">public repo</a>.</li>
|
| 125 |
-
</ul>
|
| 126 |
-
</section>
|
| 127 |
-
|
| 128 |
-
<section>
|
| 129 |
-
<h2>Which model should I use?</h2>
|
| 130 |
-
<table>
|
| 131 |
-
<tr><th>your workload</th><th>use</th><th>why</th></tr>
|
| 132 |
-
<tr><td>catalog / title search</td><td><a href="https://huggingface.co/HopitAI/moda-pro-lite">moda-pro-lite</a></td><td>strongest β€250M on catalog benchmarks; plain bi-encoder, any vector DB</td></tr>
|
| 133 |
-
<tr><td>caption-style / exact-item search</td><td><a href="https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m">MODA</a></td><td>leads the class on Fashion200K, In-Shop, Multimodal</td></tr>
|
| 134 |
-
<tr><td>best text search, zero integration</td><td><a href="https://hopit.ai">MODA Pro</a> (hosted)</td><td>rank 1β2 on 9 of 10 benchmarks, no bad benchmark</td></tr>
|
| 135 |
-
<tr><td>visually similar products (image)</td><td><a href="https://huggingface.co/HopitAI/moda-fashion-distilled">moda-fashion-distilled</a></td><td>#1 open model on LookBench</td></tr>
|
| 136 |
-
<tr><td>small index / edge</td><td><a href="https://huggingface.co/HopitAI/moda-fashion-matryoshka">matryoshka</a> @256d Β· <a href="https://huggingface.co/HopitAI/moda-fashion-vision-fp16">fp16</a></td><td>3Γ smaller index at no loss Β· 186 MB vision tower</td></tr>
|
| 137 |
-
</table>
|
| 138 |
-
<p>All models serve on CPU. Each card links the benchmarks it loses, too.</p>
|
| 139 |
-
</section>
|
| 140 |
-
|
| 141 |
-
<section>
|
| 142 |
-
<h2>Quick start</h2>
|
| 143 |
-
|
| 144 |
-
Text to image (plain bi-encoder, works with any vector DB):
|
| 145 |
|
| 146 |
```python
|
| 147 |
-
import open_clip
|
| 148 |
|
| 149 |
model, _, preprocess = open_clip.create_model_and_transforms("hf-hub:HopitAI/moda-pro-lite")
|
| 150 |
tokenizer = open_clip.get_tokenizer("hf-hub:HopitAI/moda-pro-lite")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 151 |
```
|
| 152 |
|
| 153 |
-
|
|
|
|
| 154 |
|
| 155 |
-
|
| 156 |
-
import open_clip
|
| 157 |
|
| 158 |
-
|
| 159 |
-
|
|
|
|
| 160 |
|
| 161 |
-
|
| 162 |
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 166 |
|
| 167 |
-
|
| 168 |
-
from moda_fashionsiglip_multiview import ModaFashionSigLIP
|
| 169 |
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
<ul>
|
| 179 |
-
<li>Search a fashion catalog with a natural-language query.</li>
|
| 180 |
-
<li>Find visually similar products in a fashion catalog.</li>
|
| 181 |
-
<li>Match street-style looks to shoppable items.</li>
|
| 182 |
-
<li>Deduplicate product images across marketplaces.</li>
|
| 183 |
-
<li>Build embedding indexes for ecommerce search and recommendations.</li>
|
| 184 |
-
</ul>
|
| 185 |
-
</section>
|
|
|
|
| 1 |
+
# Hopit AI
|
| 2 |
+
|
| 3 |
+
Fashion AI infrastructure β retrieval, trend intelligence and catalogue models for fashion
|
| 4 |
+
commerce, measured in public against open benchmarks.
|
| 5 |
+
|
| 6 |
+
## The MODA family
|
| 7 |
+
|
| 8 |
+
| model | what it is | availability |
|
| 9 |
+
|---|---|---|
|
| 10 |
+
| **[MODA Pro Lite+](https://huggingface.co/HopitAI/moda-pro-lite)** (213M) | Trained fashion encoder with a calibrated serving recipe. Leads the β€250M class on catalogue and title search. | open weights |
|
| 11 |
+
| **[MODA Duo](https://huggingface.co/HopitAI/moda-duo)** | Routes each query to the constituent that suits its shape. Beats either alone on a mixed workload, at one encoder per search. | open recipe Β· open weights |
|
| 12 |
+
| **[MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m)** (203M) | Zero-new-parameter serving recipe over frozen FashionSigLIP. Strongest of the open models on long descriptions. | open source Β· open weights |
|
| 13 |
+
| **MODA Pro** | Hosted retrieval system. Rank 1 or 2 on five of six full-corpus benchmarks. | closed Β· [hopit.ai](https://hopit.ai) |
|
| 14 |
+
| **[MODA-SigLIP-Distilled](https://huggingface.co/HopitAI/moda-fashion-distilled)** (203M) | Image-to-image specialist. #1 open model on LookBench. | open weights |
|
| 15 |
+
|
| 16 |
+
## Text-to-image retrieval
|
| 17 |
+
|
| 18 |
+
MAP@10, full corpus, all ground-truth queries, one evaluator (`pytrec_eval map_cut.10`).
|
| 19 |
+
Every system in each size class is shown β ours and everyone else's, wins and losses.
|
| 20 |
+
Columns ordered by number of first places, then by top-two finishes.
|
| 21 |
+
|
| 22 |
+
### Models at 250M parameters or fewer
|
| 23 |
+
|
| 24 |
+
| benchmark | [MODA Pro Lite+](https://huggingface.co/HopitAI/moda-pro-lite)<br>213M | [MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m)<br>203M | [FashionSigLIP](https://huggingface.co/Marqo/marqo-fashionSigLIP)<br>203M | [MODA Pro Lite](https://huggingface.co/HopitAI/moda-pro-lite)<br>213M |
|
| 25 |
+
|---|---:|---:|---:|---:|
|
| 26 |
+
| KAGL | **0.3201** | 0.2887 | 0.2769 | 0.3055 |
|
| 27 |
+
| Polyvore | **0.4049** | 0.3726 | 0.3664 | 0.3952 |
|
| 28 |
+
| Atlas | **0.1904** | 0.1862 | 0.1826 | 0.1814 |
|
| 29 |
+
| Fashion200K | 0.1846 | **0.1946** | 0.1858 | 0.1758 |
|
| 30 |
+
| DeepFashion In-Shop | 0.1026 | **0.1642** | 0.1586 | 0.0930 |
|
| 31 |
+
| DeepFashion Multimodal | 0.0133 | 0.0147 | **0.0148** | 0.0118 |
|
| 32 |
+
|
| 33 |
+
### All systems
|
| 34 |
+
|
| 35 |
+
| benchmark | [SO400M](https://huggingface.co/timm/ViT-SO400M-14-SigLIP-384)<br>878M | MODA Pro<br>hosted | [MODA Duo](https://huggingface.co/HopitAI/moda-duo)<br>2 indexes | [MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m)<br>203M | [FashionSigLIP](https://huggingface.co/Marqo/marqo-fashionSigLIP)<br>203M | [MODA Pro Lite+](https://huggingface.co/HopitAI/moda-pro-lite)<br>213M | [MODA Pro Lite](https://huggingface.co/HopitAI/moda-pro-lite)<br>213M | [ZooClaw](https://huggingface.co/srpone/zooclaw-fashionsiglip2)<br>375M |
|
| 36 |
+
|---|---:|---:|---:|---:|---:|---:|---:|---:|
|
| 37 |
+
| KAGL | **0.3370** | 0.3221 | 0.3201 | 0.2887 | 0.2769 | 0.3201 | 0.3055 | 0.2951 |
|
| 38 |
+
| Polyvore | **0.4378** | 0.4106 | 0.4049 | 0.3726 | 0.3664 | 0.4049 | 0.3952 | 0.3804 |
|
| 39 |
+
| Atlas | **0.2309** | 0.2009 | 0.1904 | 0.1862 | 0.1826 | 0.1904 | 0.1814 | 0.1583 |
|
| 40 |
+
| Fashion200K | 0.1353 | **0.2065** | 0.1866 | 0.1946 | 0.1858 | 0.1846 | 0.1758 | 0.1775 |
|
| 41 |
+
| DeepFashion In-Shop | **0.1694** | 0.1663 | 0.1640 | 0.1642 | 0.1586 | 0.1026 | 0.0930 | 0.1024 |
|
| 42 |
+
| DeepFashion Multimodal | 0.0079 | 0.0137 | **0.0159** | 0.0147 | 0.0148 | 0.0133 | 0.0118 | 0.0099 |
|
| 43 |
+
|
| 44 |
+
## Image-to-image retrieval
|
| 45 |
+
|
| 46 |
+
Finding visually similar products from a photo, on LookBench (2,345 queries,
|
| 47 |
+
[Gao et al., 2026](https://arxiv.org/abs/2601.14706)). Every system measured through one
|
| 48 |
+
harness; FashionSigLIP was reproduced here before any comparison was made.
|
| 49 |
+
|
| 50 |
+
| model | params | dim | Fine R@1 | nDCG@5 |
|
| 51 |
+
|---|---:|---:|---:|---:|
|
| 52 |
+
| [moda-fashion-distilled](https://huggingface.co/HopitAI/moda-fashion-distilled) | 203M | 768 | **67.63** | 53.85 |
|
| 53 |
+
| [moda-fashion-matryoshka](https://huggingface.co/HopitAI/moda-fashion-matryoshka) @256d | 203M | 256 | 67.42 | **57.48** |
|
| 54 |
+
| GR-Pro (SerendipityOne, closed) | n/a | 1024 | 67.38 | β |
|
| 55 |
+
| [moda-fashion-crossdomain](https://huggingface.co/HopitAI/moda-fashion-crossdomain) | 203M | 768 | 66.52 | 52.46 |
|
| 56 |
+
| [Tianmu-MERE](https://huggingface.co/TianmuLab/Tianmu-MERE) (Kuaishou) | 1.24B | 512 | 65.99 | 52.37 |
|
| 57 |
+
| [GR-Lite](https://huggingface.co/srpone/gr-lite) (SerendipityOne) | 303M | 1024 | 65.71 | β |
|
| 58 |
+
| [FashionSigLIP](https://huggingface.co/Marqo/marqo-fashionSigLIP) (our reproduction) | 203M | 768 | 63.84 | 49.63 |
|
| 59 |
+
| [FashionSigLIP](https://huggingface.co/Marqo/marqo-fashionSigLIP) (published) | 203M | 768 | 62.77 | 49.44 |
|
| 60 |
+
|
| 61 |
+
**Tianmu-MERE**, the top open model on the LookBench multi-attribute leaderboard, was
|
| 62 |
+
evaluated through this same harness and reproduces its own published subset scores within
|
| 63 |
+
0.5 points β which independently validates the harness. moda-fashion-distilled leads it on
|
| 64 |
+
all four subsets, on Fine R@1 and nDCG@5 alike, at 6Γ fewer parameters.
|
| 65 |
+
|
| 66 |
+
The configuration above was chosen from eight candidates ranked on LookBench itself, so the
|
| 67 |
+
headline carries the upward bias of a maximum over eight. It was independently reproduced
|
| 68 |
+
in-harness at 67.59, and the runner-up configurations land within 0.3 points.
|
| 69 |
+
|
| 70 |
+
## Which model should I use?
|
| 71 |
+
|
| 72 |
+
| your workload | use | why |
|
| 73 |
+
|---|---|---|
|
| 74 |
+
| Catalogue / title search β `red floral maxi dress` | [moda-pro-lite](https://huggingface.co/HopitAI/moda-pro-lite) | leads the β€250M class on KAGL, Polyvore and Atlas |
|
| 75 |
+
| Long descriptions, exact-item search | [moda-duo](https://huggingface.co/HopitAI/moda-duo) | routes long queries to the description-strong constituent |
|
| 76 |
+
| A mixture of both | [moda-duo](https://huggingface.co/HopitAI/moda-duo) | beats either constituent alone on a mixed workload |
|
| 77 |
+
| Agentic and multi-step search | MODA Pro ([hosted](https://hopit.ai)) | one API, no indexes to build or operate |
|
| 78 |
+
| Visually similar products from a photo | [moda-fashion-distilled](https://huggingface.co/HopitAI/moda-fashion-distilled) | #1 open model on LookBench |
|
| 79 |
+
| Same, but index size matters | [moda-fashion-matryoshka](https://huggingface.co/HopitAI/moda-fashion-matryoshka) | truncatable embeddings, 64β768d |
|
| 80 |
+
| Edge / mobile deployment | [moda-fashion-vision-fp16](https://huggingface.co/HopitAI/moda-fashion-vision-fp16) | 186 MB vision tower |
|
| 81 |
+
|
| 82 |
+
## Quick start
|
| 83 |
+
|
| 84 |
+
```bash
|
| 85 |
+
pip install open_clip_torch pillow
|
| 86 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
```python
|
| 89 |
+
import open_clip, torch
|
| 90 |
|
| 91 |
model, _, preprocess = open_clip.create_model_and_transforms("hf-hub:HopitAI/moda-pro-lite")
|
| 92 |
tokenizer = open_clip.get_tokenizer("hf-hub:HopitAI/moda-pro-lite")
|
| 93 |
+
model.eval()
|
| 94 |
+
|
| 95 |
+
with torch.no_grad():
|
| 96 |
+
img = torch.nn.functional.normalize(model.encode_image(preprocess(image).unsqueeze(0)), dim=-1)
|
| 97 |
+
txt = torch.nn.functional.normalize(model.encode_text(tokenizer(["red floral maxi dress"])), dim=-1)
|
| 98 |
+
score = (txt @ img.T).item()
|
| 99 |
```
|
| 100 |
|
| 101 |
+
Each model repository ships a runnable script β `serving_ann.py` for the retrieval systems
|
| 102 |
+
(including the ANN index), `inference.py` for the image-to-image checkpoints.
|
| 103 |
|
| 104 |
+
## Collections
|
|
|
|
| 105 |
|
| 106 |
+
- [Text-to-image retrieval](https://huggingface.co/collections/HopitAI/text-to-image-retrieval-6a7ad68d07fdebe00543a707)
|
| 107 |
+
- [Image-to-image retrieval](https://huggingface.co/collections/HopitAI/image-to-image-retrieval-6a7ad6aa12ecf6f5daec40f8)
|
| 108 |
+
- [Compact and edge deployment](https://huggingface.co/collections/HopitAI/compact-and-edge-deployment-6a7ad6ab5f5d8ac50882038d)
|
| 109 |
|
| 110 |
+
## Models
|
| 111 |
|
| 112 |
+
| model | task | params | best for |
|
| 113 |
+
|---|---|---|---|
|
| 114 |
+
| [moda-duo](https://huggingface.co/HopitAI/moda-duo) | text β image | recipe over two encoders | mixed workloads; routes per query, one encoder per search |
|
| 115 |
+
| [moda-pro-lite](https://huggingface.co/HopitAI/moda-pro-lite) | text β image | 213M | catalogue and title search; ships its serving recipe |
|
| 116 |
+
| [moda-fashionsiglip-multiview-203m](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m) | text β image | 203M | long descriptions and exact-item retrieval |
|
| 117 |
+
| [moda-fashion-distilled](https://huggingface.co/HopitAI/moda-fashion-distilled) | image β image | 203M | best overall visual similarity |
|
| 118 |
+
| [moda-fashion-matryoshka](https://huggingface.co/HopitAI/moda-fashion-matryoshka) | image β image | 203M | truncatable embeddings, 64β768d |
|
| 119 |
+
| [moda-fashion-distilled-512d](https://huggingface.co/HopitAI/moda-fashion-distilled-512d) | image β image | 203M | compact indexes |
|
| 120 |
+
| [moda-fashion-crossdomain](https://huggingface.co/HopitAI/moda-fashion-crossdomain) | image β image | 203M | street-to-shop |
|
| 121 |
+
| [moda-fashion-vision-fp16](https://huggingface.co/HopitAI/moda-fashion-vision-fp16) | image β image | 93M vision | edge and mobile, 186 MB |
|
| 122 |
|
| 123 |
+
## How these numbers are measured
|
|
|
|
| 124 |
|
| 125 |
+
Full corpus, no subsampling β every query scored against the entire benchmark corpus
|
| 126 |
+
(42,537β201,624 items). One evaluator for every system, including competitors. Serving
|
| 127 |
+
recipes are selected on held-out development data and frozen in writing before any benchmark
|
| 128 |
+
is scored. Losses are shown, not hidden.
|
| 129 |
+
|
| 130 |
+
Receipts, per-query results and confidence intervals: **[github.com/hopit-ai/Moda](https://github.com/hopit-ai/Moda)**
|
| 131 |
+
Β· Benchmarks: **[hopit-ai.github.io/Moda](https://hopit-ai.github.io/Moda/)**
|
| 132 |
+
Β· Hosted retrieval and agentic search: **[hopit.ai](https://hopit.ai)**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|