README.md

---
license: apache-2.0
language:
- en
pipeline_tag: zero-shot-image-classification
library_name: dghs-imgutils
tags:
- art
- siglip
- dghs-imgutils
- onnx
datasets:
- deepghs/danbooru2024-webp-4Mpixel
---

**WARNING**: Do *not* consider anything in this repo production ready.

### Checkpoints

- **siglip_swinv2_base_2025_02_22_18h56m54s**  
Text encoder trained on top of frozen [SmilingWolf/wd-swinv2-tagger-v3](https://huggingface.co/SmilingWolf/wd-swinv2-tagger-v3), so pretty much SigLIT style. Compatible with existing DeepGHS indexes/embeddings.
- **siglip_swinv2_base_2025_05_02_22h02m36s**  
Based on `siglip_swinv2_base_2025_02_22_18h56m54s`, with unfrozen image encoder. So SigLIP with warm start, I guess.
- **siglip_eva02_base_2025_05_02_21h53m54s**  
A test with a different architecture, trained from scratch using SigLIP.

### Usage Example

See [deepghs/search_image_by_image_or_text](https://huggingface.co/spaces/deepghs/search_image_by_image_or_text) for example usage.

### Compatibility

The checkpoints in this repo have been structured for compatibility with the [dghs-imgutils](https://github.com/deepghs/imgutils) package.

You can run it locally with the following 2 approaches

#### Infer with Code

```shell
pip install dghs-imgutils>=0.17.0
```

```python
from imgutils.generic import siglip_predict

pred = siglip_predict(
    images=[
        'https://huggingface.co/datasets/narugo1992/nzb_files/resolve/main/eshuushuu_51.webp',
    ],
    texts=[
        # short tags
        '1girl',
        '1boy',
        'orange_hair',
        'blue_hair',

        # long text
        '1girl, solo, thighhighs, orange_shirt, twintails, brown_hair, hair_bun, long_hair, double_bun, '
        'zettai_ryouiki, jar, sitting, bow, school_uniform, long_sleeves, smile, pink_bow, skirt, orange_skirt, '
        'very_long_hair, black_thighhighs, orange_dress, miniskirt',
        'food, halo, red_eyes, side_ponytail, skirt, macaron, pink_hair, sailor_collar, holding, '
        'black_sailor_collar, cake, long_hair, drumsticks, black_skirt, pleated_skirt, pink_halo, 1girl, '
        'ahoge, red_neckerchief, chibi, neckerchief, long_sleeves, holding_food, sash, blush, holding_drumsticks, '
        'multiple_views, white_cardigan, looking_at_viewer,'
    ],
    repo_id='deepghs/siglip_beta',
    model_name='smilingwolf/siglip_swinv2_base_2025_02_22_18h56m54s'
)
print(pred)
# [[2.5059912e-02 1.7571157e-04 2.1646977e-03 1.8494057e-04 1.0000000e+00
#   3.8877626e-15]]
```

#### Launch a Gradio Demo

```shell
pip install dghs-imgutils[demo]>=0.17.0
```

```python
from imgutils.generic import SigLIPModel

SigLIPModel(
    repo_id='deepghs/siglip_beta',
).launch_demo(
    default_model_name='smilingwolf/siglip_swinv2_base_2025_02_22_18h56m54s'
)
```