Instructions to use fassabilf/sea-clip-tiny-init with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- OpenCLIP
How to use fassabilf/sea-clip-tiny-init with OpenCLIP:
import open_clip model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:fassabilf/sea-clip-tiny-init') tokenizer = open_clip.get_tokenizer('hf-hub:fassabilf/sea-clip-tiny-init') - Notebooks
- Google Colab
- Kaggle
SEA-CLIP-Tiny release (ACCV 2026)
Browse files- README.md +59 -0
- open_clip_config.json +32 -0
- open_clip_pytorch_model.bin +3 -0
README.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
library_name: open_clip
|
| 4 |
+
pipeline_tag: zero-shot-image-classification
|
| 5 |
+
tags:
|
| 6 |
+
- clip
|
| 7 |
+
- open_clip
|
| 8 |
+
- multilingual
|
| 9 |
+
- southeast-asia
|
| 10 |
+
language:
|
| 11 |
+
- en
|
| 12 |
+
- id
|
| 13 |
+
- jv
|
| 14 |
+
- su
|
| 15 |
+
- ms
|
| 16 |
+
- th
|
| 17 |
+
- vi
|
| 18 |
+
- my
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# SEA-CLIP-Tiny — English initialisation (e0)
|
| 22 |
+
|
| 23 |
+
The English CLIP-KD student checkpoint used to initialise every SEA-CLIP-Tiny run in
|
| 24 |
+
[SEA-CLIP-Tiny (ACCV 2026)](https://huggingface.co/collections/fassabilf/sea-clip-tiny-accv-2026-6ab15f4d6cf7d50e98d5117c). It is the *starting point*, not a trained SEA model:
|
| 25 |
+
no Southeast Asian data has been seen at this point.
|
| 26 |
+
|
| 27 |
+
## Usage
|
| 28 |
+
|
| 29 |
+
```python
|
| 30 |
+
import open_clip
|
| 31 |
+
|
| 32 |
+
model, _, preprocess = open_clip.create_model_and_transforms('hf-hub:fassabilf/sea-clip-tiny-init')
|
| 33 |
+
tokenizer = open_clip.get_tokenizer('hf-hub:fassabilf/sea-clip-tiny-init')
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
## Model
|
| 37 |
+
|
| 38 |
+
| | |
|
| 39 |
+
|---|---|
|
| 40 |
+
| Architecture | ViT-T/16 vision tower + 12-layer / 384-wide text tower, embed dim 512 |
|
| 41 |
+
| Tokenizer | CLIP BPE, vocab 49408, context length 77 |
|
| 42 |
+
| Parameters | 46.11M (5.62M vision + 40.49M text) |
|
| 43 |
+
| Training data | English CLIP-KD pretraining (no SEA data) |
|
| 44 |
+
| Teacher | — |
|
| 45 |
+
|
| 46 |
+
## Reproduce
|
| 47 |
+
|
| 48 |
+
Training and evaluation code: [https://github.com/fassabilf/sea-clip-tiny](https://github.com/fassabilf/sea-clip-tiny).
|
| 49 |
+
The exact training configuration of this checkpoint is in `params.txt` in this repo.
|
| 50 |
+
|
| 51 |
+
## Citation
|
| 52 |
+
|
| 53 |
+
```bibtex
|
| 54 |
+
@inproceedings{seacliptiny2026,
|
| 55 |
+
title = {SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languages},
|
| 56 |
+
booktitle = {Asian Conference on Computer Vision (ACCV)},
|
| 57 |
+
year = {2026}
|
| 58 |
+
}
|
| 59 |
+
```
|
open_clip_config.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_cfg": {
|
| 3 |
+
"embed_dim": 512,
|
| 4 |
+
"vision_cfg": {
|
| 5 |
+
"image_size": 224,
|
| 6 |
+
"layers": 12,
|
| 7 |
+
"width": 192,
|
| 8 |
+
"patch_size": 16
|
| 9 |
+
},
|
| 10 |
+
"text_cfg": {
|
| 11 |
+
"context_length": 77,
|
| 12 |
+
"vocab_size": 49408,
|
| 13 |
+
"width": 384,
|
| 14 |
+
"heads": 6,
|
| 15 |
+
"layers": 12
|
| 16 |
+
}
|
| 17 |
+
},
|
| 18 |
+
"preprocess_cfg": {
|
| 19 |
+
"mean": [
|
| 20 |
+
0.48145466,
|
| 21 |
+
0.4578275,
|
| 22 |
+
0.40821073
|
| 23 |
+
],
|
| 24 |
+
"std": [
|
| 25 |
+
0.26862954,
|
| 26 |
+
0.26130258,
|
| 27 |
+
0.27577711
|
| 28 |
+
],
|
| 29 |
+
"interpolation": "bicubic",
|
| 30 |
+
"resize_mode": "shortest"
|
| 31 |
+
}
|
| 32 |
+
}
|
open_clip_pytorch_model.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7e44bb81e1465b5ec2272f63ed30b2567a273c3470d300b41992731be476a7b5
|
| 3 |
+
size 184564982
|