mnm-matin/hyperbolic-clip

Hyperbolic CLIP Models (ONNX)

This repository contains ONNX exports of hyperbolic vision-language models for hyperbolic image embeddings.

Available Models

Model	Architecture	Embedding Dim	Size	Path
hycoclip-vit-b	ViT-B/16	513	~350 MB	hycoclip-vit-b/model.onnx
hycoclip-vit-s	ViT-S/16	513	~84 MB	hycoclip-vit-s/model.onnx
meru-vit-b	ViT-B/16	513	~350 MB	meru-vit-b/model.onnx
meru-vit-s	ViT-S/16	513	~84 MB	meru-vit-s/model.onnx

Quick Start

import onnxruntime as ort
import numpy as np
from huggingface_hub import hf_hub_download

# Download a model
onnx_path = hf_hub_download(
    repo_id="mnm-matin/hyperbolic-clip",
    filename="hycoclip-vit-s/model.onnx"  # or other model path
)

# Load and run
session = ort.InferenceSession(onnx_path)
image = np.random.rand(1, 3, 224, 224).astype(np.float32)  # Your preprocessed image
embedding, curvature = session.run(None, {"image": image})

print(f"Embedding shape: {embedding.shape}")  # (1, 513) - hyperboloid format

Model Details

All models output embeddings in Lorentz/Hyperboloid format:

• Output: (t, x₁...xₙ) where t = √(1/c + ‖x‖²)
• Embedding dim: 513 (1 time component + 512 spatial)
• Curvature c is learned and exported as secondary output

Converting to Poincaré Ball

t = embedding[:, 0:1]   # time component
x = embedding[:, 1:]    # spatial components
poincare = x / (t + 1)  # stereographic projection

Usage with HyperView

import hyperview as hv
from huggingface_hub import hf_hub_download

# Download model
model_path = hf_hub_download("mnm-matin/hyperbolic-clip", "hycoclip-vit-s/model.onnx")

# Use with HyperView
ds = hv.Dataset("my_images")
ds.add_images_dir("/path/to/images")
ds.compute_embeddings(onnx_path=model_path)
hv.show(ds)

License

CC-BY-NC-4.0 (Non-commercial use only)

Based on:

• PalAvik/hycoclip
• facebookresearch/meru

Citation

@inproceedings{desai2023hyperbolic,
  title={Hyperbolic Image-Text Representations},
  author={Desai, Karan and Nickel, Maximilian and Rajpurohit, Tanmay and Johnson, Justin and Vedantam, Ramakrishna},
  booktitle={ICML},
  year={2023}
}