metadata
language: multilingual
license: other
datasets:
- jinaai/jina-vdr
pipeline_tag: feature-extraction
tags:
- embeddings
- multilingual-embeddings
- multimodal-embeddings
- text-to-image
- sentence-transformers
- sentence-similarity
- visual-document-retrieval
Custom Embedding Model
This is a custom embedding model based on the Jina Embeddings V4 architecture, specially adapted for embedding tasks involving text, images, and visual documents.
Model Description
The model supports:
- Multimodal Embeddings: Generate unified embeddings for text and images
- Multilingual Support: Works across 30+ languages
- Task-specific Modes: Optimized for retrieval, text-matching, and code tasks
- Flexible Dimensions: Dense embeddings that can be truncated with minimal performance loss
Usage
Text Embeddings
from custom_st import Transformer
# Initialize the model
model = Transformer(
model_name_or_path="path/to/model",
model_args={"default_task": "retrieval", "trust_remote_code": True},
trust_remote_code=True
)
# Encode text
texts = ["Your text here", "Another text example"]
features = model.tokenize(texts)
outputs = model.forward(features, task="retrieval")
embeddings = outputs["sentence_embedding"]
Image Embeddings
from PIL import Image
from custom_st import Transformer
# Initialize the model
model = Transformer(
model_name_or_path="path/to/model",
model_args={"default_task": "retrieval", "trust_remote_code": True},
trust_remote_code=True
)
# Load images
images = [Image.open("image1.jpg"), Image.open("image2.jpg")]
# Or use URLs
image_urls = ["http://example.com/image1.jpg", "http://example.com/image2.jpg"]
# Encode images
features = model.tokenize(images) # or model.tokenize(image_urls)
outputs = model.forward(features, task="retrieval")
embeddings = outputs["sentence_embedding"]
Requirements
- Python 3.8+
- PyTorch 2.0+
- Transformers 4.30+
- PEFT 0.4+
- Pillow 9.0+
License
This model is available under the same terms as the original model it's based on. Please refer to the license information in the repository for details.