Remove unneeded spaces before texts in image-text setups 211fa5c
Tom Aarsen commited on
How to use intfloat/mmE5-mllama-11b-instruct with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("zero-shot-image-classification", model="intfloat/mmE5-mllama-11b-instruct")
pipe(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png",
candidate_labels=["animals", "humans", "landscape"],
) # Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText
processor = AutoProcessor.from_pretrained("intfloat/mmE5-mllama-11b-instruct")
model = AutoModelForImageTextToText.from_pretrained("intfloat/mmE5-mllama-11b-instruct")How to use intfloat/mmE5-mllama-11b-instruct with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("intfloat/mmE5-mllama-11b-instruct")
sentences = [
"The weather is lovely today.",
"It's so sunny outside!",
"He drove to the stadium."
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]