metadata
license: mit
language:
- en
base_model:
- microsoft/git-base-coco
Given a photo of a face, will describe it. Be careful as it can be unflattering.
Based on the GIT-Base-COCO image to text model and fine-tuned on Face2Text.
How to use:
from transformers import AutoProcessor, AutoModelForCausalLM, AutoTokenizer
import cv2
DEVICE = 'cpu' # cpu or cuda
IMG_PATH = 'face.png'
processor = AutoProcessor.from_pretrained('microsoft/git-base-coco')
model = AutoModelForCausalLM.from_pretrained('mtanti/face-describer')
tokeniser = AutoTokenizer.from_pretrained('microsoft/git-base-coco')
model.eval()
model.to(DEVICE)
img = cv2.imread(IMG_PATH)
tensor_img = processor(
images=[img[:, :, ::-1]],
return_tensors='pt',
)['pixel_values'].to(DEVICE)
desc = tokeniser.decode(
model.generate(pixel_values=tensor_img, max_length=100, repetition_penalty=1.05, do_sample=True)[0, :],
skip_special_tokens=True,
)