| | --- |
| | library_name: transformers |
| | license: apache-2.0 |
| | pipeline_tag: image-to-text |
| | --- |
| | |
| | # rmfg |
| |
|
| | <!-- Provide a quick summary of what the model is/does. --> |
| | <img src="https://i.pinimg.com/736x/7e/46/a6/7e46a6881623dfd3e1a2a5a2ae692374.jpg" width="300"> |
| |
|
| |
|
| |
|
| | ## Example |
| |
|
| | **Image** |
| | <img src="https://media-cldnry.s-nbcnews.com/image/upload/t_fit-760w,f_auto,q_auto:best/rockcms/2023-12/231202-elon-musk-mjf-1715-fc0be2.jpg" width="300"> |
| | **Output** |
| | > A man in a black cowboy hat and sunglasses stands in front of a white car, holding a microphone and speaking into it. |
| |
|
| | ----------------------------------------------------------------------------------- |
| |
|
| | - underfit, doesn't perform well |
| | - this marks the beginning of my tiny vision language model series, with this model serving as a prelude to what's to come in the next few days. |
| |
|
| | ``` |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | from PIL import Image |
| | |
| | model_id = "aloobun/rmfg" |
| | model = AutoModelForCausalLM.from_pretrained( |
| | model_id, trust_remote_code=True |
| | ) |
| | tokenizer = AutoTokenizer.from_pretrained(model_id) |
| | |
| | image = Image.open('692374.jpg') |
| | enc_image = model.encode_image(image) |
| | print(model.answer_question(enc_image, "Describe this image.", tokenizer)) |
| | ``` |
