---
base_model: BLIP-2
library_name: peft
---

# Model Card for BLIP-2: Bootstrapping Language-Image Pre-training

BLIP-2 is a unified vision-language model designed for tasks such as image captioning, visual question answering, and more. It employs a novel pre-training strategy that leverages frozen pre-trained image encoders and large language models (LLMs) to efficiently bridge the modality gap between vision and language. 

## Model Details

### Model Description

BLIP-2 (Bootstrapping Language-Image Pre-training) introduces a lightweight Querying Transformer (Q-Former) that connects a frozen image encoder with a frozen LLM. This architecture enables effective vision-language understanding and generation without the need for end-to-end training of large-scale models. The model is capable of zero-shot image-to-text generation and can follow natural language instructions. 

- **Developed by:** Salesforce AI Research
- **Funded by:** Salesforce
- **Shared by:** Official BLIP-2 repository
- **Model type:** Vision-language model
- **Language(s):** English
- **Finetuned from model:** BLIP-2 base pretrained on COCO dataset

### Model Sources

- **Repository:** [BLIP-2 Official GitHub](https://github.com/salesforce/LAVIS/tree/main/projects/blip2)
- **Paper:** [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://arxiv.org/abs/2301.12597)
- **Dataset:** [Recipes Dataset](https://www.kaggle.com/datasets/pes12017000148/food-ingredients-and-recipe-dataset-with-images)