liuhaotian/LLaVA-Pretrain
Preview • Updated • 2.37k • 218
How to use xtuner/llava-v1.5-7b-xtuner-pretrain with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("visual-question-answering", model="xtuner/llava-v1.5-7b-xtuner-pretrain") # Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("xtuner/llava-v1.5-7b-xtuner-pretrain", dtype="auto")llava-v1.5-7b-xtuner-pretrain is a LLaVA projector pretrained from Vicuna-7B-v1.5 and CLIP-ViT-Large-patch14-336 on LLaVA-Pretrain dataset by XTuner.
The fine-tuned LLaVA model can be found on xtuner/llava-v1.5-7b-xtuner.
@misc{2023xtuner,
title={XTuner: A Toolkit for Efficiently Fine-tuning LLM},
author={XTuner Contributors},
howpublished = {\url{https://github.com/InternLM/xtuner}},
year={2023}
}