LinkSoul/Chinese-LLaVA-Vision-Instructions
Viewer • Updated • 1.82M • 254 • 71
How to use amitha/mllava-llama2-en-zh with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("visual-question-answering", model="amitha/mllava-llama2-en-zh", trust_remote_code=True) # Load model directly
from transformers import AutoModelForVisualQuestionAnswering
model = AutoModelForVisualQuestionAnswering.from_pretrained("amitha/mllava-llama2-en-zh", trust_remote_code=True, dtype="auto")The bilingual English/Chinese Llama2-7B-Chat VLM trained via LORA for https://arxiv.org/abs/2406.11665.
The Chinese half of the training data used for multimodal alignment and visual instruction tuning is sampled from here.