Instructions to use ConvLLaVA/ConvLLaVA-ConvNeXt-1024 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ConvLLaVA/ConvLLaVA-ConvNeXt-1024 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ConvLLaVA/ConvLLaVA-ConvNeXt-1024", dtype="auto") - Notebooks
- Google Colab
- Kaggle
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
ConvNeXt Model Card
Model details
Model type: ConvNeXt is an open-source visual encoder trained by fine-tuning LLM on multimodal caption and instruction-following data. The base model is: laion/CLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soup.
Model date: ConvLLaVA-ConvNeXt-1024 was trained in March 2024.
Paper or resources for more information: https://github.com/alibaba/conv-llava/
Where to send questions or comments about the model: https://github.com/alibaba/conv-llava/issues
Intended use
Primary intended uses: The primary use of ConvLLaVA-ConvNeXt is research on large multimodal models and chatbots.
Paper
arxiv.org/abs/2405.15738
- Downloads last month
- 10