Instructions to use llava-hf/LLaVA-NeXT-Video-7B-hf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use llava-hf/LLaVA-NeXT-Video-7B-hf with Transformers:
# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("llava-hf/LLaVA-NeXT-Video-7B-hf") model = AutoModelForMultimodalLM.from_pretrained("llava-hf/LLaVA-NeXT-Video-7B-hf", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
#13
by ervin0307 - opened
README.md
CHANGED
|
@@ -16,7 +16,7 @@ Disclaimer: The team releasing LLaVa-NeXT-Video did not write a model card for t
|
|
| 16 |
## 📄 Model details
|
| 17 |
|
| 18 |
**Model type:**
|
| 19 |
-
LLaVA-Next-Video is an open-source chatbot trained by fine-tuning LLM on multimodal instruction-following data. The model is buit on top of LLaVa-NeXT by tuning on a mix of video and image data to
|
| 20 |
The model is a current SOTA among open-source models on [VideoMME bench](https://arxiv.org/abs/2405.21075).
|
| 21 |
Base LLM: [lmsys/vicuna-7b-v1.5](https://huggingface.co/lmsys/vicuna-13b-v1.5)
|
| 22 |
|
|
|
|
| 16 |
## 📄 Model details
|
| 17 |
|
| 18 |
**Model type:**
|
| 19 |
+
LLaVA-Next-Video is an open-source chatbot trained by fine-tuning LLM on multimodal instruction-following data. The model is buit on top of LLaVa-NeXT by tuning on a mix of video and image data to achieve better video understanding capabilities. The videos were sampled uniformly to be 32 frames per clip.
|
| 20 |
The model is a current SOTA among open-source models on [VideoMME bench](https://arxiv.org/abs/2405.21075).
|
| 21 |
Base LLM: [lmsys/vicuna-7b-v1.5](https://huggingface.co/lmsys/vicuna-13b-v1.5)
|
| 22 |
|